Consider the Children: Unintended Consequences of the Jamaican Primary Education Accountability System

The Harvard community has made this article openly available. Please share how this access benefits you. Your story matters

Citation Miller, Dawn E. E. 2017. Consider the Children: Unintended Consequences of the Jamaican Primary Education Accountability System. Doctoral dissertation, Harvard Graduate School of Education.

Citable link http://nrs.harvard.edu/urn-3:HUL.InstRepos:33052860

Terms of Use This article was downloaded from Harvard University’s DASH repository, and is made available under the terms and conditions applicable to Other Posted Material, as set forth at http:// nrs.harvard.edu/urn-3:HUL.InstRepos:dash.current.terms-of- use#LAA

Consider the Children: Unintended Consequences of the Jamaican Primary Education Accountability System

Dawn Elizabeth Elaine Miller

Terrence Tivnan Pamela A. Mason John B. Willett

A Thesis Presented to the Faculty of the Graduate School of Education of Harvard University in partial fulfillment of the Requirements for the Degree of Doctor of Education

2017

i

@ 2017 Dawn Elizabeth Elaine Miller All Rights Reserved

ii

To my Father, The Most High God, Yahuah

iii

Acknowledgements

The successful completion of this doctoral journey would not have been possible without the contribution and support of others.

First, I want to acknowledge and thank the Jamaican children, parents, teachers, and principals who kindly allowed me to visit your schools and speak with you. I also want to thank the Ministry of Education for allowing me to access these schools and for giving me the data I used for my statistical analyses.

Second, I want to recognize my doctoral committee. What can I say about the people who guided me throughout this journey? Who supported me beyond the call of duty? Who advocated for me, when it was necessary, and ensured that my adventurous journey to the finish line was as smooth as possible? There is no way for me to adequately express my gratitude and love for the members of my doctoral committee: Terrence Tivnan, Pamela Mason, and John Willett. The three of you banded together as Team Dawn and gave me everything I needed to finish this race. You not only cared for my academic wellbeing but my emotional and physical as well. I was empowered knowing that I had my “three docs” with and behind me all the way. Thank you! I especially want to thank Doc T for your unwavering support of my work and me. Your selfless commitment and dedication to coaching and assisting me as I completed this degree goes beyond description. From the bottom of my heart, thank you, Doc T. I love you all dearly.

Third, to my best friend and the sister of my heart, Emily, thank you for being there. We’ve known each other since we were toddlers and, in all these years, you have never let me down. And, true to form, you were with me every step of this journey - sometimes encouraging, sometimes consoling, sometimes reprimanding but always supporting. You contributed to my success and I appreciate you so much. I love you dearly. Fourth, to my Mother, Esmie, and my brother, Andrew, thank you both for believing in me and for supporting me in this endeavor.

Finally, to my Heavenly Father, Yahuah, thank you for calling and equipping me for this work. You brought me through every trial, removed every obstacle out of my way, and gave me victory again and again and again. Thank you. Please be glorified in who I am, in what I do, and in where I go.

iv

Table of Contents

ACKNOWLEDGEMENTS ...... III CHAPTER 1: INTRODUCTION ...... 1 1.1. SOME BACKGROUND ON JAMAICA ...... 2 1.2. STATEMENT OF THE PROBLEM ...... 4 1.3. RESEARCH QUESTIONS ...... 7 CHAPTER 2: BACKGROUND AND CONTEXT ...... 8 2.1. BRIEF HISTORY OF THE JAMAICAN EDUCATION SYSTEM ...... 8 2.2. STRUCTURE OF THE JAMAICAN EDUCATION SYSTEM ...... 9 2.3. HISTORY OF EDUCATION REFORM IN JAMAICA ...... 13 2.4. RESEARCH FOCUS: JAMAICAN PUBLIC-PRIMARY EDUCATION ...... 14 2.5. CURRENT PRIMARY EDUCATION ACCOUNTABILITY SYSTEM ...... 15 CHAPTER 3: HIGH-STAKES ASSESSMENT ...... 19 3.1. HIGH-STAKES ASSESSMENTS ...... 19 3.2. HISTORY OF HIGH-STAKES ASSESSMENTS ...... 20 3.3. DEBATE ON HIGH-STAKES ASSESSMENTS ...... 22 3.4. INTENDED CONSEQUENCES OF HIGH-STAKES ASSESSMENTS ...... 22 3.5. UNINTENDED CONSEQUENCES OF HIGH-STAKES ASSESSMENTS ...... 24 3.5.1. NARROWING OF THE CURRICULUM ...... 24 3.5.2. ADVERSE EFFECTS ON DISADVANTAGED STUDENTS ...... 25 3.5.3. MISLEADING MEASURE OF OVERALL ACHIEVEMENT ...... 26 3.6. HIGH-STAKES ASSESSMENT IN JAMAICA ...... 26 CHAPTER 4: RESEARCH DESIGN ...... 30 4.1. NATIONAL ACHIEVEMENT DATA ...... 30 4.2. ANALYTIC SAMPLE ...... 31 4.3. MEASURES ...... 32 4.4. DATA ANALYTIC STRATEGY ...... 36 4.5. A QUALITATIVE PEEK: JAMAICAN SCHOOL VISITS ...... 37 CHAPTER 5: RESULTS ...... 42 5.1. QUANTITATIVE FINDINGS ...... 42 5.1.1. WERE JAMAICAN CHILDREN SUCCESSFUL IN PASSING THE GFLT? ...... 42 5.1.2. DID STUDENTS WHO FAILED EVENTUALLY BECOME ELIGIBLE FOR HIGH SCHOOL? . 49 5.2. QUALITATIVE FINDINGS ...... 54 5.2.1. STUDENT PERSPECTIVES ON THE GRADE 4 LITERACY TEST (GFLT) ...... 55 5.2.2. PARENT PERSPECTIVES ON THE GRADE-4 LITERACY TEST (GFLT) ...... 63 5.2.3. TEACHER PERSPECTIVES ON THE GRADE-4 LITERACY TEST (GFLT) ...... 68 5.2.4. PRINCIPAL PERSPECTIVES ON THE GRADE-4 LITERACY TEST (GFLT) ...... 73 5.3. LINKING THE QUALITATIVE AND QUANTITATIVE FINDINGS ...... 76 CHAPTER 6: DISCUSSION ...... 79 6.1. EVIDENCE OF LOW PERFORMANCE LEVELS ON INITIAL SITTING AND ON RETAKE ...... 80 6.2. GENDER DIFFERENCES IN GFLT PERFORMANCE ...... 82 6.3. SCHOOL SES AND SCHOOL-SIZE DIFFERENCES ON THE GFLT ...... 83 v

CHAPTER 7: CONCLUSION AND IMPLICATIONS ...... 85 7.1. IMPLICATIONS ...... 85 APPENDICES ...... 87 APPENDIX A: THE JAMAICAN NATIONAL ASSESSMENT PROGRAMME (NAP) ...... 87 APPENDIX B: SCHEMATIC OF HOW THE COMPETENCE-BASED TRANSITION POLICY (CBTP) WORKS ...... 89 APPENDIX C: GRADE-4 LITERACY TEST (GFLT) TEST-ITEM STRUCTURE ...... 90 APPENDIX D: CODEBOOK OF VARIABLES ...... 94 APPENDIX E: SAMPLE GIS MAP OF JAMAICAN PARISH ...... 95 APPENDIX F: TAXONOMY OF STATISTICAL MODELS ...... 96 APPENDIX G: INTERVIEW GUIDE FOR GRADE-FOUR FOCUS GROUP ...... 104 APPENDIX H: INTERVIEW GUIDE FOR GRADE-5 STUDENTS WHO PASSED THE GFLT ...... 106 APPENDIX I: INTERVIEW GUIDE FOR GRADE-5 STUDENTS WHO FAILED THE GFLT ...... 108 APPENDIX J: INTERVIEW GUIDE FOR TEACHERS AND PRINCIPALS ...... 110 APPENDIX K: INTERVIEW GUIDE FOR PARENT FOCUS GROUP ...... 113 APPENDIX L: VIOLATIONS IN IMPLEMENTATION OF THE COMPETENCE-BASED TRANSITION POLICY (CBTP) ...... 116 REFERENCES ...... 123 VITA ...... 129

vi

Abstract

In a move to address the persistently low literacy levels of primary schoolchildren, in

2009, the Jamaican Ministry of Education reclassified the Grade-4 Literacy Test (GFLT) to high stakes. Since then, students must pass the GFLT before they can be promoted to high school. In this thesis, I focused on students who failed their initial attempt at the test and retook it the following year. Then, I investigated the relationships between the probability of ultimately becoming eligible for promotion to high school, on the retake of the GFLT, and selected important child and school characteristics, including: student gender, school type, school examination-cohort size, and the socioeconomic level of the schools. To supplement and enrich the descriptive statistical analyses, I also included interviews and focus groups with a small sample of students, parents, and educators at three public-primary schools, in which they discussed their experiences with the GFLT.

In the quantitative analyses, I used data on 15,287 students in 758 public-primary schools, who retook the GFLT in school year 2010/11. I used random-intercepts multilevel modeling to investigate the student recovery rate (probability of ultimately becoming eligible for promotion to high school) as a function of the selected student- and school-level variables. I found that recovery rates were modest, generally showing that 7 to 17 percent of students who had initially failed the GFLT were able to become eligible for promotion. I also found a consistent gender disparity in recovery rates, against boys, and found that students in small schools had lower probabilities of becoming eligible for promotion to high school than did their peers in larger schools, with the effect being particularly pronounced in schools in high-SES districts. Finally, even though no measures of student language were included in the provided administrative datasets that were the basis of my quantitative vii analyses, my qualitative interviews with participants suggested that students might be underperforming on the GFLT because their first language is Jamaican Creole. This is an ongoing debate in the country.

1

Chapter 1: Introduction

Jamaica, the beautiful “Land of Wood and Water,” is in its eighth year of a national campaign, Vision 2030 Jamaica, to become a developed nation by 2030 (Planning Institute of

Jamaica [PIOJ], 2009). The government of Jamaica has acknowledged that an empowered citizenry is a crucial pillar for sustained economic growth and has placed the Jamaican people at the center of the island’s long-term national development plan (United Nations Development

Program [UNDP], 2013; PIOJ, 2009).

As of 2015, Jamaica was classified as having a high value of the Human Development

Index (HDI=0.730) and was ranked 94th out of 188 countries and territories (UNDP, 2016).

The HDI is a summary measure that emphasizes a nation’s dependence on its people and their competencies for sustained economic development. It is constituted from three basic indicators of human development: life expectancy at birth, mean years of education of the adult population and the standard of living as measured by the Gross National Income (GNI).

The index is coded on a scale from a value of 0, which indicates the lowest level of development, to 1, which indicates the highest level of development (UNDP, 2016). Over the past 25 years, Jamaica’s HDI has increased by 12% (from 0.651 to 0.730) signaling the country’s progress in improving the quality of life of its citizens. However, despite these improvements, Jamaica’s HDI is below the average for all countries in the Latin American and Caribbean region (0.751) (UNDP, 2016).

Thus, much remains to be accomplished and the country has embarked on several infrastructural transformations aimed at enhancing the quality of life for all Jamaicans. 2

Particularly in education, the Ministry of Education has embarked on a nationwide initiative to transform the Jamaican education system, a crucial strategy by which they hope sustained economic growth will be fostered.

1.1. Some Background on Jamaica

Despite the presence of political stability since gaining independence from British rule, in 1962, the Jamaican economy has grown more slowly than most countries (World

Bank, 2011; Inter-American Development Bank [IADB], 2003). Prior to Independence, Jamaica’s gross domestic product (GDP) grew by approximately 6 percent each year. In the decade following Independence, Jamaica was a leader among Caribbean countries in economic development due to its growth in the mining, tourism, manufacturing and construction industries (PIOJ, 2009; World Bank, 2004/2011).

However, this growth has not been sustained to the present day, in part, because of

Jamaica’s heavy reliance on foreign investments and its vulnerability to external shocks such as hurricane, oil-price changes, and international recessions, which are all beyond the country’s control. These events impacted the economy adversely and Jamaica experienced fluctuating periods of poor and mediocre economic performance, with an average growth rate of around 1% per annum, with weak infrastructure and increasing crime rates. In fact, the effects of the global economic downturn of 2008-2009 are still evident in the fluctuating nature of the Jamaican economy. After reporting positive growth of 1.4% in 2011, Jamaica reported negative growth of -0.47% the following year (2012). Then in 2013 and 2014, the country reported positive growth rates of 0.20% and 0.53%, respectively (PIOJ, 2015).

Many economists have attributed Jamaica’s unfavorable economic performance to several constraints, including high public debt, elevated unemployment levels, low labor 3 productivity, migration of skilled labor, rising poverty levels, a high crime rate, and low levels of student educational outcomes (UNDP, 2013/2016; World Bank, 2011).

Another reason for the economic stagnation can be attributed to the quality of life of the Jamaican people, which did not improve at the same rate as did the country’s economic growth. There have been many issues of inequality and inequity in the delivery of public goods and services to the people, such as clean water and sanitation, infrastructures, health care, employment, and education (World Bank, 2011; PIOJ, 2009; Jamaica Productivity Center

[JPC], 2009). This situation, subsequently, has encouraged an increasing migration of

Jamaicans, especially the highly educated, to other countries. This diaspora further exacerbated Jamaica’s economic position (PIOJ, 2009/2015; World Bank, 2011; JPC, 2009).

Despite the challenges, Jamaica has made progress on key social outcomes. Life expectancy at birth has increased from 72 years in 1990 to 76 years in 2014, and is now slightly above the average for upper-middle income countries (74 years). Similarly, infant and maternal mortality have declined by 44% and 19% respectively since 1995 and the prevalence of undernourishment has dropped from 10.5% in 1992 to 8.1% in 2015 (World

Bank, 2017). Another aspect of Jamaica’s social condition, which, I believe, positions the island well for future economic growth, is the moderate aging of the population. The rate of population aging matters, because it impacts the dependency ratio: developing countries will struggle to meet the needs of an older population if they remain poor (UNDP, 2013). As fertility and mortality rates have declined, the working-age (15-64 years) population in

Jamaica has grown, resulting in a “demographic dividend ” that the island can capitalize on, for future economic development (World Bank, 2011; PIOJ & Statistical Institute of Jamaica

[STATIN], 2010; Bloom, Canning, & Sevilla, 2003).

4

1.2. Statement of the Problem

Despite ongoing efforts to reach “world-class” standards, Jamaica has made only mixed progress in improving its public education system. Student achievement has been less than satisfactory at all levels of the system (Caribbean Policy Research Institute [CaPRI] &

Partnership for Educational Revitalization in the Americas [PREAL], 2012). At the primary level, students continue to underperform in core subjects, such as mathematics and English-

Language Arts (ELA). Students also continue to transition to the secondary level, or exit the system entirely, with inadequate education (PIOJ, 2009; Davis, 2004). According to the

Jamaican Ministry of Education (JMOE), in 2014, 95% of students attending private schools showed mastery of the literacy test, compared with 75% among public school attendees

(Saunders, 2015). At Grade 6, the average score in English Language Arts (ELA) was 68.4%

(Linton, 2015). Of particular concern is the low achievement of boys and children from poorer communities. Performance on all the examinations was higher for girls and for students who attend private schools, who are generally wealthier than those attending public institutions. Substantial achievement gaps between girls and boys have been noted since the

1990s, with boys’ performances lagging consistently (Evans, 1999).

The world economy is experiencing rapid technological change, coupled with global movement toward knowledge-based economies. This is placing increasing demands on education systems worldwide to deliver a more highly trained and competitive workforce.

Consequently, the persistent underachievement of the Jamaican education system, especially at the primary level, is particularly troubling.

How should the country respond? What kinds of new efforts might be helpful?

One initiative in Jamaica began in 2009 with the implementation of the Competence-Based

Transition Policy (CBTP), for qualifying students for secondary education. In prior years, 5 children only sat the Grade-6 Achievement Test (GSAT) near the end of Grade 6, to determine their placement in secondary education. This new policy puts heavy emphasis on the results of a standardized, externally-administered national assessment to fourth graders, whereby they will not be allowed to sit the GSAT and advance to the secondary level unless they pass this grade-four examination (Jamaica Ministry of Education [JMOE], 2011). Students who fail the Grade-4 Literacy Test (GFLT) are given additional opportunities to improve, but the policy insists that they pass this examination before they can become eligible to take the grade-six examination and be placed into secondary school.

How was the new policy implemented? What are some of the consequences of this new policy? What has changed since the policy was instituted in 2009? The impacts of the

CBTP permeate several levels of the Jamaican education system, from the students and their families to the system’s educators and even to policymakers. For students, the high stakes associated with the new policy could potentially serve as a motivator, encouraging them to study harder and do well. The results might also help teachers to identify struggling students who need extra assistance and support. Similarly for schools, the new policy could incentivize them to develop more effective classroom and curricular strategies in order to attain high levels of student achievement. The Ministry of Education could also identify and consult with high-performing schools, across the nation and across different socioeconomic strata, to develop a database or repository of best practices that other schools could customize to fit their own context. Even parents could benefit from this policy, by increasing their awareness of their children’s learning and by fostering their participation in their children’s educational progress.

Certainly, there are many ways to examine how the new policy may be meeting the objectives of improving student outcomes on the GFLT. By carefully analyzing the test- 6 score results, we can begin to explore lessons that might guide future policies. One type of study might look at the impact of the new policy on student enrollment in secondary education. By looking at the enrollment rates for students at different levels of test performance at the primary level, we can begin to draw some conclusions about the effectiveness of the Competence-Based Transition Policy.

One approach is to focus on describing the subsequent situation for students who failed to pass the required test in Grade 4. What happened to these students? How many of them were there? How many were later able to “recover” and enroll in secondary school?

What kinds of students were most likely to be successful in the grade-four test? What kinds of public-primary schools seem to do well in getting students up to the benchmarks for enrollment in secondary education, after the students failed their initial attempt at the grade- four examination? I believe that one objective of the test should be to provide feedback to students, schools and families about which students require extra help and instruction in order to qualify for secondary school in Jamaica. In this context, it is valuable to examine the “recovery rates” for students who failed the grade-four test initially.

In the current research, I focus on such students. By using multi-level logistic regression and focusing on students who did not pass their first attempt at the Grade-4

Literacy Test (GFLT), I can investigate the probability of subsequently becoming eligible for promotion to secondary school, given initial failure. To illustrate and enrich the findings from the statistical analyses, I also include qualitative examples and vignettes from several visits and discussions that I had with students, parents and school personnel to discuss the ways in which the grade-four test results were being used and interpreted by participants in the system. The implications may be powerful as it may reveal the student-level or school- level factors that are most strongly related to student success on retake of the GFLT? 7

1.3. Research Questions

In this study, I investigate the relationship between students’ recovery rates – that is, the probability that a student will become eligible for promotion to high school, among primary schoolchildren who failed their initial attempt at the Grade-4 Literacy Test (GFLT) and selected student and school characteristics, including: student gender, primary school type, school SES and school examination-cohort size. I addressed the following research questions:

1. What is the probability of becoming eligible for promotion to high school for students who

failed their first attempt at the Grade-4 Literacy Test (GFLT)?

2. Do student recovery rates differ by student gender, the different types of public-primary

schools, school examination-cohort size and school SES?

I hope that my findings will contribute to ongoing educational improvement in

Jamaica. Education is crucial to nation building and, therefore, I believe that it is vital to investigate the challenges faced by a system which continues to produce less-than-acceptable outcomes. My research is focused at the primary level of the Jamaican education system, in particular, and on the less than satisfactory performance of students on the GFLT among primary schools in the public sector. Addressing one facet of the persistently low academic performance in this population is, I believe, an important first line of action in ensuring that

Jamaica generates high levels of human capital, crucial for sustained economic development in the future.

8

Chapter 2: Background and Context

2.1. Brief History of the Jamaican Education System

In Jamaica, prior to emancipation in 1834, there was little formal education for

Whites and no education for slaves. Wealthy colonists sent their sons back to Great Britain for schooling or hired private tutors, while the less affluent ones sent their sons to a free school that was established “through bequests of wealthy planters and merchants.” The curriculum in the free schools mirrored that offered in Great Britain and was intended to teach boys so that they would “take their place in society.” Few slave children received education back then, and then it was mostly to teach them religion and the “virtues of submission.” The oldest record of girls being educated is in 1770, and in their case they were prepared for employment in roles such as seamstresses or for the “rigors of running a home.”

After emancipation, the colonists saw education as a means of incorporating the newly freed slaves into the economy, as well as maintaining peace and the status quo.

Elementary schools were developed by missionaries to educate the newly freed slaves. The purpose of education during this period was to prepare children for work on the plantation estates. The curriculum focused on reading, writing, arithmetic, religion, geography and history. Additionally, boys and girls were tracked into different fields. Boys were trained in agriculture and artisan skills, while girls were trained in domestic science (Whyte, 1983). 9

In the early 20th century, the British population in Jamaica started to decline because people were returning to the “mother country.” This spurred the expansion of secondary education in order to train and hire native Jamaicans into intermediate occupations that were being vacated by the emigrating Whites. Following a series of reforms and reorganization efforts, the modern structure of the education system began to emerge in the 1960s, after the country gained independence.

2.2. Structure of the Jamaican Education System

The Jamaican Ministry of Education governs the country’s education through its central office and six regional educational offices. Each regional office is responsible for the supervision and maintenance of a cluster of schools, from the pre-primary to the secondary level. The structure of the education system, as specified in the Education Regulations (1980), consists of four education levels: early childhood, primary, secondary, and tertiary (JMOE,

2009; United States Agency for International Development [USAID], 2005). I describe each briefly below.

Early Childhood (Pre-Primary) & Primary Education

Most students enter the Jamaican education system through the early childhood level, which caters to children between the ages of three to five years old (see Table 1). After successful completion of pre-primary education, children are then promoted to the primary level. Primary schools serve children aged six to eleven, in Grades 1 through 6. At this level of the education system, students are taught a national integrated curriculum until grade

10

Table 1: Structure of the Jamaican Education System

Education Level School Types Age Group Grades Enrollment Infant/Nursery Schools Infant departments (primary schools) Early Childhood 3 - 5 K 99% Kindergartens Basic Schools

Primary: Private Preparatory 6 - 11 1 - 6 92.5% All Age Public Primary Primary and Junior High

Secondary: Lower 12 - 14 7 - 9 Secondary High Technical High 80.0% (All Age and Primary-and-Junior High Upper schools offer lower-secondary) 15 - 16 10 - 11

Pre-University Offered in some secondary schools 17 - 18 12 - 13

University of the West Indies Tertiary University of Technology >19 28.7% 5 teachers colleges Source: The Jamaican Ministry of Education

11 four, when they then specialize in discrete subject areas, including mathematics and English

Language Arts, in preparation for study at the secondary level (JMOE, 2009/2017).

Secondary Education

The secondary level provides five years of education to students entering Grade 7. It comprises two phases: lower and upper secondary. Lower-secondary education is offered in the first three years of high school, in Grades 7 through 9, to students twelve through fourteen years old. All-age schools as well as the primary-and-junior high schools also offer lower-secondary education in Grades 7 through 9. Upper-secondary education is offered in the remaining two years of high school, in Grades 10 to 11, to students fifteen through sixteen years old. Some secondary schools offer an additional two years of education, in

Grades 12 and 13, to students who choose to pursue advanced studies. These two years are considered by some to be part of the tertiary system, pre-university, which includes community colleges, teacher-training institutions, three universities and other post-secondary institutions. The Ministry of Education website (JMOE, 2017) also notes that:

Some high schools have a continuing education programme, provided under the Career Advancement Programme and the Sixth Form/Pre- university programme (Grades 12 and 13) where students are prepared for entry to tertiary institutions. Students who are in Sixth Form sit the Caribbean Advanced Proficiency Examination (CAPE) at the end of Grades 12 and 13.

Tertiary Education

Finally, the tertiary level caters to selected students who completed secondary education successfully and are interested in pursuing advanced degrees, diplomas or 12 certifications (JMOE, 2008/2017; CaPRI & PREAL, 2012). According to the Ministry of

Education website (JMOE, 2017):

Tertiary education is offered in a variety of public and private institutions differing in history, mission, philosophy, and to a lesser extent, in programmes and structure. These institutions possess varying degrees of autonomy. These institutions may be further divided into those that are founded and established in Jamaica, and those that are offshore institutions. The offshore institutions main campuses are located outside of Jamaica, but they offer programmes through their campuses, and departments located in Jamaica. All tertiary institutions were established in response to educational needs at different times and offer not only degrees, but certificates and diplomas. The main accreditation body for tertiary institutions and their programmes is the University Council of Jamaica.

The Ministry of Education has formulated plans and mobilized resources to improve access to all levels of the Jamaican Education System (MOE, 2009; Davis, 2004). And today, in accordance with Millennium Development Goals (Millennium Development Goals [MDG],

2014; JMOE, 2012), the country has achieved a high net enrollment at the early childhood

(99%), primary (92.5%), and lower-secondary (80%) levels (IADB, 2016). However, at the upper-secondary and tertiary levels, despite marked improvement in net enrollment, the

Ministry is still very concerned about making access to these higher levels universal, because of its implications for the educational level of Jamaica’s future workforce and the island’s development plans (CaPRI & PREAL, 2012; JMOE, 2009). However, enrollment in education does not ensure quality education, and Jamaican students continue to perform below acceptable standards at all levels of the education system, across socioeconomic groups (CaPRI & PREAL, 2012; PIOJ, 2009). This persistent trend of underperformance, if not curtailed, will lower the skill level of the nation’s labor supply, leading to a negative effect on subsequent economic growth.

13

2.3. History of Education Reform in Jamaica

In recognition of the importance of education to human-resource development and of the dire constraints to sustained economic development of low-quality human capital, the

Jamaican Ministry of Education has undertaken initiatives to improve the quality of education in the country. For example, at the primary level, in the late-1980s, the

Government of Jamaica and the Inter-American Development Bank (IADB) collaborated on a joint venture, Primary Education Improvement Project (PEIP), to address the perpetual underachievement of students in Jamaican primary education. The endeavor focused on building school capacity, implementing a national core curriculum, and establishing national- assessment standards to monitor student academic progress (Lockheed, 2006; USAID,

2005). As follow-up, the Primary Education Support Programme (PESP) was later implemented to sustain the advances made with PEIP and to improve the performance, efficiency and equity of Jamaican primary education (PIOJ, 2009; JMOE, 2009).

As a result of PEIP, in 1999, the Ministry established the National Assessment

Programme (NAP), an inventory of four standardized national examinations, to assess and monitor student-learning outcomes from Grade 1 through Grade 6: (1) Grade-1 Individual

Learning Profile (GOLIP), (2) Grade-3 Diagnostic Test (GTDT), (3) Grade-4 Literacy Test (GFLT), and (4) Grade-6 Achievement Test (GSAT) (CaPRI & PREAL, 2012; MOE, 2009; USAID,

2005). Initially, all four examinations were low-stakes classroom-formative assessments (See

Appendix A for additional details on NAP).

GOLIP is administered to students, upon entry into Grade 1. Test results are intended to inform the analysis of student readiness for the primary curriculum as well as the development of curricular strategies to promote their learning. At the conclusion of Grade

3, students then sit the GTDT, which evaluates their abilities and weaknesses in reading and 14 mathematics. Educators are expected to use this assessment data to identify the skill levels of incoming grade-four students so that they can provide the necessary support to boost student achievement (Hunter, 2013).

Near the conclusion of Grade 4, students are required to sit the GFLT. The purpose of this examination is to assess student literacy skills in order to identify areas of improvement so that interventions could be implemented to augment student learning, before they sit the later GSAT. At the end of Grade 6, students then take the GSAT to determine their future placement at the secondary level (MOE, 2013; CaPRI & PREAL,

2012). While there is no minimum passing score for the GSAT, each student has only one opportunity to take the latter examination and, based on his or her national ranking, is then placed in a particular high school, hopefully of their choice.

The Ministry of Education also administers a terminal examination at the secondary level. In Grade 11, students sit the Caribbean Secondary Education Certificate (CSEC), which certifies their academic, technical and vocational achievements. CSEC has very strong currency for Jamaica because it not only indicates students’ readiness for post-secondary education but also signals the quality of the future workforce, with implications for Jamaica’s national development plans.

2.4. Research Focus: Jamaican Public-primary Education

In this study, I focused my research on student performance at the primary level of

Jamaica’s education system for three reasons. First, for everyone, primary education lays the foundation for the acquisition of the basic literacy and numeracy skills necessary for future participation in Jamaica’s social and economic activities (CaPRI & PREAL, 2012). Second, in Jamaica, about one-third of the current “out of school” population exited the education 15 system at the primary level (CaPRI & PREAL, 2012). Third, approximately 90% of

Jamaica’s poorest people and about 56% of its wealthiest people are not certified in any secondary or post-secondary subjects (CaPRI & PREAL, 2012). These facts signal a need to enhance the educational attainment of Jamaica’s citizenry, at its earliest stages, in order to prime the educational pipeline and sustain a future knowledge-based economy (CaPRI &

PREAL, 2012). In my view, we must ensure that primary schoolchildren persist through to the secondary (and tertiary) levels of the education system, in order to become adequately educated adults who can participate in Jamaica’s future economy.

Furthermore, I chose to investigate achievement in public-primary schools rather than those in the private sector because the qualitative differences between these sectors are so stark. These sectors are an expression of the social stratification in Jamaica, a relic of

British colonialism. In the private sector, there are the preparatory schools, which serve children of the upper-socioeconomic classes and are well resourced. In the public sector, there are three types of primary schools (primary, primary-and-junior high, and all age), which serve children of the majority working class and are poorly financed. This bifurcated system provides inequitable teaching and learning conditions to different facets of Jamaican society, resulting in an unrelenting achievement gap between the wealthiest and poorest students, as well as between students from rural and urban areas (CaPRI & PREAL, 2012).

2.5. Current Primary Education Accountability System

Indeed, Jamaica has a strong record of promoting primary education. However, despite these efforts, Jamaica continues to address the challenges created by an education system whose participants under-perform at all levels persistently and that continues to 16 under-produce secondary- and tertiary-educated workers (CaPRI & PREAL, 2012; PIOJ,

2009).

Consequently, in a move to address the low literacy achievement in primary schools, in 2009, the Ministry implemented the Competence-Based Transition Policy (CBTP), which prohibits the automatic promotion of children from the primary to the secondary level.

Under this policy, all Jamaican schoolchildren must now pass the pivotal Grade-4 Literacy Test

(GFLT) in order to be eligible, ultimately, to take the grade-six examination (CaPRI &

PREAL, 2012; MOE, 2011). In other words, the previously low-stakes GFLT was transformed into a high-stakes assessment, with serious long-term consequences for children.

However, students who fail the first sitting in Grade 4 have another chance to take the high-stakes Grade-4 Literacy Test and, consequently, be certified as literate. Those who are still unsuccessful in passing the test, after exhausting these opportunities, are transferred to an intervention program, the Alternative Secondary Transitional Education Programme (ASTEP), to be remediated for up to two years before being allowed to enter secondary education. (See

Appendix B for a schematic of how the CBTP works.) It is worth mentioning that, similar to the Grade-4 Literacy Test, the Grade-4 Numeracy Test (GFNT) is also a requirement for completing primary education. However, students are not held accountable for their performance in mathematics. Whether they pass or fail the numeracy test, students are allowed to take the grade-six examination only after they have passed the literacy examination.

There has been some improvement since this comprehensive assessment regime was established. In Table 2, I illustrate the incremental improvements in student performance in the grade-four and grade-six examinations, from 2010 to 2016. 17

Table 2: Examination results for 2010 to 2014, (a) Percentage of students passing the Grade-4 Literacy Test (GFLT); (b) National mean average scores for the Grade-6 Achievement Test (GSAT); and (c) Percentage of students passing the Caribbean Secondary Education Certificate (CSEC)

GFLT 2010 2011 2012 2013 2014 2015 2016

Literacy 66.9% 71.4% 73.7% 76.4% 77.4% 86.50% 80

GSAT 2010 2011 2012 2013 2014 2015 2016

English Language Arts (ELA) 58.0% 58.0% 60.0% 63.0% 63.0% 64.4% 68.4%

Communication Task 66.2% 66.2% 75.0% 71.0% 72.0% 69.5% 72.0%

CSEC 2010 2011 2012 2013 2014 2015 2016

English Language Arts (ELA) 59% 61% 52% 64% 59% 60% 67%

18

As you can see in Table 2, between 2010 and 2016, the percentage of students passing the Grade-4 Literacy Test (GFLT) increased by 13.1 percentage points, from 66.9% to 80%. It is important to note that passing the GFLT requires that students achieve mastery on all three strands of the test (Word Recognition, Reading Comprehension, and Writing Task). Additionally, over the seven-year period displayed, the national mean average scores for English Language

Arts (ELA) and Communication Task on the Grade-6 Achievement Test (GSAT) increased by 10.4 percentage points and 5.8 percentage points, respectively. Clearly, these modest increases in achievement scores signal that Jamaican students continue to underperform at the primary level.

At the secondary level of the education system, similar underperformance is observed in students’ scores on the Caribbean Secondary Education Certificate (CSEC). As shown in Table 2, only 67% of the students who took the CSEC were certified as proficient in ELA (MOE, 2012).

In my view, taken together, these results represent cause for grave concern for the

Jamaican government, and other critical stakeholders, because they suggest that there are lingering failures in the Jamaican education system that impact student learning negatively

(CaPRI & PREAL, 2012; MOE, 2009).

19

Chapter 3: High-Stakes Assessment

3.1. High-Stakes Assessments

In today’s climate of standards-based accountability, large-scale standardized assessments are often used to measure and monitor the academic progress of students, and these assessments are often conducted annually. They are administered to large groups of students at district, state, national, or international levels (Cole & Osterlind, 2008; DePascale,

2003).

Among the many standardized tests and assessments that are administered, some are regarded or treated as high-stakes tests, while others are used in a more low-stakes manner.

This means that the exact same test might be used in different ways or for different purposes. A test may be administered as a low-stakes assessment in one setting and as a high-stakes assessment with serious consequences in another setting. The main differences between the two approaches are the ways in which the results are used. Scores from low- stakes assessments can be used to provide meaningful feedback to students and educators on student learning. They are used to inform improvements in instructional and curricular strategies. They can be used by teachers and other school personnel to identify strengths or weaknesses in the school curriculum, or to identify students who may be in need of extra help or remedial instruction. Low-stakes assessments hold no significant or public consequences for stakeholders, for teachers, for students – at least in most instances (Cole &

Osterlind, 2008; DePascale, 2003). 20

In contrast, high-stakes assessments are typically used for accountability purposes and have serious consequences in the form of rewards or sanctions. High-stakes assessments are often used to measure academic achievement and provide information on school effectiveness to policymakers and the public (Cole & Osterlind, 2008; Nichols &

Berliner, 2007; DePascale, 2003). For students, they may be used to determine promotion to the next grade or graduation from high school. For educators, they may be used for performance evaluation and compensation. For schools, the results may lead to penalties such as reduced funding, negative publicity or school restructuring (Cole & Osterlind, 2008;

Moses & Nanna, 2007; Pedulla, Abrams, Madaus, Russell, Ramos, & Miao, 2003).

It is important to notice that there are several different levels at which high-stakes assessments may operate. They may be used at a national level, a state or local level, a district or school or classroom level. There may be consequences for school districts, schools, administrators, teachers, and students. The rewards or sanctions might be largely symbolic – warm praise or accolades for doing well, for example. Or the consequences might be more real and tangible – funding changes, bonuses, salaries, promotions. For students, these consequences might be relatively small and short lived, or the consequences might be important and long lasting. There might be consequences for a student’s academic future – they might be placed into a special section or course, denied or allowed access to a course or a school or a university. There might also be emotional or psychological consequences or stresses that might be more difficult to measure or assess.

3.2. History of High-Stakes Assessments

In the United States, large-scale educational assessment was expanded with the passage of the Elementary and Secondary Education Act of 1965 (ESEA). The assessment 21 measures associated with ESEA were used to provide educators with information to monitor student academic progress and to modify curriculum and instructional strategies.

During the Minimum Competency Test movement in the 1970s, policymakers started using assessment results to make decisions about students (Editorial Projects in Education

Research Center [EPERC], 2015; Nichols & Berliner, 2007). And by the mid-1980s, with the publication of A Nation at Risk in 1983, large-scale assessment, as we experience it today, was born. The report fueled public dissatisfaction with the quality of American public education and was a catalyst for the emergence of formal accountability systems, in the

1990s, that provided incentives or sanctions to students and schools on the basis of test scores (EPERC, 2015; Nichols & Berliner, 2007).

In 2001, the Elementary and Secondary Education Act was reauthorized as the No Child

Left Behind Act (NCLB) and signed into law in 2002. NCLB was intended to reduce the achievement gap between poor and wealthy students by providing them the opportunity to obtain a “high-quality” education. The law mandated that educators must assess students annually in Grades three through eight, and again in high school. With the passage of this legislation, states started administering large-scale assessments with greater frequency and with increasingly higher stakes, with serious consequences attached to them (EPERC, 2015;

Nichols & Berliner, 2007).

In developing countries, large-scale standardized assessment was jumpstarted after the 1990 World Conference on Education for All in Jomtien, Thailand. Education for All (EFA) is a global commitment to provide quality basic education to all children, youth, and adults. At the 1990 conference, 155 country delegates and 150 representatives from governmental and non-governmental organizations reaffirmed education as a human right and agreed to make primary education accessible to all children and erase illiteracy by the end of the decade. 22

However, a decade later, many countries had not fulfilled their EFA goals. Thus, in 2000, the international community (164 governments) reconvened at the 2000 World Education

Forum in Dakar, Senegal. There they identified and pledged to meet six EFA goals by 2015

(United Nations Educational, Scientific and Cultural Organization [UNESCO], 1990/2015).

3.3. Debate on High-Stakes Assessments

Since the advent of NCLB in the United States, high-stakes assessment has been one of the most controversial issues in education. Advocates assert that high-stakes testing is an important component of school improvement because it provides the necessary pressure, through incentives and sanctions, to (1) encourage students to work harder by providing information about their academic progress, (2) motivate teachers to adopt more effective classroom and instructional practices that promote student learning, and (3) hold schools accountable for academic achievement (Kulm, 2013; Jones & Egly, 2004). Critics, on the other hand, argue that these tests are a threat to the quality of teaching and learning, which leads ultimately to educational failure as well as signals to stakeholders that the primary purpose of learning is to score well on tests (Nichols & Berliner, 2007; Mitchell, 2006;

McMillan, 2005). Critics also contend that these tests: (1) narrow the curriculum, (2) promote class and ethnic disparities by holding students and teachers in high-poverty, resource-strapped schools to the same standards, and (3) increase test anxiety in students and teachers.

3.4. Intended Consequences of High-Stakes Assessments

Advocates of high-stakes testing contend that these assessments have positive consequences for American public education. By holding schools accountable for student 23 performance, they reason, teachers will align instruction to state content standards more accurately, schools will increase teacher professional development, and schools will offer effective remediation programs, which will lead to improvements in student learning (Kulm,

2013; Yeh, 2005; Pedulla et al., 2003).

A major intended consequence of the standards-based accountability movement is improvement of instructional practice, which would further lead to gains in student learning.

Proponents of high-stakes large-scale assessment believe that test results not only provide information to teachers on student academic progress but also promote curricular alignment among state content standards, classroom instruction, and examinations (Kulm, 2013; Jones

& Egly, 2004).

Another intended outcome of high-stakes assessment, often mentioned by supporters, is increased educational equity. The introduction of high-stakes assessments will ensure that all students will meet the requisite state standards. Advocates claim that schools can use test results to monitor and evaluate student performance in order to identify areas of improvement and to design instructional strategies tailored to the academic needs of low- achieving/at-risk students (Yeh, 2005; Jones & Egly, 2004). The argument is often made that without high-stakes tests and the related standards for achievement, there will be more students left behind and greater variability in overall levels of achievement. The high-stakes assessments are seen as a way to spur improvements for all kinds of students and reduce achievement gaps.

24

3.5. Unintended Consequences of High-Stakes Assessments

3.5.1. Narrowing of the Curriculum

Research shows the adverse effects of high-stakes standardized testing on classroom instruction and student learning (Yeh, 2005; McMillan, 2005; Jones & Egly, 2004; Pedulla et al., 2003). The increasing frequency and heightened stakes of standardized assessments have compelled teachers to modify instructional strategies and narrow the scope of the curricula.

Numerous studies provide evidence of how curriculum narrowing has occurred: (1) subject areas that are not tested are either removed from the curriculum or downgraded to lesser instructional time, (2) untested topics within tested subject areas are excluded, and (3) classroom instruction replaced by excessive test preparation (Nelson, 2013; Perelman, 2012;

Valli & Buese, 2007).

The current high-stakes testing movement has resulted in subjects such as art, music, social studies and foreign languages, which students are interested in, being “squeezed out” of the curriculum because they are not tested (Nelson, 2013; Perelman, 2012; Valli & Buese,

2007; Rhone, 2006). Similarly, untested topics within tested subjects, mainly English Language

Arts (ELA) and mathematics, are overlooked. For example, ELA teachers are required to focus their instruction only on the literacy skills measured on standardized tests. And for writing, students are limited to learning low-level skills such as answering multiple-choice and short-answer test items, at the expense of learning high-level skills such as composing extended prose and using invention strategies (Brimi, 2012; Perelman, 2012; Yeh, 2005). As a result of high-stakes pressure, classroom instructional strategies have moved from 25 authentic learning experiences involving higher-level thinking to repetitive rote memorization and drill-and-practice teaching methods to raise student scores.

There is no debate about the benefits of test preparation to student success on examinations. However, the heightened pressures of high-stakes assessments influence teachers and schools to devote excessive amounts of time, attention, and effort to test preparation, at the expense of instructional time, so that schools can increase the likelihood of high scores on these tests (Nelson, 2013; Rhone, 2006; Volante, 2005; Jones & Egly,

2004).

3.5.2. Adverse Effects on Disadvantaged Students

One justification for high-stakes standardized assessments is that they are intended to improve educational equity. However, numerous studies have shown that this is rarely the outcome: students from disadvantaged backgrounds have been affected adversely by these tests disproportionately more than their advantaged peers. These negative effects frequently lead to limited educational opportunities for them. Consequently, high-stakes standardized tests are not necessarily indicative of student achievement but rather of the inequities within and between schools. Additionally, research have found that narrowing the curriculum and changing the nature of teaching to improve test scores occur most often in under-resourced schools that serve low-income and minority students, schools that are typically the most pressured to improve test scores (Nelson, 2013; Moses & Nanna, 2007;

Nichols & Berliner, 2005; Diamond & Spillane, 2004).

Diamond and Spillane (2004), in their study of high-performing and low-performing elementary schools in Chicago, found that high-performing schools used high-stakes test results to identify problem areas and adopt intervention strategies for all students, whereas 26 low-performing schools focused intervention efforts only on students with the greatest probability of passing the examination.

3.5.3. Misleading Measure of Overall Achievement

The great importance placed on high-stakes standardized assessments has had a major influence on classroom practices. These pressures have led many teachers to abandon instructional strategies that engender student lifelong learning, in favor of increased emphasis on test-preparation activities. Consequently, the credibility of test scores is compromised because it is possible that high scores on examinations may reflect student familiarity with testing strategies rather than actual achievement (Jacob, 2005; Volante, 2005).

Also, it increases the risk of students being erroneously categorized as high performing, without any genuine improvement in their learning (Volante, 2005; Jones & Egly, 2004).

Critics of high-stakes testing have cautioned that decision-making about student academic progress, such are grade promotion and graduation, should not be based on a single test score but combined with additional academic records that reflect student achievement more accurately (Jacob, 2005; Volante, 2005).

3.6. High-stakes assessment in Jamaica

Jamaican educators have also been involved in the debates and discussions about high-stakes assessment. Over the past decade, the Ministry of Education has introduced several new initiatives, and the results are beginning to be available for researchers. In this paper, I will focus on the shift that has made the Grade-4 Literacy Test (GFLT) a high-stakes assessment. Part of the rationale for giving the test at grade four was that time would be 27 available for students to re-take the test. If they failed on a first attempt, they could still be allowed to pass at a later time and then become eligible for advancement to secondary-level education. In the current research, I focused on the cohort of students who took part in the new program in 2010. In this iteration, many students were able to pass, but a substantial number of them – from many different types of schools in locations all over Jamaica – did not achieve passing scores on the initial administration of the examination.

In my research, I investigate what happened to those students who failed. How many of them were able to pass the test on the next occasion? What percentage of students was able to “recover” and become eligible after an initial failure? It is these kinds of unknowns that drive my first research question, below.

In addition, I am interested in the variability in recovery that is evident across students and schools. In particular, given the historic differences that I have summarized in the way that the Jamaican Education System has treated boys and girls, both in terms of access and content, I want to examine whether failed girls recover their access to high school differently from failed boys. Do girls tend to recover from earlier failure more successfully than failed boys, or is the reverse the case? And, what hypotheses account for any extant differences? Given the recent research on Jamaican education that has consistently found large differences in academic achievement between girls and boys in Jamaica, I hypothesize that I will see differences in the recovery rates. The recent research shows that boys in

Jamaica start off with lower levels of academic performance and engagement, and the gaps persist throughout all levels of the education system (Clarke, 2005; Figueroa, 2004; Evans,

1999).

I am also interested in documenting and understanding any differences in recovery that may occur across schools. For instance, do children who fail the Grade-4 Literacy Test 28

(GFLT) recover at different rates in different types of school, in larger versus smaller schools or in schools from wealthier regions? I anticipate that such differences will be evident because several studies have identified substantial differences across Jamaican schools in resources devoted to instruction, in school leadership qualities, and levels of teacher preparation (Lockheed, Harris, & Jayasundera, 2010). The system in Jamaica has a long history of inequalities among schools – “haves” and “have-nots” – and thus I believe differences in school performance may at least partly reflect differences in socio-economic circumstances.

School sizes also vary substantially in Jamaica. This is important because of the school funding system for the government-funded public schools. Schools are provided funding based on total student enrollments, with larger schools receiving larger amounts of support. While in some settings, smaller-size schools may have advantages, I anticipate that this may not necessarily be true in Jamaica, where larger schools have access to greater resources.

There are three types of public-primary schools in Jamaica: all-age schools covering grades one through nine, primary schools covering grades one through six, and primary-and- junior-high schools cover grades one through nine. Historically, the all-age schools have been considered the weakest, and the primary-and-junior-high schools were developed in an attempt to upgrade these schools. Despite the different histories, recent research has not identified major differences based on school type. Factors such as socioeconomic levels and school size have been more important (Lockheed, et al., 2010).

Thus, in the current research, I focus on the extent to which Jamaican children who failed the high-stakes grade-four assessment at the first sitting were able to continue through 29 the system and eventually become eligible for promotion to high-school. Specifically, I addressed the following research questions:

1. What is the probability of becoming eligible for promotion to high school for students who

failed their first attempt at the Grade-4 Literacy Test (GFLT)?

2. Do their recovery rates differ by critical student and school characteristics, including:

student gender, the different types of public-primary schools, school examination-cohort size

and school SES?

30

Chapter 4: Research Design

In this chapter, I describe the major steps that I completed in addressing my research questions. I begin by describing the national data sets of student and school information that I used for the major statistical analyses, and that were provided to me by the Jamaican

Ministry of Education. I present examples of the kinds of test items that were included in the Grade-4 Literacy Test. In the next section, I describe my analytic sample of students – those who failed the test in its first administration. I explain how the key variables used in the statistical analyses were coded, and I present an outline of the main analytic strategy. I also describe the steps involved in the small-scale qualitative strand of my study, which I conducted to enrich and expand my interpretation of the quantitative findings. Thus, I outline two additional research trips that I made to Jamaica during which I met and talked with selected students, parents, teachers, and school principals. These latter discussions were helpful in finding out what participants knew about the new policies for determining eligibility for secondary school and what their reactions and recommendations might be.

4.1. National Achievement Data

The primary data source that I used in this study was provided by the Jamaican

Ministry of Education, and forms the basis for all my statistical analyses. It consists of two datasets that contain national records of primary schoolchildren in Jamaica. The first dataset contained demographic and achievement information for all Jamaican 4th-grade students, in 31 private and public schools, who registered for, and took, the Grade-4 Literacy Test (GFLT) in school year, 2009/10. (Appendix C provides examples of the three subtests that make up the GFLT.) The second dataset contained demographic and achievement information for all the nation’s 6th-grade students who registered for and took the Grade-6 Achievement Test

(GSAT) in school years 2011/12 through 2013/14. Specifically, these datasets contained detailed information concerning: student gender, school attended, scores on the all subtests of the GFLT and scores on all subtests of the GSAT. In the analyses that I present here, I elected to focus only on students in the Jamaican public-primary schools because these schools: (1) serve approximately 70% of the Jamaican schoolchildren who are mostly from lower socioeconomic backgrounds, and (2) the inequities in Jamaica’s education system are mostly experienced by students in these institutions (CaPRI & PREAL, 2012; Bailey, 2007).

I merged the two datasets using the unique school codes and student names contained in the datasets as the identifiers.

4.2. Analytic Sample

A total of 57,000 students in 1,041 Jamaican primary schools, public and private, took the Grade-4 Literacy Test (GFLT) in 2010. Of this population of test takers, 46,120 of them attended public-primary schools. Of those students in public-primary schools, about

61% of them passed (27,936) the first sitting of the GFLT, while 33% of them failed

(15,287) and 6% were absent (2,897).

In my quantitative analyses, I focused on the 15,287 fourth-grade students in 758 public-primary schools who failed their initial attempt at the GFLT in school year 2009/10.

I was interested in discovering how many of them overcame that first failure and, subsequently, became eligible to be promoted regularly to high school or never recovered 32 from that first failure and never became eligible for promotion and, consequently, transitioned into ASTEP.

4.3. Measures

In Appendix D, I provide a full codebook, with variable and value definitions of each of the variables included in my quantitative analyses. Below, I summarize briefly the principal variables included in the analysis, organized into categories by their function: outcome, question predictors and covariates.

Outcome

My outcome variable, PROMO, is a dichotomous variable that indicates whether a student became eligible for regular promotion to high school, one year after leaving grade six, (=1) or not (=0). Thus, children with PROMO=0 are those who did not pass the retake of the GFLT and were not eligible for promotion to high school.

Student-level Question Predictor

At the student level, I included student gender (FEMALE) as a question predictor in my analyses. FEMALE is a dichotomous variable that has a value of 1 if the student is a girl, and 0 if the student is a boy.

School-level Question Predictors

At the school level, I included question predictors to distinguish: (a) the three types of primary schools in the public sector, (b) the examination cohort size of the public-primary schools, and (c) the socioeconomic status of the families whose children fed the public- 33 primary schools, and (d) the level of urbanization of the fourteen parishes of Jamaica. I discuss each briefly below.

Public-primary school type:

SCHTYPE is a categorical predictor to distinguish the three types of public-primary schools: all age, primary, and primary-and-junior high. I recoded SCHTYPE into three dichotomous variables as follows:

1. SCHPUB1: A dichotomous predictor that has a value of 1 for students in the all-

age schools, 0 otherwise.

2. SCHPUB2: A dichotomous predictor that has as a value of 1 for students in the

primary schools, 0 otherwise.

3. SCHPUB3: A dichotomous predictor that has as a value of 1 for students in the

primary-and-junior high schools, 0 otherwise.

In my analyses, I omitted SCHPUB2 from my statistical models, to create a reference category, because I found that primary schools, as a whole, served the largest proportion

(approximately 70 percent) of the 2010 examinees and had the highest number of students passing at the first sitting of the GFLT. In addition, in preliminary analyses, I found that – after other school-level covariates had been included in my statistical models –school type no longer impacted recovery rates, and so this question predictor was omitted from my final statistical model

34

School Examination-Cohort Size:

SCHCSIZE is a continuous question predictor, ranging from 1 to 442, which records the number of students taking the GFLT in each school. I created this measure by counting the number of examinees, in each school, who initially took the test in school year 2009/10.

It serves as a proxy descriptor of school size. Because – in preliminary analyses -- I found that the distribution of SCHCSIZE was highly skewed, , I recoded it into three dummy variables for the purposes of analysis, as follows:

1. SM_CSIZE: a dichotomous predictor that takes on a values of 1 for schools with

examination-cohort sizes between 1 and 120 students, 0 otherwise.

2. MD_CSIZE: a dichotomous predictor that has as a value of 1 for schools with

examination-cohort sizes of 122 to 253.

3. LG_CSIZE: a dichotomous predictor that has as a value of 1 for schools with

examination-cohort sizes of 266 to 442.

School Socioeconomic Status:

SCHSES is a categorical predictor that records the poverty levels of the primary schools that students attended. I created this measure (SCHSES) from geographic information system (GIS) maps1 of the fourteen parishes in Jamaica. These maps were segmented and color coded by five poverty levels, with the names of primary schools superimposed on the images. (In Appendix D, I present a sample map of a Jamaican parish.)

I then recoded SCHSES into three dichotomous predictors as follows:

1 These maps were provided by the Planning Unit of the Jamaican Ministry of Education. 35

1. HiSES: a dichotomous predictor that takes on a value of 1 the first poverty level

(1) and represents the wealthiest communities. I omitted this predictor from my

statistical models to create a reference category. Approximately 19% of the

students attended schools were designated as HiSES.

2. MidSES: a dichotomous predictor that takes on a values of 1 for schools in the

second and third poverty levels (2 & 3), and were therefore public-primary

schools in “middle-income” communities. Approximately 55% of the students

attended MidSES schools.

3. LoSES: a dichotomous predictor that takes on a values of 1 for schools in the

two lowest poverty levels (4 & 5), and were therefore public-primary schools in

the poorest communities. Approximately 26% of the students attended LoSES

schools.

Urbanization Level:

I created predictor PERCENT_URBAN from public census data provided by the

Statistical Institute of Jamaica (STATIN) on the number of households per parish. Based on its distribution, I further recoded it into two dichotomous variables as follows:

1. RURAL: a dichotomous predictor that takes on a value of 1 for schools in the 10

parishes with urbanization levels between 0.11 and 0.36.

2. URBAN: a dichotomous predictor that takes on a value of 1 for schools in the 4

parishes with urban levels between 0.55 and 1.

36

4.4. Data Analytic Strategy

After extensive data checking and cleaning, I conducted a series of exploratory analyses during which I fitted taxonomies of random-intercepts multilevel logistic-regression models to examine the relationship between my outcome – subsequent eligibility for high- school -- as a function of student gender and school type, SES and size, among students who previously failed the grade-four assessment. I chose to use logistic regression analysis because my outcome – PROMO – was a dichotomy. My analysis was multilevel because students were clustered within schools.

In conducting my analyses, I entered predictors into my statistical models in an orderly fashion, first as main effects and then as higher-level interactions. At each step, I conducted appropriate General Linear Hypothesis (GLH) tests to determine which predictors needed to be retained in a final parsimonious model, to best explain the relationships between the probability of becoming eligible for promotion to high school and the previously mentioned student- and school-level predictors. In Appendix E, I provide a table containing a more comprehensive taxonomy of models fitted during this process, along with ancillary statistics and selected tests. My final fitted model was of the following form, for student i in school j:

𝐿𝑜𝑔𝑂𝑑𝑑𝑠 𝑃𝑅𝑂𝑀𝑂!" = 1 = 𝛽! + 𝛽! 𝐹𝐸𝑀𝐴𝐿𝐸!" + 𝛽! 𝑆𝑀_𝐶𝑆𝐼𝑍𝐸! + 𝛽! 𝑀𝑖𝑑𝑆𝐸𝑆!

+𝛽! 𝐿𝑜𝑆𝐸𝑆! + 𝛽! 𝑀𝑖𝑑𝑆𝐸𝑆!× 𝑆𝑀_𝐶𝑆𝐼𝑍𝐸! +

𝛽! 𝐿𝑜𝑆𝐸𝑆!×𝑆𝑀_𝐶𝑆𝐼𝑍𝐸!

In my findings chapter, which follows, I provide estimates of the fitted parameters from this final model, along with ancillary inferential and goodness-of fit statistics. I interpret my findings by antilogging parameters and interpreting the result as fitted odds- ratios (FEMALE) and I provide graphical representations of the more complex results that 37 consist of statistical interactions among school size and district SES. Notice that the level of district urbanization and school type do not appear as predictors in my final model, suggesting that – controlling for both School SES and School-Size – these predictors had no further impact on the probability of a failed child’s promotion to high-school.

4.5. A Qualitative Peek: Jamaican School Visits

As a follow-up to my analysis of the quantitative examination and school data, I collected additional information by way of in-depth open-ended interviews and focus groups with selected students, parents or guardians, and upper-primary teachers and principals in selected Jamaican primary schools. The anecdotes and impressions that I garnered from these visits were constructive and gave me insight into the Jamaican-school context, helping me interpret the patterns observed in my analyses of the national performance data. In what follows, I have integrated findings from analyses of this qualitative information into my discussion of results from the quantitative work, as appropriate.

During 2015, I made two data-collection trips to Jamaica during which I conducted a series of in-depth semi-structured interviews and held focus groups with students in Grades

4 to 6, along with their parents, principals and teachers at three public-primary schools. For reasons of access and economy, I only visited schools in Kingston, the Capital and largest city in Jamaica. To accomplish these visits, I collaborated with the Jamaican Ministry of

Education to select the specific public-primary schools, each representing one of the public- primary school types. Primarily, in my interviews and focus groups, I explored students’ and their families’ perspectives on the GFLT and their understandings of its impact on children’s academic futures. Secondarily, I examined school personnel’s perspectives on the literacy examination and their responses to the relatively new accountability climate in Jamaica. 38

Thus, while not causally conclusive, my research is illustrative of how these stakeholders understood the nature and consequences of the administration of the GFLT, as well as how they interpreted and communicated results of performance on the test. Below, I provide a brief overview of the details of each visit.

First Qualitative Research Visit: May 2015

On my first research trip, I visited three public-primary schools during May, one month before the administration of the Grade Four Literacy Test (GFLT) on June 30. I conducted focus groups with 16 fourth-graders, who would be sitting the test for the first time, and individual interviews with 20 fifth-graders, who would be retaking the test. I also interviewed eight upper-primary teachers and the three principals at these schools.

Fourth-Grade Student Focus Groups

At each of the three schools, I conducted focus groups with fourth graders who were preparing for their initial sitting of the Grade-4 Literacy Test. Using this data-gathering strategy with this sub-sample of students, I was able to: (a) capitalize on group interaction and elicit a diversity of perspectives on students’ knowledge about the test and whether it was prominent in their minds, and (b) facilitate the sharing of information, which potentially benefited these students by increasing their own understanding of the high-stakes test they were about to take. (See Appendix F for the interview guide for the grade-four focus group.)

Fifth-Grade Individual Student Interviews

For the fifth-grade sample, I conducted individual interviews with students who had taken the GFLT previously in grade four. Some of them had passed on their initial attempt, 39 and some were retaking the test later that month. I employed this particular strategy in order to: (a) provide a psychologically-safe environment for test repeaters to share their feeling about the GFLT, without feeling singled out, and (2) to use the students who already passed the test for purposes of comparison. In my interviews, I sought to extract information on fifth-graders’ feelings about, and experiences with, the GFLT (See Appendices G and H for the interview guides used with the Grade-5 interviews.)

Interviews with School Personnel

I also conducted a series of individual interviews with the principal of each school and selected teachers at each school. During these interviews, I gained understanding about school policies, and practices, for the Grade-4 Literacy Test (GFLT) and how educators’ perceptions of the test may have affected children’s responses to failing the test. (See

Appendix I for the interview guide.)

Second Research visit: November 2015

After analyzing the data from my first research trip to Jamaica, I returned to the country in November 2015, about one month after scores from the June 2015 Grade-4 Literacy Test were published, in order to gain further insight into students’ and parents’ experiences with, and responses to, the recently published GFLT scores. During this second phase of the study, I returned to one of the schools I had visited previously. There, I conducted: (a) a series of in-depth semi-structured interviews with students in Grades 5 and 6, and (b) two focus groups with parents. I collaborated with school administrators to select the samples of students and parents.

40

Fifth- and Sixth-Grade Individual Student Interviews

I conducted individual interviews with children who had recently been promoted to

Grade 5 and who had just been informed that they had failed the first sitting of the GFLT.

These students were about to begin remediation and preparations for retaking the test, on the next administration. I also interviewed students who had recently been promoted to

Grade 6 and who had just been informed that they had failed their second and last opportunity to sit the GFLT. These students were not certified as literate and, therefore, were not eligible to sit the Grade 6 Achievement Test (GSAT) and be promoted to high school.

Instead, if they remained in the school system, they would be transferred to the Alternative

Secondary Transitional Education Program (ASTEP) to be remediated for up to two years. I conducted these interviews in order to understand how these children had been impacted by the news that they had failed, especially the 6th graders who had exhausted all opportunities to retake the GFLT and were now on a precarious path to their future (Appendix H contains the interview guides for these students.)

Parent Focus Groups

I divided the sample of parents whom I interviewed into two focus groups: (a) parents whose children had recently passed the GFLT, and (b) parents whose children had recently failed the test. My aim in conducting these focus groups was two-fold: (a) I wanted to shed light on parents’ knowledge of the GFLT, of how test results were reported and handled, and seek their perspectives on their children’s experiences with the test, and (b) I wanted to see if there were any differences in understanding and perspective between the 41 parents of students who had passed the test and the parents of children who had not passed the test (Appendix J contains the interview guide for the parent focus groups.)

42

Chapter 5: Results

In this study, I investigated the relationships between the probability of becoming eligible for promotion to high school and student gender, school SES and school examination-cohort size. In this chapter, I review descriptive information on the Jamaican primary schoolchildren who sat the Grade Four Literacy Test (GFLT) in school year 2009/10 as well as analytic results generated from my random-intercepts multilevel logistic regression analysis of student GFLT performance data. First, I present a general overview of the

Jamaican primary education system, followed by results from my statistical analyses.

5.1. Quantitative Findings

5.1.1. Were Jamaican Children Successful in Passing the GFLT?

In the beginning phase of my data analysis, I discovered inconsistencies and exceptions as to how the Competence-Based Transition Policy (CBTP) was implemented. In

Appendix K, I present information on the steps I took in this initial investigation of the test results. After these first steps, I refined my focus for these analyses by directing attention to the results of the initial administration of the Grade-4 Literacy Test (GFLT) and the results of the follow-up test for students who did not pass on their first try.

The Grade-4 Literacy Test (GFLT) was administered in June 29, 2010 to approximately

57,000 children across the nation, in private and public primary schools. Of these students,

46,120 of them attended public-primary schools and were sitting the test for the first time. 43

For the remainder of this thesis, I focus on student performance in the public-primary schools. Overall, 61% of public-primary schoolchildren passed their first attempt at the

GFLT, and the passing rates differed widely across schools. Univariate statistics reveal that at least 20 public-primary schools had 100% of their students passing their first sitting, while at least 10 had none of their examinees passing the test.

Below, I present more details on the main results. To provide some context, I begin with descriptive information about the Jamaican education system. Next, I present descriptive information on the initial administration of the Grade-4 Literacy Test in 2010. I then present the passing rates on the initial sitting, and show how the pass rates differ across different types of schools, schools of different sizes, schools from different socio-economic levels, and schools in urban and rural locations. I note differences between the performances of the girls and boys.

These initial descriptive statistics provide a snapshot of overall passing rates on the

GFLT. Then, I focus my attention on the students who did pass on their first attempt. I present the main results from my logistic-regression analysis first. Then, in the final section of this chapter, I present selected results from my visits and discussions with students, parents, teachers and principals as a way of enriching the findings from my quantitative analyses.

In Table 3, I summarize participation in the 2010 administration of the Grade 4

Literacy Test (GFLT). A total of 57,000 students sat the examination. These students came from 1,041 public and private schools across the nation. For my analyses, I focused attention on the 46,120 who were attending the 792 public primary schools, who make up

76% of the entire examinee population.

44

Table 3: Descriptive statistics on students in Jamaican public-primary schools who sat the GFLT in 2010

Student and School Characteristics* Number of Schools % Schools Number of Students % Students Primary Schools Private 249 23.9 23.9 Public 792 76.1 76.1 Total 1,041

Student Demographics (Public): Student Gender Girls 22,554 48.9 Boys 23,566 51.1 Total 46,120

School Characteristics (Public): Public-Primary School Type: All Age Schools 159 20.1 7,693 16.7 Primary Schools 546 68.9 32,259 69.9 Primary-and-Junior High Schools 87 11.0 6,168 13.4 Total 792 46,120 Parishes: Kingston 25 3.2 2,588 5.6 St. Andrew 78 9.8 7,080 15.4 St. Thomas 42 5.3 1,773 3.8 Portland 44 5.6 1,344 2.9 St. Mary 59 7.4 1,908 4.1 St. Ann 68 8.6 3,186 6.9 Trelawny 32 4.0 1,460 3.2 St. James 43 5.4 3,501 7.6 Hanover 33 4.2 1,344 2.9 Westmoreland 55 6.9 2,818 6.1 St. Elizabeth 75 9.5 2,870 6.2 Manchester 58 7.3 2,934 6.4 Clarendon 86 10.9 4,780 10.4 St. Catherine 94 11.9 8,534 18.5 Total 792 46,120 Urbanization Level: Urban 21,703 47.1 Rural 24,417 52.9 Total 46,120 Socioeconomic Status of School: High SES 8,966 19.4 Middle SES 25,226 54.7 Low SES 11,928 25.9 Total 46,120 Relative School Size (Exam-Cohort Size): Small School 23,768 51.5 Medium School 16,609 36.0 Large School 5,743 12.5 Total 46,120 * Data provided by the Jamaican Ministry of Education

45

There is a balance of girls and boys, and over two-thirds of the participants were attending primary schools. The remaining students were in all-age schools or primary-and- junior high schools. Jamaica consists of 14 regions called parishes, and Table 3 shows that there was representation across all regions of the country. Although a majority of the parishes are considered to be rural, the student population is roughly evenly distributed in urban (47%) and rural (53%) schools.

Table 3 also shows that over one-half of the students (55%) attended schools that were considered to be in middle-SES communities. We should keep in mind, however, that as I described earlier in Chapter 4, the information on SES was not available for individual students. This is a designation for the location of the schools only. Thus there may be students who were attending schools in middle-SES locations, although some of the individual students would not be considered middle-SES.

In Table 3, I also provide the relative sizes of the schools. I constructed this variable by using the number of examinees from each school. About one-half (52%) attended schools that were relatively small (fewer than 120 students in the cohort). Thirty-six percent were in middle-size schools (122 to 250 students in the cohort). About 12 percent attended large schools with cohort sizes ranging from 266 up to 442.

My next step was to examine the passing rates from the initial administration of the

Grade-4 Literacy Test in 2010. In Table 4, I present these results. Of the 46,120 students who sat, the passing rate was just over 60% (60.6%). There was a substantial difference in the passing rates for girls and boys. Among the girls, 72.4% passed, while for boys the rate was only 49.3%. (The issue of gender differences in school achievement in Jamaica has been noted in many studies over the past 20 years [Evans, 1999].) In the remainder of Table 4, I

46

Table 4: Selected descriptive statistics for students who passed the first sitting of the Grade 4 Literacy Test (GFLT) in 2010

Student and School Characteristics* Girls % Girls Boys % Boys Overall % Overall Number of Students Passing: 16,324 72.4 11,612 49.3 27,936 60.6

Public-Primary School Type: All Age Schools 2,509 68.1 1,709 42.6 4,218 54.8 Primary Schools 11,859 74.2 8,454 52.0 20,313 63.0 Primary-and-Junior High Schools 1,956 67.9 1,449 44.1 3,405 55.2

Relative School Size (Exam-Cohort Size): Small School 7,895 69.4 5,492 44.3 13,387 56.3 Medium School 6,210 75.4 4,560 54.5 10,770 64.8 Large School 2,219 75.5 1,560 55.6 3,779 65.8

Socioeconomic Status of School: High SES 3,483 76.6 2,506 56.7 5,989 66.8 Midddle SES 8,697 71.8 6,330 48.3 15,027 59.6 Low SES 4,144 70.4 2,776 45.9 6,920 58.0

Urbanization Level: Urban 7,773 73.0 5,616 50.8 13,389 61.7 Rural 8,551 71.9 5,996 47.9 14,547 59.6 * Data provided by the Jamaican Ministry of Education

47 report the passing rates for each gender separately – note the consistently higher performance of girls over boys.

There were also differences in passing rates by school characteristics. The rates differed across the three types of public-primary schools in Jamaica, with the primary schools showing a slightly higher overall pass rate (63%) compared to the all-age and primary-and-junior high schools, which were both at about 55%. Notice here, however, that there are still consistent and large differences between the rates for girls and boys in all three of these school types, with gender differences being much larger than the smaller differences evident across school types. In looking at the impact of school sizes, note the relatively small differences in the overall passing rates. Interestingly, the smaller schools had slightly lower passing rates than did the larger schools. Comparing passing rates across the levels of socio-economic status of the school communities, we see that the higher-SES schools had higher pass rates (69%), with only a small difference between the middle-SES (60%) and low-SES (58%) schools. Finally, urbanization-level differences were quite small, with rural schools showing only a slightly higher rate (62%) compared to urban schools (60%).

My next step in exploring results from the initial sitting of the Grade-4 Literacy Test in

2010 was to examine variability in the passing rates across individual schools. I estimated the passing rate for each of the 792 schools, and present a sample histogram of these rates in

Figure 1 (top panel). While the average passing rate for schools is close to 60%, the histogram also shows how widely schools differ, with a few schools showing pass rates of

100%, while a few others had pass rates of 0%. Most schools were between 30% and 75%.

In Figure 1 (bottom panel), I show the sample relationship between school passing rates and the sizes of the schools. The relationship is not strong, but larger schools tended to have

48

Figure 1: (a) Sample histogram of estimated passing rates by school and (b) Sample relationship between school passing rates and school size (n=792)

49

higher passing rates. There were several small schools that had 100% of their students passing the first attempt, and some of the small schools did very well.

Overall about 60% of the students were able to pass the examination on the first occasion. In addition, about 6% registered but were absent. Thus, a third of all students had failing scores. As noted earlier, in my review of the Jamaican education system, the

Competence-Based Transition Policy (CBTP) allows children who fail on the first attempt to retake the examination in Grade 5 so that they may become eligible for promotion to high school. What happened to those students? How many of them recovered on the next try and became eligible? These are the students we will focus on in the rest of these analyses. I will present the results of my statistical analyses. We will see the rates of success for these students, and we will see which subgroups of schools seem to do relatively well in helping their students recover and become eligible for promotion. I will then report on my discussions with students and parents and teachers and principals to present some of their reactions and perspectives.

5.1.2. Did Students Who Failed Eventually Become Eligible for High

School?

In Table 5, I present the final logistic-regression model that portrays the fitted probability of becoming eligible for promotion to high school for boys and girls by school examination-cohort size and school SES, among the sample of students who failed the first sitting of the Grade-4 Literacy Test in 2010. I have included three columns in Table 5. In the first 50

Table 5: Parameter estimates, standard error and p-values from the final fitted probabilities of becoming eligible for promotion to high school for boys and girls, who failed the first sitting of the Grade-4 Literacy Test (GFLT), in schools of different examination-cohort sizes and in different communities (n=15,287)

Estimated Regression Parameter (standard error) p-value

Fixed Effects: Main Effects: Intercept -1.6864 0.000 (0.1201) Gender: Female 0.1992 0.000 (0.0482) Small school exam-cohort size* -0.9894 0.000 (0.2280) School socioeconomic status (SES): MidSES -0.1808 0.229 (0.1502) LoSES (poorest school community)** -0.1809 0.387 (0.2093)

Two-way interactions: MidSES x Small school exam-cohort size 0.9612 0.000 (0.2525)

LoSES x Small school exam-cohort size 0.8470 0.004 (0.2939) Random Effects: Between-school variance 0.3459 (0.0519) Intraclass correlation 0.0951 (0.0129) Goodness of fit: -2LL 12792.2772

Notes: * Schools with large and medium exam-cohort sizes are pooled as the reference category ** High SES schools (poverty level = 1), wealthiest school communities, are the reference category

51 column, I list the predictors in the model. In the second column, I list the parameter estimates and standard errors associated with each estimate, along with the associated p- values in the last column. I have organized the rows of Table 5 under two subheadings: Fixed

Effects and Random Effects. Within these sections, I have listed all predictors remaining in the final model and statistically significant interactions among them. In the fifth to ninth rows, I display parameter estimates associated with the main effects of student gender, school examination-cohort per school and school socioeconomic status (SES). In rows eleven and twelve, I present the fitted impact of the pair of terms that describe the two-way interaction between school examination-cohort size and school socioeconomic status. Within the random-effects category, in rows thirteen to seventeen, I list estimates of the school-level variance components, the intra-class correlation (ICC) and the deviance statistic (-2LL), a measure of model goodness-of-fit. The ICC was about 9.5%, suggesting that almost ten percent of the variation in the outcome, PROMO, occurred between schools. I discuss these different effects in more detail below, first presenting the impact of the main effect of student-level gender and then describing the joint effects of school-level socio-economic status and examination-cohort size.

Do Boys and Girls Who Have Failed the GFLT Differ in Their Eligibility for High School?

Recall from our earlier inspection of the initial results from the GFLT that there was a substantial difference in the passing rates for girls and boys. Now, we focus our attention on the girls and boys who did not succeed on their initial attempt. How did these students do on the retake?

The results in Table 5 indicate that recovery rates are low, regardless of student gender. First, notice that the effect of gender (FEMALE) is present only as a main effect in 52 the final fitted logistic-regression model. This signifies that, across all school locations and across all school examination-cohort sizes, there is a small and consistent gender-difference in the subsequent high-school eligibility of students retaking the GFLT, on average.

Specifically, the fitted odds that a girl, who failed her first attempt at the GFLT, will become eligible for promotion to high school is 1.220 (e!.!""#) times the fitted odds that a boy, who also failed the first sitting of the grade-four test, will do the same.

Is The High-School Eligibility Of Students Who Have Failed The GFLT Different If They Attended

Schools Of Different Sizes And Socio-Economic Status?

Due to the presence of the two-way interactions between school socio-economic status and examination-cohort size in my final fitted model, the unique impacts of these two predictors on high-school eligibility are difficult to interpret. Rather, their effects must be interpreted jointly. To facilitate this interpretation, in Figure 2, I provide a three-dimensional

(3D) graphical representation of the fitted probabilities of high-school eligibility, by both school examination-cohort size and socioeconomic status.

In Figure 2, I show that, overall, the recovery rates of students retaking the GFLT were disappointingly low, ranging from 7% to 17%. Across all socioeconomic communities, on average, students in larger schools had higher probabilities of becoming eligible for promotion to high school than did their peers in small schools. However, due to the presence of the statistically significant interactions between the predictors in the final fitted model, this examination-cohort size effect is not identical across schools of different socio- economic status. Overall, it was failing students who attended larger schools in the wealthiest communities who had the highest probability of becoming eligible for ultimate

53

0.165

0.142 0.180 0.142 0.160 0.140 0.139 0.120 0.125 0.100 0.069 0.080 0.060 0.040 0.020 Larger school 0.000

HiSES Small school MidSES LoSES

Figure 2: Graphical representation of the fitted probabilities of becoming eligible for promotion to high school for a student, who failed the first sitting of the 2010 GFLT, by school examination-cohort size and school SES (n=15,287)

54 promotion to high school, (16.5%). On the other hand, in the population of students retaking the GFLT, the students with the lowest probability of becoming eligible for promotion were those in the small schools that were located in the wealthiest communities

(6.9%), on average. Thus, the largest differential in recovery rates between small and larger cohorts is among students attending schools in the wealthiest communities.

For instance, among schools located in communities with high socioeconomic status

(HiSES), the fitted odds that a student in a large school will become eligible for promotion to high school is approximately 2.7 times the fitted odds that a student in a smaller school will become eligible for promotion, on average. In contrast, in the middle and lower SES communities, the differentials between these same types of schools are almost negligible, on average.

5.2. Qualitative Findings

In my descriptive quantitative analyses, I noted a substantial difference between the passing rates for girls and boys at the first sitting. Then, in interpreting findings from my analysis of recovery rates, I also see girls doing somewhat better than boys, but overall recovery rates are relatively low. I was somewhat surprised to see that small schools actually had slightly lower recovery rates, although this difference was not dramatic. The puzzling finding was that the role of SES was inconsistent, and I noted that smaller schools in high-

SES regions had the lowest recovery rates on average. Below, I report additional results from my interviews and discussions with students, parents, teachers and principals in

Jamaica. I believe their comments may help to shed some light on the findings from both the initial sitting and the retake of the GFLT.

55

5.2.1. Student Perspectives on the Grade 4 Literacy Test (GFLT)

During my discussions with the students, I asked questions about four major topics:

(1) What did students know about the GFLT? How familiar were they with the test and what it might mean for them? (2) Who informed the students about the test? Teachers?

Parents? Others? (3) How did they learn about how they did on the test? How were the results or scores communicated to them? (4) For those students who had failed the test in

Grade 4, I asked them about their feelings concerning their failure, and I also asked if they were preparing in any special way for the retake of the examination.

Five main themes emerged from my interviews and discussions with students about these topics. First, students differed widely in how much they knew about the GFLT, and it appeared that students who were better informed performed better and received higher scores on the test. Second, teachers seemed to be a crucial source of information for the students, but also parents and family members played a key role. Third, there was no single system or approach for reporting the results to students and families. Schools seemed to have very different approaches to this. Fourth, for students who failed in Grade 4, the delay in the consequences may have led to some confusion. Fifth, while I did not include questions about home language initially in my discussions with students, one theme that emerged as I interacted with them was the important role of skill with Jamaican Standard

English. I present a summary of these main findings in Table 6. I discuss them in more detail below, offering examples and explanations.

Student Knowledge of the GFLT

My initial intent was to conduct one focus group with fourth graders at each of the

56

Table 6: Main themes that emerged from interviews and discussion with students about the Grade-4 Literacy Test (GFLT) General Themes Findings Knowledge of the GFLT Student knowledge varied widely. Their knowledge was correlated with how well they performed on GFLT.

Sources of information about the test Students said teachers played a big role. Many students also mentioned parents or other family members.

Reporting of the results to students and There was no consistent policy for this. families Schools varied. Some schools read out the results in classrooms. Others conferred with students privately.

Delays in consequences may have been Students who failed, particularly those who confusing for some students failed again on the retake, were not aware of the true consequences. They thought they were still eligible for GSAT.

Issues around home language This is a theme that emerged as I spoke with students. While some of them were very comfortable speaking and learning in English, others were struggling.

57 three schools that I visited in May 2015. However, due to logistical difficulties, I was only able to do so at two schools – the primary and the primary-and-junior high. Each of these focus groups was composed of eight fourth-grade students. Most of them understood that they had to pass the GFLT or else, in the words of one child, “Miss, we don’t get to tek di

GSAT” [Translation: Miss, we won’t be able to take the GSAT].

When I asked them what would happen instead, none of them knew about the intervention program, Alternative Secondary Transitional Education Programme (ASTEP). In general, their knowledge of the consequences of failing the GFLT stopped at the grade-six examination.

When I asked the fourth graders who had told them about the GFLT, most of the primary-school students mentioned that their parents or an older sibling had told them about the test in Grade 3 and the rest indicated that they had learned about the test from their teachers, upon entering Grade 4. In contrast, all the students at the primary-and-junior high school told me that their teachers had informed them about the test when they entered

Grade 4. I am delighted to report that on my return trip to the primary school in November

2015, I discovered that all the children who had participated in my focus group at that school had passed the GFLT.

In my individual interviews with fifth graders, I met with some students who had passed their first attempt at the GFLT and some who had failed. Overall, the fifth graders who had passed their first sitting of the GFLT were knowledgeable about the test and its implications for advancement to the secondary level. All of them mentioned their parents, guardians, siblings as instrumental in first informing them about the test. On the other hand, I found mixed results when I interviewed fifth graders who had failed their first 58 attempt at the GFLT. First, students who got almost mastery2 on the examination knew more about the GFLT than their peers who got non-mastery; however, they knew less about the test than fifth graders who had passed on their first attempt. In fact, I found that children who got non-mastery hardly knew anything about the test.

Now, it would be remiss of me not to pause here and mention the one special case of a little girl who should have passed her first attempt at the GFLT but got almost mastery instead. She knew at least as much as her peers who had passed the first time and she was able to tell me exactly what part of the Writing Task #1 (WT1) subtest she had struggled with. The issue, she explained, was that she got hung up on that particular grey area and ran out of time. In fact, she had gotten high marks on both the Word Recognition (WR) and

Reading Comprehension (RC) strands of the GFLT. She eloquently expressed: (a) her dismay at failing the test, especially when her classmates taunted her that she was not that smart after all (she was a high-achieving student), and (b) her determination to “ace it” next time. She also shared with me that her mother had encouraged her that “sometimes these things happen and I must just buckle down and work harder.” I encouraged her to tell her teacher about the “glitch” in the WT1, just as she had done with me, so that he could help her. And she promised, “Miss, I'm going to ace it next time.” She did.

Second, few of the children who failed their first attempt at the GFLT reported that their parents encouraged them to do better next time. However, more of them told me that their parents were displeased and warned them that “if mi nuh pass next time, mi ah guh ah

2 Students must pass all three strands of the Grade 4 Literacy Test (GFLT) to be certified as literate (mastery). Those who only pass two of these strands (almost mastery) or pass one or none of these strands (non-mastery) are not certified as literate and must retake the GFLT, before they are allowed to sit the Grade 6 Achievement Test (GSAT) for placement at the secondary level.

59 dunce school” [Translation: if I do not pass the next time I take the GFLT, I will be sent to a school for dunce people].

Communication of GFLT Results

When I asked the fifth-grade students how the GFLT scores were reported to them, many of them informed me that the scores were read out aloud in class. For those who had passed, it was a crowning moment because the announcement “proved” they were smart.

As one little boy had stated, “It mek mi feel good, Miss, ah feel smart” [Translation: It made me feel good, Miss, I feel smart]. Unfortunately, the experience was not so positive for students who had failed. Most of them expressed sadness and embarrassment and admitted to being nervous about the retake of the GFLT. Some of them shared their experiences with being teased and taunted by classmates. In contrast, a few of them reported that their teacher had told them their scores privately. Therefore, they did not experience any embarrassment, only disappointment that they had not passed the test.

Mixed Messages: GFLT Failure and Promotion to Higher Grades

When I revisited Jamaica in November 2015, I was particularly interested in understanding how sixth graders, who had just been informed that they had failed their second and last opportunity to sit the GFLT, had been impacted by the news of their failure and whether they understood the implications for their future or not. To my dismay, I found that all sixth graders I interviewed believed that they were going to take the GSAT, despite having failed their last chance at the GFLT. They shared with me their top five high 60 schools of choice3 and expressed their hopes of getting into one of them. As appalled as I was, I understood their confusion. Unlike other high-stakes examinations, the consequences of failing the GFLT do not become apparent until the sixth grade, two years after the first administration of the test. Therefore, despite failing the GFLT, children continue to be promoted to higher grades, from Grade 4 to Grade 5 to Grade 6, without experiencing any dire consequences for failing the test. Furthermore, they are allowed to register for the

GSAT, conditioned on their performance on the retake of the GFLT4. Hence, given the mixed messages children received, it is not a stretch of the imagination to comprehend their confusion. Unless they are told explicitly, how else are they to understand the accountability process? Fortunately, one of the school’s guidance counselors was present in the interview room and he noted the urgency of informing and supporting these children before the next administration of the GSAT in March 2016.

The Importance of Home Language and Exposure to Formal English

Considering all the students I interviewed, I noticed that they fell into three categories: (a) those who understood Jamaican Standard English (JSE)5, (b) those who understood it somewhat, and (c) those who did not understand JSE adequately. I discovered that students who got mastery on the GFLT were fluent in the JSE. Meaning they understood my questions and were able to engage in a meaningful conversation. The second group of students, however, didn’t quite understand every word of English that I spoke.

3 Previously, when registering for the GSAT, students could choose their top 5 high-schools to attend. As of 2016, students now have the opportunity to choose their top 7 high-schools. Five can be located anywhere on the island; however, two must be in close proximity to children’s current primary school (Jamaica Observer, 2015). 4 GSAT registration occurs before GFLT scores are published. Therefore, all fifth-grade students are allowed to register for the examination at the beginning of the school year. For those who are waiting for GFLT results, they do so probationally. 5 The Jamaican Standard English is similar to the British English. 61

Therefore, early on in the interviews, I switched to “patlish,” a combination of JSE and

Jamaican Creole, patois. Once I did that, I was able to engage the students and garner relevant information. Below, I share an excerpt of such a conversation with a fifth-grade boy. To protect his identity, I have called him Roger:

Me: How you did on the test? Roger: (sighs) Miss, I didn’t like it at all.

Me: Why not? Roger: Miss, mi neva really undastan wat they were asking mi. [Translation: Miss, I didn’t really understand what they were asking me]

Me: Roger, there was nothing at all on the test you liked? Nothing at all? Roger: (after thinking a couple seconds) Miss, di one wid di picha. Weh yuh match di picha wid a word. [Translation: Miss, the subtest with pictures that you have to match with a word.]

Me: You mean Word Recognition? Roger: Yes, Miss. That one was ahright. Mi do okay pan it. [Translation: Yes, Miss. That one was all right. I did okay on it]

Me: What about reading? [Translation: How did you perform on Reading Comprehension?]

Roger: Miss, most time mi neva know wat they were asking mi. Some of dem mi understood and did. But a lot of dem, me neva know wat they were asking mi, Miss. [Translation: Miss, most of the time, I did not understand the questions. Some of them I understood and was able to complete them. But I did not understand most of them.]

Me: So what did you do? Roger: Miss, mi jus fill een a ansah [Translation: Miss, I just filled in an answer]

Me: (I just looked at him in askance)… Roger: Miss, yuh know eeni meeni myni mo? Mi jus pick a numba. It betta dan mi get zero, don’t? [Translation: Miss, you know, “eeny meeny miny mo?” I simply picked a number from the multiple-choice list. It’s better than getting a score of zero, right?]

The third group of students did not have an adequate grasp of Jamaican Standard

English. With these students, I had to dig deep into my own cache of patois, which I have been told is “old patois.” Here is an excerpt of a conversation I had with another fifth-grade 62 boy. Before switching to patois, one might have thought he was learning disabled; however, when we started conversing in patois, he lit up like a light bulb. An excerpt of our conversation follows (again, I have used a pseudonym):

Me: Bobby, are you retaking the grade-four test next June? Bobby: (pause)… Miss, mi nuh undastan [Translation: Miss, I do not understand what you’re saying.] Me: Okay… (I paused and regrouped)

Me: Bobby, sarry seh mi neva taak good. Mi gwine try again, yuh ear? [Translation: Bobby, I'm sorry that I didn’t speak clearly. I'm going to try again okay?] Bobby: Okay, Miss. Me: Tanks. Yuh tek di test ahready last year? [Translation: Thanks. Did you take the test last year?] Bobby: Yes, Miss. Me: Wah yuh get? [Translation: what was your score?] Bobby: Miss, mi fail, mi get non-maastry. [Translation: Miss, I failed, I got non-mastery.]

Me: How yuh feel bout dat? [Translation: how do you feel about that?] Bobby: Bad. Mi neva like fi fail, Miss. [Translation: Bad. I didn’t like failing, Miss.]

Me: But yuh ah guh try again right? [Translation: But you’re going to try again right?] Bobby: Yes, Miss. Me: Yuh ready fi it? [Translation: Are you ready for the examination?]

Bobby: Ah tink suh, Miss. [Translation: I think so, Miss.]

Me: Yuh TINK suh? [Translation: You THINK so?] Bobby: (laughs) Mi KNOW suh, Miss! [Translation: I KNOW so, Miss!]

I believe these two examples serve to illustrate the challenges some students in

Jamaica face when the language of instruction differs from their home language. I’ll return to this issue in the conclusions section. 63

5.2.2. Parent Perspectives on the Grade-4 Literacy Test (GFLT)

I conducted two focus groups with parents. One group was composed of parents whose children had passed the GFLT and the other of parents whose children had failed the

GFLT. I wanted to explore if there were any differences between them and whether these differences could explain what I had observed in my discussions with students and found in my quantitative analyses. In general, I found small differences between the two sets of parents who participated in the focus group discussions. Five main themes emerged from my interviews and discussions with parents: (a) parents knew enough about the GFLT; (b) parents did not introduce their children to the test; however, most gave pep talks leading up to test day; (c) parents had mixed reactions to test results; (d) they unanimously disagreed with how scores were reported to students and families; and (e) parents were frustration with lack of discussion about students’ test results. I present a summary of these main findings in

Table 7, and discuss them in more detail below, offering examples and explanations.

Parent Knowledge About the GFLT

Overall, I found that both sets of parents were very involved in their children’s education, which is not surprising since the school administrator had selected parents most likely to participate in the study.

For the most part, parents were able to describe the content covered in each strand of the test, although they did not know the proper names for the strands (Word Recognition,

Reading Comprehension, Writing Task). They all knew about the Alternative Secondary Transitional

Education Programme (ASTEP). In particular, parents of children who failed the first sitting of

64

Table 7: Main themes that emerged from interviews and discussion with parents about the Grade-4 Literacy Test (GFLT) General Themes Findings Knowledge of the GFLT Parents knew enough about the GFLT, its consequences and implications for their children's future.

Parent-Child Discussion about the GFLT Parents did not introduce their children to the GFLT; however, most of them gave their children pep talks before the examination.

Parent Reactions to GFLT Results Parents had mixed reactions to test results. All were proud of their children for doing their best. Some parents, however, were displeased with the almost mastery classification because it served no purpose.

Parent Reactions to Score Reporting Parents unanimously agreed that the way in which scores are communicated needed to be improved. Schools should notify parents of results before informing students.

Parent-Teacher Discussions About GFLT Parents were frustrated with the lack of Results discussion about their children's performance on the test. They wanted to know more so they could do more.

65 the GFLT expressed grave concerns about their children possibly participating in the program. They claim that even though some ASTEP students completed the program successfully and were able to matriculate at “good” high schools, the numbers were very low. One parent already had a child participating in ASTEP and he shared that he had to implore his son’s teacher to provide some individualization of the instruction:

Tek di time fi get fi know im, fi help im, bicaa im is a different learner. Ah beg di teacha. An now, mi son is doing betta. It supposed to be a remediation program, not the same-o same-o like inna di regula school.

[Translation: Take the time to know my child, to help him, because he is a different learner. I begged the teacher. And now, my son is doing better. It is supposed to be a remediation program, not the same-o same-o as it is in the regular school.]

Parent-Child Communication About the GFLT

When I asked parents how, if at all, did they first introduce the GFLT to their children, most of them had not discussed the test with their children explicitly. However, more of them admitted to giving their children pep talks before the test. It may be that for many parents in Jamaica, they see the schools as having primary responsibility for the education, and parents have a more limited role. They often interact in only limited ways when it comes to the education of the child.

Parent Reactions to Children’s Scores

Overall, parents agreed that their children mostly struggled with comprehension, “it’s the understanding part of it holding them back.” Parents whose children had passed the

GFLT were relieved that their children had succeeded. Some of them even expressed surprise that their children had done well. One of the mothers, laughingly revealed, 66

Mi suhprize mi son pass di test caw im nuh like school. Mi suh glad im pass.

[Translation: I was surprised that my son passed the test because he does not like school. I’m so glad he passed.]

Conversely, parents of children who had failed were disappointed. Particularly, parents whose children had received almost mastery felt “ripped off” by this classification.

They felt that the outcome for receiving almost mastery was the same as that for receiving non- mastery: students had to retake the GFLT. So what was the point? As one parent said,

Wi nuh stupid. Is di same ting as non-maastry. Dem might as well mek it ongle two results, maastry and non-maastry, becaas almost maastry is foolishniss.

[Translation: We are not stupid. It (almost mastery) is the same thing as non-mastery. They might as well make it only two results, mastery and non-mastery, because almost mastery is foolishness.]

In fact, one parent admitted she was “very vexed” when her child failed by one point,

One likkle mistake in har letta writing… one likkle mistake and shi fail di whole exam.

[Translation: one little mistake in her letter writing… one little mistake and she failed the entire examination.]

Interestingly, despite their disappointment, some parents whose children had failed admitted they felt good. They know their children worked hard and did their best. One mother revealed that at first she had doubts about her son’s ability to pass the GFLT and she “couldn’t believe it when he failed by only one point.”

67

GFLT Scores Communication

Parents disagreed unanimously with the way in which scores were communicated.

Children were informed of test results before their parents were notified and, in some schools, scores were read out aloud in class. One parent reported that her daughter came home crying because she had failed the test and her classmates were calling her dunce. Her daughter already did not believe she could pass the retake next time:

Di teacha dem don’t tell di kids its non-maastry or maastry. Dem call out dem name and sey “yuh pass… yuh no pass. Dem nuffi do dat! It urt di pickney dem!

[Translation: The teachers do not tell the children that they got non-mastery or mastery. They call out their names and say “you pass… you did not pass. They should not do that! It hurts the children!]

Another parent shared that as she was walking across the schoolyard to attend a consultation with her son’s teacher, one of her son’s classmates yelled out that her son “neva pass.” Another parent reported the following:

Mi get inna di schoolyaad and children a tell mi seh mi dawta was crying todeh. Mi ask, wah wrang wid har? Dem seh, Miss seh shi nuh pass. Mi nuh feel dem fi tell di kids dem, at least nuh like dat.

[Translation: I got in the schoolyard and children were telling me that my daughter was crying today. I asked them why she was crying. They told me that the teacher had announced that she had not passed the test. I don’t feel they should tell the children, at least not like that.]

The parents all agreed that scores are personal information that could harm their children and, therefore, should be confidential. As one parent suggested, “It backwud!

Contak di parents wid di infamation and let di parents dem tell dem children.” [Translation:

It’s backwards! Contact the parents with the information and let the parents tell their children.]

68

Parent-School Discussions About GFLT Results

Parents of children who had failed their first attempt at the GFLT were generally dissatisfied with the lack of information about their children’s performance on the test.

Mi feel dem need fi explain to mi wat is di problem wid my child. Explain to mi more suh dat mi can undastan weh mi child fell down. Dem only seh dat “one mark mek dem nuh pass” – no adda explanation. Mi waah fi know which part im guh wrang, which part im need fi improve, which part im weak…

[Translation: I feel they need to explain to me the problem with my child. Explain to me so that I can understand where my child fell short. They only say that “he missed by one point.” - no other explanation. I want to know where he went wrong, where he needs to improve, where he is weak…]

When I asked the group what they would do with the additional information, they asserted they would try to rectify the situation and get their children the help they needed

“come what may” because “dem fyucha is pon di line!” [Translation: Their futures are at risk!]

5.2.3. Teacher Perspectives on the Grade-4 Literacy Test (GFLT)

During my research visits to three public-primary schools in Jamaica, I interviewed twelve teachers in the upper-primary grades, grade 4 to grade 6. Five themes emerged from our discussions that could help explain the low recovery rates of students who retook the

Grade-4 Literacy Test (GFLT) in Grade 5: (a) student entry-level proficiency, (b) parent involvement, (c) test preparation for the GFLT, (d) communication of scores to students and families, and (e) intervention strategies in Grade 5. In Table 8, I present a summary of my findings from the teacher discussions, and I discuss them in more detail below.

69

Table 8: Main themes that emerged from interviews and discussions with teachers about the Grade-4 Literacy Test (GFLT) General Themes Findings Student Proficiency Issues Some fourth graders were at or below Grade-1 level. Many students had physical, mental and learning disabilities, which held them back.

Parental Involvement Issues Teachers reported low levels of parental involvement. Schools offer workshops and other programs to educate and engage them. However, the problem persists.

Test Preparation Preparations began at the start of the school year in September, with a special focus on Reading Comprehension. Children were made to practice a lot to familiarize them with the GFLT.

Score Reporting Strategy Schools received results from the Ministry of Education on CDs. Slips with results were prepared for students or parents. However, some teachers admitted to calling out results in the classroom.

No Formal Remediation Programs Students who had failed their first attempt at the GFLT in Grade 4 received no intervention in Grade 5.

70

Student Proficiency

Teachers reported that they have students who are very weak, whose “reading level is not up to par.” Some of these fourth graders are at, or below, Grade-1 level because they either had not attended basic (pre-primary) school or had received inadequate education there. One strategy that some primary teachers have employed is getting children’s reading up to at least the Grade-2 level, which would make them just proficient enough to pass the

GFLT.

Another challenge teachers were also confronted with is the physical, mental, and learning disabilities of some of their students, which make them “very, very slow learners who are not fit for [the current education] system.” They further argued that the Ministry of

Education is aware of these challenges but have yet to do anything about the situation6.

Consequently, teachers continue to struggle in meeting the educational needs of these students. Many expressed frustration with having to observe their special-needs students sitting the GFLT multiple times and knowing they did not have a chance of passing the test.

In response to my query about student knowledge of the GFLT, all teachers reported that some students understood the importance of the test, while others did not, especially those with developmental delays and learning disabilities.

Parent Involvement

All teachers complained that parent involvement, or the lack thereof, is one of the major causes of student underachievement. Some examples of this dilemma include parents not monitoring their children’s academic progress, not assisting them with homework, and

6 This is as of May 2015. 71 not sending their children to free extra lessons or holiday classes offered by teachers. As one teacher explained,

Parents are partly to blame… if your child completes Grade 1 and is not reading, cannot identify things, is not writing, seek help! Teachers in the earlier grades may say to them “go and get your child assessed, make sure he or she is not disabled.” But they don’t. So their children reach Grade 4 and we are expected to work a miracle. We cannot. We need parents to follow through.

When I asked teachers what could explain parents’ resistance to having their children assessed, they said collectively that parents believed there was a stigma attached to having their children certified as “special needs.” In response to this pervasive issue, schools have offered parental workshops in an attempt to educate parents on child development, learning disabilities, and strategies they can use to better support their children. Another type of parental workshop offered by some schools gives parent first-hand experience with the

GFLT. Parents sit a mock examination under the same conditions as the real test. Teachers hope the experience will help parents to understand how they can assist their children in preparing for the test at home. Yet, even now, enough parents are not playing their part.

All teachers identified parent involvement as a critical factor in academic achievement. One teacher expressed it well,

If parents would do their part, things would improve. It starts at home. Parents have to instill in their children the value of education. We instill it here and then they go home and it changes again. And you know, the children whose parents work with us are doing excellent. So there is evidence that parent involvement is key.

Test Preparation

In preparing students for their first attempt at the GFLT, grade-four teachers reported preparing children for the test from the start of the school year in September. They taught students “how the test is set up and then [they] give a lot of practice.” Teachers 72 focused especially on Reading Comprehension because most children were failing that strand.

At the time of the interview in May 2015, one month before the administration of the

GFLT, teachers were giving children mock examinations every Friday – exact replicas of the real GFLT.

GFLT Scores Communication

The Ministry of Education (MOE) releases GFLT scores on compact discs. Upon receipt, school administrators make hardcopies of the results and distribute them to the relevant teaching staff. Then, little slips of results are prepared for each student to be delivered to them or their parents. Certificates from the MOE are also given to students who pass the GFLT. However, many grade-five teachers admitted to calling out student names with results, in class.

Remediation Programs

Across the schools I visited, none of them had an intervention program implemented in Grade 5, to rectify issues arising from the grade-four high-stakes testing.

There was no streaming of students and test repeaters continued on to higher-level learning, without ever encountering any specific strategies to address their areas of difficulty. For example, teachers mentioned keeping to their regular class schedules and then, closer to the examination date, some teachers gave more attention to these students by offering them extra lessons after school.

73

5.2.4. Principal Perspectives on the Grade-4 Literacy Test (GFLT)

Five main findings emerged from my interviews with three Principals and Assistant

Principals across the three public-primary schools. These findings covered issues related to

(a) school funding, (b) availability of literacy specialists, (c) school knowledge of GFLT, (d) parent involvement, and (e) remediation programs. I present a summary in Table 9, and discuss these themes in greater detail below.

School Funding

All government schools are funded by grants that are based on the size of the student population. Therefore, schools with a small student population receive similarly small government grants. Additionally, when one considers the context of schools, such as the demographics of the population of children served as well as the urban-rural designation of the school community, it is not farfetched to imagine the dire straits these schools are in.

As one Principal shared,

When I came to this school, we had little or no resources to work with. Little or no money. The Ministry sends some money but sometimes it’s still not adequate to do what we need to do for our students. You see, our students are very slow. They face many challenges in every area of their lives. And we need to do what we can to increase their chances of future success. We need adequate resources for that.

Literacy Specialists

Literacy specialists are assigned to schools that are underperforming. However, the number of underperforming schools far exceeds the number of literacy specialists available and “everything is based on the budget.” Therefore, there are schools “waiting for their turn,” while their students continue to underperform in the Grade-4 Literacy Test.

74

Table 9: Main themes that emerged from interviews and discussions with principals about the Grade- 4 Literacy Test (GFLT) General Themes Findings Government Funding Issues Government schools are funded by government grants, based on the size of the student population.

Insufficient Literacy Specialists Not enough literacy specialists to assist all the underperforming schools in the nation. Schools are “waiting for their turn” to get a specialist sent to their schools.

Whole-School Knowledge of GFLT The Ministry of Education informs principals via bulletins and principal meetings. School grade coordinators are trained by the Ministry and then return to school to train other teachers. Parents are informed through consultations and PTA meetings. Finally, parents and teachers alike inform students.

Parental Involvement Issues Lack of parental involvement is pervasive in the Jamaican school system.

No Formal Remediation Programs Students who had failed their first attempt at the GFLT in Grade 4 received no intervention in Grade 5.

75

Whole-School Knowledge of GFLT

Principals were initially informed about the GFLT through bulletins from and principal meetings at the Ministry of Education. In addition, grade coordinators from each school had attended training sessions. And these coordinators then returned to their respective schools where they trained colleagues during grade-level and staff meetings.

However, as one principal noted,

This was not enough because people learn differently and some of my staff would have been better served had they attended training sessions conducted by the Ministry either at the Ministry or here at the school.

Schools also bore the responsibility of informing parents and children about the test.

Guidance counselors and teachers met with parents through consultations and PTA meetings to inform them about the GFLT and to explain the consequences associated with the newly transformed test7. Unfortunately, parental participation in these events was less than desirable. After these meetings, parents and teachers alike informed students about the new testing regime.

Parent Involvement

Some parents are not involved in their children’s education enough. Some of them are completely “checked out.” One major challenge that school leaders continue to grapple with is the low levels of parent involvement in their children’s education.

Parental involvement is a major challenge we face every day. We make home visits; we invite parents for workshops and consultations; and we offer them other services to the best of our abilities. And they do not all participate. Of those that do participate, some of them “commit” to do better and then do nothing. Meanwhile, their children are languishing in our classrooms with various disabilities and other issues. This disconnect between school and home has a detrimental effect on student learning.

7 The Grade 4 Literacy Test (GFLT) had been transformed from low-stakes to high-stakes in 2009. 76

How to reach my parents, that is the question?

Remediation Programs

There are no specific guidelines for remediating children who fail their first sitting of the Grade 4 Literacy Test (GFLT). Schools that are underperforming could be assigned a literacy specialist; however, this depends on the budget. Schools that are in good standing have to find their own resources and means of providing intervention for students who failed the test.

5.3. Linking the Qualitative and Quantitative Findings

While people in Jamaica seemed to be generally aware of the new testing policy, there were clearly shortcoming and inconsistencies in how the new policy was implemented.

Very little attention was given to remediation efforts for students who failed the first sitting, and parents reported dissatisfaction with how the results were being used by the schools.

They saw little use of the test results to guide or modify instruction for their children.

Teachers reported feeling pressure to narrow their teaching to help children prepare for the standardized assessments, yet they were not given additional time or resources to assist their students. They also reported having limited resources for remediating students who were struggling. The principals’ perspectives and insights were helpful here as well. The principals explained how the funding mechanisms are primarily based on school enrollments. Small schools ended up with lower levels of resources, even though at times the students in these small schools have high learning needs. Especially in small schools, there is little room in the budget for literacy specialists, for example, or additional staff who could provide remediation. 77

These qualitative findings help us understand some of the trends noted in the quantitative analyses presented earlier. One main finding was that the overall recovery rates were low. Only between 7% and 17% of the students who failed initially were able to pass on the retake. These low percentages are disappointing, but they should not be a surprise in light of the very limited remediation efforts that were implemented. There was apparently no system in place to help the schools and teachers make use of the GFLT results to improve instruction. Few adjustments were made by schools, and very little was done for individual students.

In addition, reports from the principals about the funding system may provide a way to understand the anomalous finding from the statistical analyses showing that smaller schools in high-SES locations tended to have lower recovery rates. This came as a surprise.

The principals have explained, however, that small schools receive less funding through the system of government grants. Small schools in wealthier communities may be at a particular disadvantage, because families may have other choices and options that are available in larger schools in those communities. For schools in lower-income locations, however, the options to the small schools may be more limited. In other contexts, researchers have often found that students in smaller schools have some advantages, and overall achievement for students in large schools tends to be slightly lower (Voyer & Voyer, 2014). But in the Jamaican context, I noted a trend in the other direction, with larger schools having slight advantages.

The principals have pointed out a possible explanation. These explanations may not be correct, of course, and we do not have enough detailed information to resolve the issues, but the suggestions from the discussions with school principals give us some intriguing ideas for additional research. 78

Another major trend in the quantitative results was the large differences between girls and boys. In the initial GFLT results there were large differences in the passing rates, and the differences persisted when the students who failed initially retook the examination.

Issues related to gender gaps in achievement have received attention in Jamaica in recent years, so these findings are consistent with other trends. When I conducted my interviews, gender differences were not a focus of my discussions. Perhaps as a result, people did not comment on this or give ideas that might explain what we observed in the quantitative analyses. This should be an active area for future research.

79

Chapter 6: Discussion

In this study, I investigated student performance on the initial sitting of the Grade-4

Literacy Test (GFLT), which was administered as a high-stakes test beginning in 2009. Prior to that year, the GFLT was an important examination that was intended to be used for classroom assessment. But, in response to concerns that students were entering secondary schools with very limited skills, the Jamaican Ministry of Education implemented the new policy of making a passing score on the GFLT a requirement for eligibility for secondary school. In order to be eligible for the Grade-6 Achievement Test (GSAT), which is the examination that is used for admission to public secondary schools in Jamaica, students are now required to achieve a passing score on the corresponding grade-four examination, either on their initial attempt or on a retake opportunity in Grade 5.

Using nationwide data made available by the Ministry of Education, I focused on

46,120 students attending 792 public schools in Jamaica who first took the GFLT in the

2010 academic year. Several main trends emerged. First, passing rates on the initial sitting in

Grade 4 and the retake in Grade 5 were low overall, although there was substantial variability across schools. Second, there were large and consistent gender differences, with girls showing higher levels of performance. Third, interesting differences emerged that were related to school SES and school size.

I also engaged in a set of discussions with students, parents, teachers, and principals of three Jamaican public schools, and asked them about their experiences with the examination and the new policy. Some of their responses and recommendations provide interesting perspectives and enrich the interpretation of my quantitative findings. They 80 commented on how limited school resources seem to contribute to the low levels of performance. Another issue that emerged was the important role of students’ home languages, and the big challenges many students face in using Jamaican Standard English in school.

6.1. Evidence of Low Performance Levels on Initial Sitting And on Retake

On the initial sitting of the GFLT, the overall student passing rate was only about

60%. Some students were absent, and over one-third received failing grades. On the retake examination in Grade 5, the pass rates (what I have referred to, throughout my thesis, as recovery rates) were disappointingly low – consistently less than 20% for most subgroups of students.

What could account for these disappointing results? During discussions with principals and teachers, two themes emerged consistently. First, resources for all public schools in Jamaica are limited. Many schools are not able to make use of literacy specialists, for example, and class sizes are typically large (commonly 40 or more students per class).

Teachers and principals often mentioned the challenges they face in attempting to prepare students for the national tests, both initially and on retake. They feel frustrated with the limited resources they have available to meet the big challenges.

Teachers also mentioned that they worked hard in Grade 4 to prepare students for

GFLT. They reported that they tailored the curriculum and spent substantial amounts of time throughout the school year on test preparation. But, apparently, there was no corresponding plan for remedial support for students who had to retake the test in Grade 5.

The plan was to focus time and attention in Grade 4, but no additional resources were 81 devoted to the students who would need extra help. The limited efforts at remediation may have played a major role in the relatively low recovery rates for test retakers.

In addition to the resource constraints and limited planning for remediation that were mentioned by teachers and principals, in my conversations with students I became aware of another important factor that is undoubtedly contributing to the relatively low levels of performance I detected in the quantitative analyses – that is, the role of students’ home language.

The common vernacular spoken throughout Jamaica is Jamaican Creole, commonly referred to as patois. In schools and formal settings, however, Jamaican Standard English is used. Students’ exposure to and skill in English varies very widely. Some students grow up in households where both English and patois are spoken. But many students are exposed only to patois at home and in their local communities, and it is only once they begin formal schooling that they are confronted with the challenges of mastering formal English. In my conversations with students about their experience with the GFLT, I noted a clear pattern.

The students who had done well on the initial GFLT were relatively fluent in English. They understood my questions posed to them in English, and they were able to engage in a comfortable conversation. Another group of students who had generally achieved more moderate scores tended to be more mixed in their English-language fluency. Generally, they could follow and respond to my questions in English, but I sometimes had to engage in code switching and use a mix of patois and English (“patlish”). A surprising large proportion of the students that I interviewed showed very limited mastery of English. They often did not seem to understand the questions that I posed to them in English, and I switched to patois to continue the conversations. These students’ limited English skills were a major stumbling block for them. 82

The policy in Jamaica is that Standard English is the language used in schools, and there are no provisions made for teaching English as a second language or for supporting students who have limited English skills. Thus, this may certainly be a major factor contributing to poor academic achievement in schools throughout the country.

6.2. Gender differences in GFLT performance

There were large differences in the performances of girls and boys, and these were noted on the initial sitting of the GFLT as well as on the retake. On the initial sitting, the differences were dramatic. Girls had an initial passing rate of about 70%, while the rate for boys was just under 50%. These are very large gaps, but large differences between girls and boys in school achievement have been noted in Jamaica over the past two decades (Bailey,

2004/2009; USAID, 2005; Figueroa, 2004). The gender differences in Jamaica are much larger than gender differences found in most U.S. settings (Voyer & Voyer, 2014). The gaps on the initial sitting of the GFLT between girls and boys appeared to be relatively consistent across all types of schools.

Looking at performance on the retake, I also noted statistically significant differences between girls and boys, although here the magnitude of the gender difference was not as large. In looking at the probability of becoming eligible for promotion to high school, the fitted odds for girls was 1.22 times the odds for boys. These differences emerged as a main effect only, and gender did not appear in any of my tests for the presence of statistical interactions. This signals a consistent pattern in the recovery rates, with girls performing better. Lastly, the gender differences noted in my analyses appear to agree with other research from Jamaica (Clarke, 2005; Figueroa, 2004; Evans, 1999). I therefore consider this 83 an important issue that I recommend be the focus of careful thinking and research in the future.

6.3. School SES and School-Size Differences on the GFLT

It was interesting to find that the larger schools tended to have slightly higher levels of performance. I expected to find that smaller schools would have an advantage. Research on U.S. schools tends to show a slight small-school advantage for elementary schools.

Results are more mixed for secondary schools (Lee & Smith, 1997; Fowler & Walberg,

1991).

In my findings, results from the first sitting of the GFLT showed a slight trend towards students in larger schools doing better. On the retake, I once again found that students in larger schools had a higher recovery rate. Perhaps an important factor here is the issue of school-funding formulas. All public schools in Jamaica are financed through government grants, and these are based primarily on student enrollments. Larger schools have access to more funding and, according to the schools principals that I interviewed, schools try to enroll as many students as possible so they can take advantage of the greater funding that will result. The situation in Jamaica makes it less likely that a “small-school advantage” would exist. (In fact, several of the highest performing schools in my dataset were small schools, but the overall trend shows small schools doing less well.) Small schools have to work within smaller budgets. It may take special circumstances for small schools to do well in Jamaica.

In additional to school size, I also found that school SES played an important role.

However, I did not see large and consistent differences related to SES. Studies in the U.S. tend to show large disparities across schools with different levels of SES (Sirin, 2005). The 84 differences across schools in Jamaica were more modest. In particular, I noted that smaller schools in higher-SES locations tended to do less well, while the larger schools did very well indeed, when compared to all others. This emerged as a statistically significant interaction between SES and school size in my final statistical model of the recovery rates. This finding was an anomaly, but perhaps the funding mechanisms in Jamaica play a role. Recall that small schools are at a disadvantage, and the disadvantage may become even greater for small schools in high-SES locations. Perhaps students in those high-SES communities have other education options. I recommend that these differences become the subject of future research.

My examination of the impact of SES on recovery rates was limited, however, because of the way that I had to develop information on SES. I did not have information about individual students’ SES levels. Instead, I obtained maps from the Ministry of

Education with school locations superimposed on regions demarcated by poverty levels. I used these to create an SES code for each school. But the measure I obtained is based only on school location and the estimated SES level for the general region. A better measure would be based on student-level or family-level information, but such information was not available to me.

85

Chapter 7: Conclusion and Implications

It’s been eight years since Jamaica started its national development campaign. Even though there have been large-scale improvements across all sectors of the economy, results have been mixed: fluctuating economic growth, increased poverty and crime levels, and academic underperformance at all levels of the education system. In this study, I focused on student performance at the primary level of the Jamaican education system, in particular in the public-primary schools. Recently, the educational system has made some improvements.

However, it falls short of where the system needs to be in order to increase labor-force skills, which will improve not only national productivity and income level but also the standard of living of the citizenry, as a whole. My findings suggest that much work remains to be done in addressing the persistently low performance of students at the primary level. There is a significant gender disparity against boys across all public-primary school types (primary, primary-and-junior high, and all-age) and across all socioeconomic levels of the schools they attend.

7.1. Implications

Undeniably, the Jamaican government has provided good access to primary education for all Jamaican children. However, the challenge has been to improve the quality of the primary schools that students now attend and to raise their achievement levels. The results of my analyses suggest that much work remains to be done. The large and persistent 86 gender differences in achievement and recovery rates should be addressed through a variety of strategies that may involve social, political, and cultural changes going beyond the education sector.

Current policies are aimed at raising the standards and levels of school achievement, but the implementation has had some serious shortcomings. Attention and effort has been focused on fourth-grade preparation, but little consideration has been given to using the results to help support the students who have failed. Remedial efforts are very limited. The schools need more guidance and support, if they are to get more students back on track.

Finally, I call attention to the issue of home language and the major role it appears to be playing, particularly for students who have limited exposure to Standard English at home. I believe that expanding opportunities for language development, and focusing on academic language important for school, will have major benefits for children as they move through the education system. Other countries, such as Singapore, may have developed policies that could be customized and made applicable in Jamaica (OECD, 2010; Dixon, 2005).

I hope that my findings will inform the country’s current reform efforts to improve literacy levels among Jamaican primary schoolchildren. Without basic literacy, Jamaican schoolchildren will not be able to advance through the education system to become “well- rounded and qualified individuals… able to function as creative and productive individuals in all spheres of our society and be competitive in a global context” (Planning Institute of Jamaica, 2009, p.93).

87

Appendices Appendix A: The Jamaican National Assessment Programme (NAP)

Grade-One Individual Learning Profile (GOILP)

The GOILP is designed to measure students' academic progress and their social readiness for primary school. It is administered to students prior to the start of Grade one. Some of the skills measured by the GOILP instrument are reading readiness, numbers, concepts, oral language, writing and drawing.

The Grade-Three Diagnostic Test (GTDT)

At the end of Grade three, the GTDT in Mathematics and Language Arts are administered to students. The Mathematics test assesses students’ skills in using basic fractions reading instruments such at the clock, thermometer and ruler, identifying geometric figures such triangles, squares and polygons, etc. The Language Arts test assesses students’ ability to read and understand simple stories, factual texts, maps, labels and other documents. Students are also tested on capital letters and basic punctuation marks used in Standard Jamaican English.

Grade-Four Literacy and Numeracy Tests (GFLT & GFNT)

At the end of Grade four, the Grade Four Literacy and Numeracy Tests are administered to children to determine their literacy and numeracy levels at Grade four in keeping with international requirements and standards.

Ø The GFLT measures three sub-skills – word recognition, reading comprehension

and writing. The test determines students’ ability to recognize words, read and 88

understand simple stories, factual text, directions, list and tables. It also verifies their

ability to write simple stories, reports and letters. The GFLT forms part of the

Competence-Based Transition Policy being implemented since 2009, which allows

for students to be certified literate before being allowed to sit the Grade Six

Achievement Test (GSAT).

Ø The GFNT measures six sub-skills - number representation, number operation,

measurement, geometry, algebra and statistics. The test assesses students’ ability to

master these mathematical concepts and principles.

Grade-Six Achievement Test (GSAT)

Near the end of Grade six, children are required to sit the Grade Six Achievement Test

(GSAT) in Mathematics, Language Arts, Social Studies, Science and Writing

(Communication Task). It is the assessment instrument used by the Ministry of Education to place students in secondary/high schools. Twenty percent of the questions on the GSAT come from work developed for grade four, thirty percent from work covered in grade five and fifty percent from material designed for grade six.

Adapted from the Jamaican Ministry of Education, http://moe.gov.jm/about.

89

Appendix B: Schematic of How the Competence-Based Transition Policy (CBTP) Works

90

Appendix C: Grade-4 Literacy Test (GFLT) Test-Item Structure

The Jamaican Grade-4 Literacy Test is comprised of three subtests: Word Recognition,

Reading Comprehension and Writing Task. The Word Recognition subtest (Figure C.1) is used to assess student proficiency in word identification. It is divided into two sections, each consisting of 20 questions. Students are given five minutes to complete each section. In one section, students are required to identify a word that matches a picture; while in the other, they are required to identify a picture that matches a word. The maximum score that a student can obtain on this strand is 40 points. In the initial sitting in 2010, the average score on Word Recognition was 36.1 (SD = 9.2).

The Reading Comprehension subtest (Figure C.2) of the grade-four assessment assesses students’ ability to read and construct meaning from written text. This subtest is composed of a combination of exercises ranging from short poems to story excerpts to diagrams.

Students must complete 30 questions in 40 minutes. In the sample of grade-four students who took the test in 2010, scores on the Reading Comprehension subtest had an average of 18.0

(out of 30), with a standard deviation of 7.2.

Finally, the Writing Task subtest (Figure C.3) of the grade-four assessment measures student’s ability to engage in thoughtful deliberation and to express their ideas and knowledge on a particular topic. It is divided into two tasks, one requires students to complete or fill out a form and the other requires them to write an essay, story or letter.

Students have 40 minutes to complete this strand and can score a maximum of 8 points.

The average 2010 score was 5.1 (SD = 1.7).

91

Figure C.1: Sample of Word Recognition strand of the Grade-4 Literacy Test

92

Figure C.2: Sample of Reading Comprehension strand of the Grade-Four Literacy Test

93

Figure C.3: Sample of Writing Task strand of the Grade-Four Literacy Test

94

Appendix D: Codebook of Variables

Variable Values Description

Outcome: promo 1 = eligible Eligibility for promotion to high school 0 = not eligible Covariates: schcode 1001 ≤ x ≤ 14999 Primary school code

schname String variable Primary school name

Pub1 = All Age; Pub2 = Primary; schtype Primary school type Pub3 = Primary & Jnr High

schcsize 1 ≤ x ≤ 442 Primary school exam cohort size (proxy for school size)

HiSES = Level 1 schoolSES MidSES = Levels 2 & 3 Socioeconomic status of the school (proxy for student SES) LoSES = Levels 4 & 5

fname String variable Student first name

mname String variable Student middle name

lname String variable Student last name

female 1 = female student gender 0 = male

wr_2010 0 ≤ x ≤ 40 Student scores on Word Recognition, first sitting in 2010

rc_2010 0 ≤ x ≤ 30 Student scores on Reading Comprehension, first sitting in 2010

wt12_2010 0 ≤ x ≤ 8 Student scores on Writing Task, first sitting in 2010

rural, 0.11 ≤ x ≤ 29, percent_urban rural-urban, 0.30 ≤ x ≤ 0.60 % urbanization per parish urban, 0.61 ≤ x ≤ 1

95

Appendix E: Sample GIS Map of Jamaican Parish

96

Appendix F: Taxonomy of Statistical Models

Model A Model B Model C Model D

Parameter Parameter Parameter Parameter Estimate p- Estimate p- Estimate p- Estimate p- (Standard Error) value (Standard Error) value (Standard Error) value (Standard Error) value Fixed Efects: Intercept -1.781 0.000 -1.781 0.000 -1.776 0.000 -1.776 0.000 (0.154) (0.154) (0.151) (0.150) Female 0.318 0.047 0.318 0.044 0.318 0.044 0.316 0.036 (0.160) (0.158) (0.158) (0.151) All Age (AA) -0.554 0.349 -0.555 0.343 -0.538 0.351 -0.534 0.343 (0.591) (0.586) (0.577) (0.563) Primary & Junior High (PJH) 0.129 0.721 0.130 0.716 0.100 0.747 0.098 0.752 (0.360) (0.356) (0.311) (0.310) MidSES -0.151 0.443 -0.158 0.420 -0.180 0.341 -0.156 0.403 (0.196) (0.196) (0.189) (0.187)

LoSES (poorest school community) -0.275 0.279 -0.252 0.318 -0.221 0.367 -0.294 0.219 (0.254) (0.253) (0.245) (0.239) Small school exam-cohort size -0.726 0.034 -0.725 0.029 -0.748 0.013 -0.752 0.005 (SM_CSIZE) (0.343) (0.331) (0.302) (0.266) Large school exam-cohort size 0.239 0.525 0.239 0.525 0.234 0.533 0.234 0.533 (LG_CSIZE) (0.376) (0.376) (0.375) (0.374)

Two-way interactions: Female x AA -0.052 0.951 -0.049 0.953 -0.048 0.954 -0.061 0.936 (0.855) (0.831) (0.832) (0.762) Female x PJH -0.708 0.098 -0.711 0.062 -0.717 0.059 -0.700 0.062 (0.428) (0.380) (0.381) (0.375) Female x MidSES -0.031 0.878 -0.011 0.956 -0.010 0.962 -0.077 0.675 (0.205) (0.200) (0.200) (0.183) Female x LoSES 0.078 0.764 0.024 0.926 0.017 0.945 0.194 0.349 (0.259) (0.254) (0.253) (0.207) Female x SM_CSIZE -0.197 0.670 -0.201 0.617 -0.202 0.616 -0.186 0.179 (0.463) (0.402) (0.403) (0.139) Female x LG_CSIZE 0.445 0.160 0.445 0.158 0.445 0.158 0.446 0.152 (0.316) (0.315) (0.315) (0.312)

AAxMidSES 0.536 0.379 0.531 0.379 0.506 0.393 0.521 0.367 (0.609) (0.603) (0.593) (0.578) AAxLoSES 0.487 0.429 0.500 0.413 0.494 0.411 0.452 0.440 (0.615) (0.610) (0.601) (0.585) PJHxMidSES -0.159 0.734 -0.116 0.801 0.015 0.965 0.012 0.973 (0.469) (0.459) (0.354) (0.353) PJHxLoSES 1.169 0.145 0.945 0.229 0.482 0.245 0.446 0.278 (0.803) (0.786) (0.414) (0.411) PJH x SM_CSIZE -0.245 0.704 -0.250 0.668 -0.167 0.559 -0.151 0.596 (0.645) (0.583) (0.285) (0.284) PJH x LG_CSIZE -0.224 0.782 -0.225 0.781 -0.196 0.805 -0.193 0.807 (0.810) (0.809) (0.791) (0.790) MidSES x SM_CSIZE 0.810 0.030 0.822 0.023 0.870 0.007 0.830 0.003 (0.374) (0.361) (0.322) (0.281) MidSES x LG_CSIZE 0.044 0.920 0.052 0.907 0.074 0.866 0.050 0.909 (0.442) (0.442) (0.439) (0.438) LoSES x SM_CSIZE 0.738 0.075 0.702 0.081 0.677 0.067 0.792 0.015 (0.414) (0.402) 0.369 (0.324) LoSES x LG_CSIZE 1.066 0.154 1.043 0.163 1.012 0.175 1.085 0.144 (0.747) (0.747) (0.746) (0.743)

97

Three-way interactions: femalexAAxMidSES 0.026 0.977 0.041 0.962 0.040 0.963 -0.007 0.993 (0.880) (0.854) (0.854) (0.778) femalexAAxLoSES -0.138 0.877 -0.173 0.841 -0.177 0.838 -0.058 0.941 (0.888) (0.864) (0.864) (0.787) femalexPJHxMidSES 0.520 0.327 0.397 0.364 0.397 0.363 0.405 0.344 (0.531) (0.437) (0.436) (0.428) femalexPJHxLoSES -1.678 0.163 -0.491 0.351 -0.507 0.336 -0.392 0.441 (1.204) (0.526) (0.527) (0.510) femalexPJHx SM_CSIZE 0.846 0.327 0.861 0.009 0.872 0.009 0.806 0.013 (0.863) (0.331) (0.331) (0.326) femalexPJHx LG_CSIZE 0.295 0.661 0.298 0.643 0.304 0.636 0.287 0.653 (0.671) (0.642) (0.642) (0.639) femalexMidSESx SM_CSIZE -0.078 0.874 -0.114 0.787 -0.115 0.786 (0.495) (0.422) (0.422) femalexLoSESx SM_CSIZE 0.196 0.712 0.285 0.541 0.295 0.528 (0.529) (0.467) (0.468) femalexMidSESx LG_CSIZE -0.631 0.096 -0.651 0.084 -0.652 0.083 -0.585 0.112 (0.379) (0.376) (0.376) (0.368) femalexLoSESx LG_CSIZE -1.581 0.005 -1.527 0.007 -1.521 0.007 -1.697 0.002 (0.564) (0.561) (0.561) (0.542) PJHxMidSESx SM_CSIZE 0.284 0.699 0.220 0.737 (0.734) (0.655) PJHxLoSESx SM_CSIZE (0.746) 0.455 (0.463) 0.617 (0.9982) (0.9273) Four-way interactions: femalexPJHxMidSESx SM_CSIZE -0.178 0.851 (0.946) Female x PJH x LoSES x SM_CSIZE 1.341 0.357 (1.456) Random Effects: Between-school variance 0.323 0.323 0.324 0.323 (0.050) (0.051) (0.051) (0.051) Intraclass correlation 0.089 0.089 0.090 0.090 (0.013) (0.013) (0.013) (0.013) Goodness of fit: -2LL 12752.417 12754.235 12754.994 12756.917 GLH Test:

femalexPJHxMidSESxcsize1 femalexAAxMidSES = PJHxMidSESxcsize1 femalexMidSESxcsize1 = = femalexAAxLoSES = Ho: = femalexLoSESxcsize1 femalexPJHxLoSESxcsize1 femalexPJHxMidSES = PJHxLoSESxcsize1 = 0 = 0 = 0 femalexPJHxLoSES = 0

Chi-square 1.51 0.76 1.92 5.38 df 2 2 2 4 p-value 0.469 0.683 0.383 0.251 Decision Fail to reject Ho Fail to reject Ho Fail to reject Ho Fail to reject Ho

98

Model E Model F Model G Model H Parameter Parameter Estimate Estimate Parameter Parameter (Standard p- (Standard p- Estimate p- Estimate p- Error) value Error) value (Standard Error) value (Standard Error) value Fixed Efects: Intercept -1.763 0.000 -1.783 0.000 -1.782 0.000 -1.802 0.000 (0.147) (0.143) (0.143) (0.137)

Female 0.287 0.041 0.291 0.038 0.289 0.039 0.294 0.034 (0.140) (0.140) (0.140) (0.139)

All Age (AA) -0.528 0.301 -0.066 0.542 -0.066 0.542 -0.066 0.543 (0.511) (0.109) (0.109) (0.109)

Primary & Junior High (PJH) 0.037 0.901 0.144 0.511 0.139 0.522 0.147 0.502 (0.297) (0.220) (0.217) (0.219)

MidSES -0.188 0.304 -0.161 0.358 -0.161 0.358 -0.157 0.330 (0.183) (0.175) (0.175) (0.161)

LoSES (poorest school community) -0.271 0.248 -0.239 0.300 -0.239 0.299 -0.148 0.500 (0.234) (0.231) (0.231) (0.219)

Small school exam-cohort size -0.755 0.005 -0.877 0.000 -0.877 0.000 -0.864 0.000 (SM_CSIZE) (0.266) (0.244) (0.243) (0.242) Large school exam-cohort size 0.221 0.554 0.241 0.519 0.230 0.531 0.358 0.058 (LG_CSIZE) (0.373) (0.373) (0.367) (0.189)

Two-way interactions: Female x AA -0.080 0.604 -0.082 0.595 -0.082 0.595 -0.081 0.600 (0.154) (0.154) (0.154) (0.154)

Female x PJH -0.506 0.036 -0.504 0.037 -0.488 0.026 -0.483 0.027 (0.242) (0.242) (0.219) (0.218)

Female x MidSES 0.002 0.990 -0.004 0.979 -0.004 0.980 -0.011 0.946 (0.161) (0.161) (0.161) (0.159)

Female x LoSES 0.146 0.414 0.142 0.428 0.142 0.426 0.124 0.484 (0.179) (0.179) (0.179) (0.178)

Female x SM_CSIZE -0.188 0.168 -0.186 0.173 -0.184 0.176 -0.179 0.188 (0.137) (0.137) (0.136) (0.136)

Female x LG_CSIZE 0.476 0.121 0.472 0.124 0.495 0.067 0.465 0.073 (0.307) (0.307) (0.270) (0.259)

AAxMidSES 0.519 0.321 (0.523)

AAxLoSES 0.432 0.413 (0.528)

PJHxMidSES 0.152 0.638 (0.323)

PJHxLoSES 0.286 0.444 (0.373)

PJH x SM_CSIZE -0.134 0.633 -0.058 0.825 -0.053 0.839 -0.060 0.819 (0.280) (0.262) (0.260) (0.261)

PJH x LG_CSIZE -0.132 0.866 -0.239 0.753 -0.201 0.781 -0.308 0.652 (0.785) (0.761) (0.721) (0.682)

MidSES x SM_CSIZE 0.838 0.003 0.963 0.000 0.963 0.000 0.963 0.000 (0.281) (0.260) (0.260) (0.253)

MidSES x LG_CSIZE 0.082 0.851 0.054 0.900 0.064 0.881 (0.436) (0.434) (0.430)

LoSES x SM_CSIZE 0.787 0.015 0.896 0.003 0.896 0.003 0.812 0.006 (0.324) (0.300) (0.300) (0.294)

LoSES x LG_CSIZE 1.061 0.152 1.030 0.165 1.040 0.160 (0.741) (0.742) (0.740)

99

Three-way interactions: femalexAAxMidSES femalexAAxLoSES femalexPJHxMidSES femalexPJHxLoSES femalexPJHx SM_CSIZE 0.755 0.012 0.756 0.012 0.739 0.009 0.734 0.010 (0.302) (0.302) (0.284) (0.284) femalexPJHx LG_CSIZE 0.093 0.870 0.091 0.873 (0.571) (0.571) femalexMidSESx SM_CSIZE femalexLoSESx SM_CSIZE femalexMidSESx LG_CSIZE -0.664 0.063 -0.658 0.065 -0.679 0.041 -0.660 0.035 (0.357) (0.357) (0.332) (0.314) femalexLoSESx LG_CSIZE -1.650 0.002 -1.645 0.002 -1.667 0.001 -1.506 0.003 (0.532) (0.532) (0.515) (0.501) PJHxMidSESx SM_CSIZE

PJHxLoSESx SM_CSIZE

Four-way interactions: femalexPJHxMidSESx SM_CSIZE

Female x PJH x LoSES x SM_CSIZE

Random Effects: Between-school variance 0.323 0.326 0.326 0.332 (0.051) (0.051) (0.051) (0.051)

Intraclass correlation 0.089 0.090 0.090 0.092 (0.013) (0.013) (0.013) (0.013) Goodness of fit: -2LL 12762.351 12764.031 12764.057 12766.258 GLH Test:

AAxMidSES = MidSESxcsize3 PJHxcsize1 AAxLoSES = Ho: = = PJHxMidSES = LoSESxcsize3 = 0 PJHxcsize3 = 0 PJHxLoSES =0

Chi-square 1.67 2.22 0.22 df 4 2 2 p-value 0.797 0.330 0.896 Decision Fail to reject Ho Fail to reject Ho Fail to reject Ho

100

Model I Model J Model K

Parameter Parameter Parameter Estimate p- Estimate p- Estimate p- (Standard Error) value (Standard Error) value (Standard Error) value Fixed Efects: Intercept -1.800 0.000 -1.759 0.000 -1.740 0.000 (0.133) (0.126) (0.124)

Female 0.295 0.033 0.200 0.000 0.175 0.001 (0.138) (0.053) (0.051)

All Age (AA) -0.064 0.551 -0.103 0.275 -0.102 0.277 (0.108) (0.094) (0.094)

Primary & Junior High (PJH) 0.097 0.411 0.059 0.608 0.054 0.639 (0.118) (0.115) (0.115)

MidSES -0.147 0.356 -0.191 0.201 -0.200 0.179 (0.159) (0.150) (0.149)

LoSES (poorest school community) -0.144 0.508 -0.115 0.580 -0.143 0.489 (0.218) (0.208) (0.207)

Small school exam-cohort size -0.870 0.000 -0.926 0.000 -0.936 0.000 (SM_CSIZE) (0.235) (0.230) (0.229) Large school exam-cohort size 0.333 0.066 0.380 0.031 0.330 0.054 (LG_CSIZE) (0.182) (0.177) (0.171)

Two-way interactions: Female x AA -0.082 0.593 (0.154)

Female x PJH -0.477 0.023 (0.210)

Female x MidSES -0.014 0.931 (0.159)

Female x LoSES 0.122 0.493 (0.177)

Female x SM_CSIZE -0.175 0.192 (0.134)

Female x LG_CSIZE 0.455 0.078 (0.258) AAxMidSES

AAxLoSES

PJHxMidSES

PJHxLoSES

PJH x SM_CSIZE

PJH x LG_CSIZE

MidSES x SM_CSIZE 0.955 0.000 0.993 0.000 1.000 0.000 (0.252) (0.251) (0.251) MidSES x LG_CSIZE

LoSES x SM_CSIZE 0.810 0.006 0.821 0.005 0.848 0.004 (0.293) (0.293) (0.292) LoSES x LG_CSIZE

101

Three-way interactions: femalexAAxMidSES femalexAAxLoSES femalexPJHxMidSES femalexPJHxLoSES femalexPJHx SM_CSIZE 0.722 0.006 0.218 0.171 0.241 0.129 (0.262) (0.159) (0.158) femalexPJHx LG_CSIZE femalexMidSESx SM_CSIZE femalexLoSESx SM_CSIZE femalexMidSESx LG_CSIZE -0.645 0.039 -0.120 0.481 (0.312) (0.170) femalexLoSESx LG_CSIZE -1.492 0.003 -0.837 0.045 (0.500) (0.418) PJHxMidSESx SM_CSIZE

PJHxLoSESx SM_CSIZE

Four-way interactions: femalexPJHxMidSESx SM_CSIZE

Female x PJH x LoSES x SM_CSIZE

Random Effects: Between-school variance 0.333 0.334 0.331 (0.051) (0.051) (0.051)

Intraclass correlation 0.092 0.092 0.091 (0.013) (0.013) (0.013) Goodness of fit: -2LL 12766.478 12778.428 12783.074 GLH Test: femalexAA = femalexPJH = femalexMidSESxcsize3 = femalexMidSES = Ho: femalexLoSESxcsize3 femalexLoSES = = 0 femalexcsize1 = femalexcsize3 = 0 Chi-square 11.83 4.38 df 6 2 p-value 0.066 0.112 Decision Fail to reject Ho Fail to reject Ho

102

Model L Model M Model N (FINAL)

Parameter Parameter Parameter Estimate p- Estimate p- Estimate p- (Standard Error) value (Standard Error) value (Standard Error) value Fixed Efects: Intercept -1.761 0.000 -1.705 0.000 -1.686 0.000 (0.124) (0.121) (0.120)

Female 0.198 0.000 0.199 0.000 0.199 0.000 (0.048) (0.048) (0.048)

All Age (AA) -0.108 0.249 -0.111 0.242 (0.094) (0.095)

Primary & Junior High (PJH) 0.118 0.267 0.106 0.319 (0.106) (0.107)

MidSES -0.196 0.187 -0.175 0.242 -0.181 0.229 (0.149) (0.150) (0.150)

LoSES (poorest school community) -0.136 0.513 -0.168 0.421 -0.181 0.387 (0.207) (0.209) (0.209)

Small school exam-cohort size -0.916 0.000 -0.973 0.000 -0.989 0.000 (SM_CSIZE) (0.229) (0.229) (0.228) Large school exam-cohort size 0.335 0.050 (LG_CSIZE) (0.171)

Two-way interactions: Female x AA

Female x PJH

Female x MidSES

Female x LoSES

Female x SM_CSIZE

Female x LG_CSIZE

AAxMidSES

AAxLoSES

PJHxMidSES

PJHxLoSES

PJH x SM_CSIZE

PJH x LG_CSIZE

MidSES x SM_CSIZE 0.995 0.000 0.976 0.000 0.961 0.000 (0.250) (0.252) (0.252) MidSES x LG_CSIZE

LoSES x SM_CSIZE 0.839 0.004 0.872 0.003 0.847 0.004 (0.292) (0.294) (0.294) LoSES x LG_CSIZE

103

Three-way interactions: femalexAAxMidSES femalexAAxLoSES femalexPJHxMidSES femalexPJHxLoSES femalexPJHx SM_CSIZE femalexPJHx LG_CSIZE femalexMidSESx SM_CSIZE femalexLoSESx SM_CSIZE femalexMidSESx LG_CSIZE femalexLoSESx LG_CSIZE

PJHxMidSESx SM_CSIZE

PJHxLoSESx SM_CSIZE

Four-way interactions: femalexPJHxMidSESx SM_CSIZE

Female x PJH x LoSES x SM_CSIZE

Random Effects: Between-school variance 0.331 0.342 0.346 (0.051) (0.052) (0.052)

Intraclass correlation 0.091 0.094 0.095 (0.013) 0.013 (129085.000) Goodness of fit: -2LL 12785.358 12789.144 12792.277 GLH Test:

schtype_pub1 Ho: = schtype_pub3 = 0

Chi-square 3.14 df 2 p-value 0.208 Decision Fail to reject Ho

104

Appendix G: Interview Guide for Grade-Four Focus Group

Hello! My name is Dawn Miller and I am a researcher at Harvard University in the United

States of America. Thank you for agreeing to talk with me about my research. I am very interested in learning about the grade-four literacy test (GFLT) and I am hoping that all of you will teach me. I would like to learn about the GFLT, how you are preparing for it, and why it is important to you. I am meeting with you as a group because I want to get as much information as possible. And when we meet like this, we can talk and share our different ideas and opinions. We can get much more information and it’s fun!

Here is how it works: I am going to ask one question and then as many of you who want to answer can do so. Then I will ask another question and you can answer if you want to. And we will continue like this until our meeting ends. Please know that you do not have to answer, but I really hope you do because I want to hear what everyone of you has to say.

There is no right or wrong answer. You are giving your opinion. Also, I am asking that none of you tell anyone else what is said here. Let’s agree that everything that is said in our meeting will remain between us. Agreed?

Now before we begin, I want you to know that everything you tell me will be kept private. I will not share anything you say with anyone else. In fact, I am going to give you a codename and the only person who will know this codename will be me. And when I speak with someone about the study or write an essay about it, I will use your codename. All information that I collect about you will be safely locked up. Finally, if you want to leave the 105 meeting, at any time, just let me know and you can leave. Okay? Any questions? I am really excited to hear what you have to say! Let’s begin.

Interview Questions

Personal and warm-up 1. What is your name? 2. How old are you? 3. Who is your favorite athlete?

Knowledge of the grade-four literacy test (GFLT) 4. What is the grade-four literacy test? a. When are you taking it? 5. Why is it important to take the test? 6. Who told you about taking the test? a. Has anyone else told you about it? 7. Did you know about the test before entering grade-four? a. How did you find out about it? 8. What do you think your teacher will do with the test results? a. What makes you say that? 9. Do you know anyone who took the GFLT before? a. What did they think about it?

Perception of the grade-four literacy test (GFLT) 10. Have you been preparing for the test? a. How have you been preparing for it? 11. What has your teacher told you about taking the test? 12. How do you feel about taking the test? 13. Do you think you are ready for the test?

106

Appendix H: Interview Guide for Grade-5 Students Who Passed the GFLT

Hello! My name is Dawn Miller and I am a researcher at Harvard University in the United States of America. Thank you for agreeing to talk with me about my research. I am very interested in learning about the grade-four literacy test (GFLT), what your experience was with it, and why it is important to you. I am hoping that you will teach me. There is no right or wrong answer. You are giving your opinion. Please know that if you are uncomfortable with any question I ask, we can skip it. Just tell me.

Now before we begin, I want you to know that everything you tell me will be kept private. I will not share anything you say with anyone else. In fact, I am going to give you a codename and the only person who will know this codename will be me. And when I speak with someone about the study or write an essay about it, I will use your codename. All information that I collect about you will be safely locked up. Finally, if you wish to leave the meeting, at any time, just let me know and we will stop the interview. Okay? Any questions? I am really excited to hear what you have to say! Let’s begin.

Interview Questions

Personal and warm-up 14. What is your name? 15. How old are you? 16. Who is your favorite athlete?

Knowledge of the grade-four literacy test (GFLT) 17. What is the grade-four literacy test? b. When did you take it? 18. Why is it important to take the test? 19. Who told you about taking the test? 20. Did you know about the test before entering grade-four? 107

a. How did you find out about it? 21. How did you do on the test? a. Who told you your score on the test? 22. What do you think your teacher did with the test results? a. What makes you say that? 23. Do you know anyone who took the GFLT before? a. What did they think about it?

Perception of the grade-four literacy test (GFLT) 24. Do you know what mastery and non-mastery mean? a. Are there any differences between students in the mastery and non-mastery groups? b. Do you know anyone in the mastery group? c. Do you know anyone in the non-mastery group? d. How did you find out? 25. How did your family/parents/guardian talk to you about taking the test? 26. Did you and your friends talk about the test?

Knowledge and perception of the grade-six achievement (GSAT) 27. What is the grade-six achievement test? a. When do you take it? b. How many times can you take it? 28. Do you get to choose the high school you want to attend? a. How do you do this? 29. What are your top-five high schools? 30. Are you preparing for the GSAT? a. How are you preparing for it? 31. Do you know anyone who took the GSAT before? b. What did they think about it? 32. What has your teacher told you about taking the GSAT? 33. How do you feel about taking the GSAT?

108

Appendix I: Interview Guide for Grade-5 Students Who Failed the GFLT

Hello! My name is Dawn Miller and I am a researcher at Harvard University in the United States of America. Thank you for agreeing to talk with me about my research. I am very interested in learning about the grade-four literacy test (GFLT) and I am hoping that you will teach me. I would like to learn about the GFLT, how you are preparing for it and why it is important to you. There is no right or wrong answer. You are giving your opinion. I want you to know that if you are uncomfortable with any question I ask, we can skip it. Just tell me.

Now before we begin, I want you to know that everything you tell me will be kept private. I will not share anything you say with anyone else. In fact, I am going to give you a codename and the only person who will know this codename will be me. And when I speak with someone about the study or write an essay about it, I will use your codename. All information that I collect about you will be safely locked up. Finally, if you wish to leave the meeting, at any time, just let me know and we will stop the interview. Okay? Any questions? I am really excited to hear what you have to say! Let’s begin.

Interview Questions

Personal and warm-up 1. What is your name? 2. How old are you? 3. Who is your favorite athlete?

Knowledge of the grade-four literacy test (GFLT) 4. What is the grade-four literacy test? c. When are you taking it? 5. Why is it important to take the test? 6. Who told you about taking the test? a. Has anyone else told you about it? 7. Did you know about the test before entering grade-four? 109

b. How did you find out about it? 8. Is this your first time taking it? 9. How many more times do you have to take the test? a. Did you know this the first time you took the test? b. Who told you how often you could take the test? 10. Who told you your score on past tests? 11. What do you think your teacher will do with the test results? a. What makes you say that? 12. Do you know anyone who took the GFLT before? a. What did they think about it?

Perception of the grade-four literacy test (GFLT) 13. You have taken the test before, how do you feel about taking it again? 14. How did you feel after you took the test the first time? a. What did your teacher say after you got your results? b. What did your teacher do after you got your results? 15. Do you know what mastery and non-mastery mean? a. Are there any differences between students in the mastery and non-mastery groups? b. Do you know anyone in the mastery group? c. How did you find out? d. Do you know anyone in the non-mastery group? e. How do you feel about that? 16. How did your family/parents/guardian talk to you about taking the test again? 17. What about your friends? Do you and your friends talk about the test? 18. Do you think you are ready for the GFLT now? a. What have you been doing to prepare for the test? b. Is this different from when you took it before? How?

110

Appendix J: Interview Guide for Teachers and Principals

Thank you for agreeing to meet with me to discuss my research. I am interested in gaining some insight into your perceptions of the grade-four literacy test (GFLT), including how you prepare students for the examination, how test results are handled, and your perspective on students’ performance on and attitude toward the test. I am not looking for a particular response. There are no right or wrong answers. I want your unique opinion. As stipulated in the consent form you signed, if at any time you are uncomfortable with a particular question, just say so and we will change direction. Additionally, if you wish to end the interview, at any time, just tell me and we will stop. Any questions?

Before we begin, I want you to know that everything you tell me will be kept confidential and securely stored at Harvard University. To further protect your identity, I will replace your real name with a pseudonym, which I will use in any discussion, presentation and publication. No one will be able to identify you. Do you have any questions or concerns? Shall we begin?

Interview Questions

Personal and warm-up 1. What is your name? 2. How long have you been an educator? How long have you been with this school? 3. What grade(s) do you teach? 4. How many students are in your class(es)?

Pre-Test: Preparation 5. How does your school prepare students for the GFLT? a. Are there special services offered to students in preparing for the test? 6. How do you manage the school schedule to accommodate test preparations? a. Do you reorganize the regular class time? Do you extend the school day? 111

b. How close to the test date do you make these adjustments? 7. What are children expected to know about the test? a. Do you tell them about the GFLT? What do you tell them? b. Do you inform them about the consequences of the test? c. Do you think they understand how critical the test is for their futures? d. Do you think they know about the test before entering grade-four? e. What do students not know about the GFLT? 8. How do you motivate students before test day? 9. How do you prepare the school facilities for test day, if at all? 10. How do you prepare the school environment for test day, if at all? 11. Are there any challenges in preparing your school and students for the GFLT?

During-Test: Execution 12. What happens on test day? a. How are students organized for testing? Whole class or in smaller groups? b. Are there any provisions for students on test day? 13. How long does the test last? a. Are there breaks during the test? 14. Who monitors the test?

Post-Test: Test reporting and Feedback 15. How are test results presented to your school community? a. In what format do you receive them from the Ministry? b. Is this the same format presented to students and families? 16. How are test results delivered to students and their families? a. Does the Ministry of Education mail the results? Does your school? b. Do parents/guardians collect the results from the school? 17. Do the students actually get a classification, or label, of mastery or non-mastery? a. Is the classification explicitly printed on test reports? For both the school and families? b. How do you use these labels, if at all? 112

18. For students who pass the GFLT, do they receive any kind of reward or recognition from the Ministry or the school? a. For example, are scores posted outside the principal’s office? 19. For students who fail the GFLT, are they told differently? a. Who informs them that they failed? 20. How do you talk to students after they failed the test? 21. What actions do you take to help them cope or deal with this failure? 22. What do you tell your students to prepare them for retaking the test? 23. How do you use the test results?

Experience 24. The GFLT was first low-stakes, now it’s high-stakes. How do you feel about that? a. How has this policy change impacted your school community? 25. What are some of the positive outcomes of the GFLT? Please name a few. 26. Have you observed any negative consequences? Please name a few. 27. Based on your experience, how have students handled the GFLT? 28. How do you think students feel about the test? 29. For 4th-graders, how would you describe their level of anxiety? 30. For 5th-graders who are retaking the test in June, how would you describe their level of anxiety? 31. What would you say are some of the effects on students of failing the test? 32. Do you think classifying students as non-mastery puts more pressure on them, in addition to failing the test? 33. What are your thoughts on the requirement for children who fail the test having to take it again?

113

Appendix K: Interview Guide for Parent Focus Group

Thank you for agreeing to meet with me to discuss my research. I am interested in gaining some insight into your experiences with the grade-four literacy test (GFLT), including how test results are handled and your thoughts on your children’s performance on the test. I am not looking for a particular response. I am meeting with you as a group because I want to get as much information as possible.

Here is how it works: At the beginning, I am going to ask a few questions and then as many of you who want to answer can do so. Then, we’ll follow the conversation flow and have a group discussion where we’ll share rich information. There is no right or wrong answer because you will be sharing your experiences with the GFLT. Furthermore, everything said here will remain here. Let’s all agree that everything discussed in our meeting will remain between us. Agreed?

On my end, I want you to know that everything you tell me will be kept confidential and securely stored at Harvard University. To further protect your identity, I will replace your real names with pseudonyms, which I will use in any discussion, presentation and publication. No one will be able to identify you, but me.

As stipulated in the consent form you signed, if at any time you are uncomfortable with a particular question, feel free not to respond. Additionally, if you wish to exit the group discussion, at any time, just tell me and you will be excused. Do you have any questions or concerns? Let’s begin.

Interview Questions

Knowledge of the grade-four literacy test (GFLT) 1. What is the grade-four literacy test? 114

a. Do you know what content is actually tested? What are children expected to know for the test? b. How often can children take the test? c. What are the possible scores they can get? 2. Why is it important to take the test? a. What happens if children pass the test? b. What happens if they fail the test? c. Do you know what ASTEP is? 3. What do you think is done with the test results? a. What makes you say that? 4. Do you talk to your children about the GFLT? a. What do you tell them? b. How would you describe their attitude toward the test?

Perception of the GFLT 5. How does your school prepare students for the GFLT? b. Are there special services offered to students in preparing for the test? c. What are these services? d. When are these services offered? e. Does your child participate? 6. What happens on test day? a. How long does the test last? 7. How are test results reported/delivered to you? To your child? a. How do you feel about this process?

For parents whose children failed the GFLT 8. Your children failed the test. How do you feel about that? a. How have you spoken to your children about their scores? b. How do you think your children feel about their recent performance on the test? c. What is the school doing about your children’s recent performance on the test? 115

d. What is the school doing to prepare your children for retaking the test? e. Are you doing anything to prepare your children for retaking the test?

Conclusion: everyone 9. If you could change or improve anything about the GFLT, what would that be? a. What changes/improvement would you make?

116

Appendix L: Violations in Implementation of the Competence-Based Transition Policy (CBTP)

According to the data, children were allowed to sit the grade-six test despite failing all attempts at the GFLT. This is in violation of the CBTP policy, which states (in part):

Under the Competence-Based Transition Policy: no child will be allowed to sit the Grade Six Achievement Test unless he/she is certified literate (MOE, 2011, p.3)

To get an understanding of the proportion of students who were “legitimately” qualified to take the Grade-Six Achievement Test (GSAT) and those who were not, in Figure

L.1, I present the different pathways students took after their initial attempt at the Grade-Four

Literacy Test (GFLT). It is important to note that during this first phase, I had defined

PROMO as the probability of being promoted to high school. As you will see, this definition changed in the second phase of my work. I have divided the diagram into divided into three main columns, each representing student yearly progress through the different administrations of the GFLT and culminating in the administration of the Grade-Six

Achievement Test (GSAT).

In the first column, I show the distribution of public-primary students who first sat the GFLT in school year 2009/10. In the corresponding rows, the students are arranged into three groups: those who passed their first attempt at the GFLT, those who failed their initial attempt, and those who were absent for the examination. Of the 46,120

117

2010 GFLT 2011 GFLT (First Sitting) (Retake) 2012-14 GSAT

Pass* Eligible 27,936 for 27,936 2,507 High School Pass (PROMO=1)

2,325 2,507 1,781 35,553

182 247 Fail Fail 15,287 5,187 3,082 5,913 726

1,306 4,132 Not Eligible Absent Absent for 923 High School 1,982 2,897 2,229 (PROMO=0)

10,567 4,453 6,469 1,066 Not Registered

7,535

Notes: * Total number of public-primary schoolchildren in cohort 2010: 46,120 GFLT - Grade Four Literacy Test GSAT - Grade Six Achievement Test Figure L.1: Graphical representation of the different pathways of students, from the first sitting of the GFLT in 2010 to becoming eligible for promotion to high school (n=46,120).

118

schoolchildren who took the test, 61% of them passed (27,936) on their first attempt, were certified as literate, and were eligible for promotion to high school (PROMO=1). Inversely,

33% of the examinees failed (15,287) and 6% were absent (2,897). These children were not eligible initially for promotion to high school; however, they had another chance to retake the GFLT in Grade 5 and become eligible for promotion.

In the second column, I illustrate the different pathways that children, who either failed or were absent for the first sitting of the GFLT, took as they traversed through the accountability system in Grade 5. Four pathways emerged from the data: some children passed on the second attempt, some failed again, others were absent for the retake, and the rest apparently never registered for the test. Among the 15,287 children who failed their first attempt, 15% (2,325) of them passed the retake of the GFLT and became eligible for promotion to high school. However, 34% (5,187) of these students failed again, 9% (1,306) were absent, and 42% (6,469) of them apparently never registered for the retake of the

GFLT. These latter groups of children were not certified as literate and, therefore, were not eligible for promotion to high school.

Among the 2,897 children were absent for the first administration of the GFLT, 6%

(182) of them passed the retake and became eligible for promotion to high school.

Conversely, 25% (726) of these students failed, 32% (923) were absent again, and 37%

(1,066) of these children apparently never registered for the retake of the test. The children in these latter groups were not certified as literate and, therefore, were not eligible for promotion to high school. All the children who failed the retake of the GFLT or never took the test (absent or never registered) were transitioned into the Alternative Secondary Transitional

Education Programme (ASTEP), to be remediated for up to two years. 119

In the third column, I present the number of children who actually sat the Grade-Six

Achievement Test (GSAT), or not. As previously stated, there seems to be a deviation from what the Competency-Based Transition Policy (CBTP) intended. Under this policy, only children who passed the GFLT – on their initial attempt in grade four or on the retake in grade five – were allowed to sit the GSAT. Here we see another story unfolding in column three: children who failed or missed all attempts at the grade-four test and were not permitted to sit the grade-six test, were still allowed to take the examination. As is expected, the proportions generated from this arrangement do not accurately reflect policy effectiveness because the results are inflated in favor of success. Among the students who sat the grade-four test in

2010, 77% (35,553), of them were identified as eligible for promotion and were allowed to sit the grade-six test. However, among this group of students, 28% (5,913 + 1,781 + 2,229) of them were in violation of the CBTP. This information suggests that there are some conceivable issues with how the CBTP was implemented because children were still being allowed to advance to the secondary level without being certified as literate.

One possible explanation for these discrepancies is that principals and school administrators might have exercised discretionary power in allowing children to take the

GSAT, even though they failed all attempts at the grade-four test. Given this scenario, while

I agree that school administrator’s should have discretionary power to use other student data, along with test scores, to determine which students are prepared adequately for advancement to the secondary level, I contend that amendments should be made to the

CBTP that would normalize such processes as a legitimate supplement to the policy.

Otherwise, such practices could amount to poor implementation and even corruption.

120

Second Take

After discovering these exceptions to the promotion rule, I decided to recreate my outcome variable, PROMO, using information from the externally administered grade-four test, and redefined it as follows: the probability of becoming eligible for promotion to high school. I chose this strategy because I determined that using information from the Grade-Six test would not have given an accurate measure of the effectiveness of the CBTP. Therefore, I regenerated PROMO using student performance data on the GFLT, in Grades Four and

Five, as follows:

• If student passed the first sitting of the GFLT in 2010 or the retake in 2011, then

PROMO=1

• Otherwise, PROMO=0

The logic behind this approach is that recalculating student eligibility directly from the grade-four test results will control for incidences, such as school administrators overriding policy dictates in the service of their students, for example. In Figure L.2, I display the same pathways students took, from the first sitting of the GFLT to grade-six; however, with the new PROMO, we now have an outcome variable that more closely matches the Competence-Based Transition Policy.

121

2010 GFLT 2011 GFLT (First Sitting) (Retake) 2012-14 GSAT Pass* 27,936 Eligible 27,936 for High School Pass 2,507 (PROMO=1)

2,325 2,507 30,443

182

Fail Fail 15,287 5,187 5,913 726

1,306 5,913 Not Eligible Absent Absent for 923 High School 2,229 2,897 2,229 (PROMO=0)

15,677 7,535 6,469 1,066 Not Registered

7,535

Notes: * Total number of public-primary schoolchildren in cohort 2010: 46,120 GFLT - Grade Four Literacy Test GSAT - Grade Six Achievement Test Figure L.2: Graphical representation of the different pathways of students, from the first sitting of the GFLT in 2010 to becoming eligible for promotion to high school, with PROMO redefined to match the Competence-Based Transition Policy (n=46,120).

122

Notice in column three that students are now assigned to the correct groups. All students who (1) failed the retake of the GFLT, (2) were absent for the test, or (3) did not register for the test, were correctly assigned to PROMO=0. With this adjustment, the proportion of children who became eligible for promotion to high school is now 66%

(30,443), as opposed to the previous value of 77% (35,553). And the proportion of children who were not eligible and so were not promoted to high school is 34% (15, 677), as opposed to the previous rate of 23% (10,567).

123

References

Bailey, E. K. (2007). Teacher education in a post-colonial context: A phenomenological study of the experience of Jamaican teachers' college lecturers (Order No. 3254932). Available from ProQuest Dissertations & Theses Global. (304839532).

Bailey, B. (2004). Gender and Education in Jamaica: Who Is Achieving and by Whose Standard? Prospects: Quarterly Review Of Comparative Education, 34(1), 53-69. Retrieved from the ERIC database. (EJ774797).

Bloom, D., Canning, D., Sevilla, J., Population Matters (Project), & Rand Corporation (2003). demographic dividend: A new perspective on the economic consequences of population change. Santa Monica, California: Rand.

Brimi, H. (2012). Teaching writing in the shadow of standardized writing assessment: An exploratory study. American Secondary Education 41 (1), 52-78. Retrieved from the ERIC database. (EJ1001572).

Caribbean Policy Research Institute [CaPRI] & Partnership for Educational Revitalization in the Americas [PREAL] (2012). Prisms of Possibility: A report card on education in Jamaica. Retrieved from http://prealblog.files.wordpress.com/2013/01/jamaica-rc-preal-full- version.pdf.

Clarke, C. (2005). Socialization and Teacher Expectations of Jamaican Boys in Schools: The Need for a Responsive Teacher Preparation Program. International Journal of Educational Policy, Research, and Practice: Reconceptualizing Childhood Studies, 5(4), 3-34. Retrieved from the ERIC database. (EJ795142).

Cole, J. S. & Osterlind, S. J. (2008). Investigating differences between low- and high-stakes test performance on a general education exam. The Journal of General Education 57(2), 119-130. Retrieved from https://muse.jhu.edu/article/248665.

Davis, R. (2004). Task force on educational reform Jamaica: A transformed education system. Kingston, Jamaica: Ministry of Education.

DePascale, C. (2003). The ideal role of large-scale testing in a comprehensive assessment system. Journal of Applied Testing Technology, 1-10.

Diamond, J. B., & Spillane, J. P. (2004). High-Stakes Accountability in Urban Elementary Schools: Challenging or Reproducing Inequality? Teachers College Record, 106(6), 1145-1176. doi:10.1111/j.1467-9620.2004.00375.x.

124

Dixon, L. Q. (2005). Bilingual education policy in Singapore: An analysis of its sociohistorical roots and current academic outcomes. International Journal of Bilingual Education and Bilingualism, 8(1), 25-47. DOI: 10.1080/jBEB.v8.i1.pg25.

Editorial Projects in Education Research Center [EPERC] (2015). Issues A-Z: No Child Left Behind: An overview. Education Week. Retrieved from http://www.edweek.org/ew/section/multimedia/no-child-left-behind-overview- definition-summary.html/.

Evans, H. 1999. Gender differences in education in Jamaica, Education for All in the Caribbean: Assessment 2000 Monograph Series. Kingston: UNESCO and Planning Institute of Jamaica.

Figueroa, M. (2004). Male privileging and male academic underperformance in Jamaica. In R. E. Reddock (Ed.), Interrogating Caribbean masculinities: Theoretical and empirical analyses (pp 137-166). Mona, Jamaica: University of the West Indies Press.

Fowler, W. J., & Walberg, H. J. (1991). School size, characteristics, and outcomes. Educational Evaluation and Policy Analysis, 13, 189-202. Retrieved from the ERIC database. (EJ430692).

Hunter, J. (2013, September 24). Education Ministry Urges Schools to Prepare Students for Curriculum. Jamaica Information Service. Retrieved from http://jis.gov.jm/education-ministry-urges-schools-to-prepare-students-for- curriculum/.

Inter-American Development Bank [IADB] (2003). Jamaica: Productivity and competitiveness in the Jamaican economy. Retrieved from http://ctrc.sice.oas.org/TRC/Articles/Jamaica/Competitiveness.pdf.

Inter-American Development Bank [IADB] (2016). Jamaica: IDBG country strategy with Jamaica 2016-2021. Retrieved from http://idbdocs.iadb.org/wsdocs/getdocument.aspx?docnum=40713849.

Jacob, B.A. (2005). Accountability, Incentives and Behavior: The Impact of High-Stakes Testing in the Chicago Public Schools. Journal of Public Economics, 89(5-6), 761-796. doi: 10.3386/w8968.

Jamaica Ministry of Education [JMOE] (2008). The development of education: National report of Jamaica. Kingston, Jamaica: Ministry of Education. Retrieved from http://www.ibe.unesco.org/National_Reports/ICE_2008/jamaica_NR08.pdf.

Jamaica Ministry of Education [JMOE] (2009). Vision 2030 Jamaica: Education sector plan 2009-2030. Kingston, Jamaica: Ministry of Education. Retrieved from http://www.vision2030.gov.jm/Portals/0/Sector_Plan/Microsoft Word - Education.pdf. 125

Jamaica Ministry of Education [JMOE] (2011). Alternative Secondary Transitional Education Programme (ASTEP): Conceptual framework. Kingston, Jamaica: Ministry of Education.

Jamaica Ministry of Education [JMOE] (2012). National education strategic plan: 2011-2020. Kingston, Jamaica: Ministry of Education. Retrieved from https://www.mona.uwi.edu/cop/library/jamaica-national-education-strategic-plan- 2011-2020.

Jamaica Ministry of Education [JMOE] (2013). Jamaica education statistics 2012-2013: Annual statistical review of the education sector. Kingston, Jamaica: Ministry of Education.

Jamaica Ministry of Education [JMOE] (2017). About. Retrieved from http://moe.gov.jm/about.

Jamaica Observer (2015, November 10). GSAT students to have choice of 7 schools. Jamaica Observer. Retrieved from http://www.jamaicaobserver.com/news/GSAT-students-to-have-choice-of-7- schools.

Jamaica Productivity Center [JPC] (2009). Jamaica: Productivity summary report 1972-2007. Kingston, Jamaica. Retrieved from http://jpc.com.jm/jamaica-productivity-summary-report-1972-2007/.

Jones, B., & Egley, R. (2004). Voices from the frontlines: teachers’ perceptions of high- stakes testing. Education Policy Analysis Archives, 12(39). Retrieved from the ERIC database. (EJ853506).

Kulm, G. (2013). Back to the future: Reclaiming effective mathematics assessment strategies. Middle Grades Research Journal, 8(2), 1-10.

Lee, V. E., & Smith, J. B. (1997). High school size: Which works best and for whom? Educational Evaluation and Policy Analysis, 19, 205-227. Retrieved from the ERIC database. (EJ554778).

Linton, L. (2015). Students shine in GSAT. Jamaica Information Service. Retrieved from http://jis.gov.jm/students-shine-gsat/.

Lockheed, M., Harris, A., Gammill, P., Barrow, K., & Jayasundera, T. (2006). New horizons for primary schools in Jamaica: Inputs, outcomes and impact. Washington, DC: Academy for Educational Development Education Policy and Data Center Report to USAID.

Lockheed, M., Harris, A., & Jayasundera, T. (2010). School improvement plans and student learning in Jamaica. International Journal of Educational Development, 30(1), 54-66. Retrieved from the ERIC database. (EJ857528).

126

McMillan, J. H. (2005). The impact of high-stakes test results on teachers' instructional and classroom assessment practices. Retrieved from the ERIC database. (ED490648).

Millennium Development Goals [MDG] (2014). The Millennium development goals: Eight goals for 2015. Retrieved from http://www.un.org/millenniumgoals/ .

Mitchell, R. (2006). Research Review: Effects of High-Stakes Testing on Instruction. The Center for Public Education, Alexandria, VA. Retrieved from http://www.centerforpubliceducation.org.

Moses, M. S. & Nanna, M. J. (2007). The Testing Culture and the Persistence of High Stakes Testing Reforms. Education and Culture, 23(1), 55-72.

Nelson, H. (2013). Testing more, teaching less: What America’s obsession with student testing costs in money and lost instructional time. New York: American Federation of Teachers. Retrieved from http://www.aft.org/sites/default/files/news/testingmore2013.pdf.

Nichols, S.L., & Berliner, D.C. (2005). The Inevitable Corruption of Indicators and Educators Through High- Stakes Testing. Education Policy Research Unit, Arizona State University, Tempe, AZ. Retrieved from the ERIC database. (ED508483).

Nichols, S. L., & Berliner, D. C. (2007). Collateral damage. Cambridge, MA: Harvard Education Press.

Organization for Economic Co-operation and Development [OECD] (2011). Singapore: Rapid improvement followed by strong performance. In Lessons from PISA for the United States. DOI: http://dx.doi.org/10.1787/9789264096660-8-en.

Pedulla J. J., Abrams, L. M., Madaus, G. F., Russell, M. K., Ramos, M. A., & Miao, J. (2003). Perceived effects of state-mandated testing programs on teaching and learning: Findings from a national survey of teachers. National Board on Educational Testing and Public Policy.

Perelman, L. (2012). Length, score, time, and construct validity in holistically graded writing Assessments: The case against automated essay scoring (AES). In C. Bazerman, C. Dean, K. Lunsford, S. Null, P. Rogers & A. Stansell, et al. (Eds), New Directions in International Writing Research (121-132). Anderson, SC: Parlor Press.

Planning Institute of Jamaica [PIOJ] (2009). Vision 2030 Jamaica: National Development Plan. Kingston, Jamaica: Planning Institute of Jamaica.

Planning Institute of Jamaica [PIOJ] (2015). Medium term socio-economic policy framework 2015- 2018: Achieving inclusive growth and sustainable development. Kingston, Jamaica: Planning Institute of Jamaica.

Planning Institute of Jamaica [PIOJ] & Statistical Institute of Jamaica [STATIN] (2010). Jamaica survey of living conditions. Kingston, Jamaica: Planning Institute of Jamaica. 127

Rhone, A.E. (2006). Preparing minority students for high-stakes tests: Who are we cheating? Childhood Education, 82(4), 233-235. Retrieved from the ERIC database. (EJ754809).

Saunders, A. (2015, February 25). I’m not satisfied. Jamaica Observer. Retrieved from http://www.jamaicaobserver.com/news/-I-m-not-satisfied-_18461181.

Sirin, S. (2005). Socioeconomic status and academic achievement: A meta-analytic review of research. Review of Educational Research, 75(3), 417-453. Retrieved from http://steinhardt.nyu.edu/scmsAdmin/media/users/lec321/Sirin_Articles/Sirin_20 05.pdf.

Tsang, M. C., Fryer, M., & Arevalo, G. (2002). Access, equity and performance: Education in Barbados, Guyana, Jamaica, and Trinidad and Tobago. Washington, D.C.: Inter-American Development Bank.

United Nations Development Programme [UNDP] (2013). Human development report 2013: The rise of the south: Human progress in a diverse world. Retrieved from http://hdr.undp.org/sites/default/files/reports/14/hdr2013_en_complete.pdf.

United Nations Development Programme [UNDP] (2016). Human development report 2016: Human development for everyone. Retrieved from http://hdr.undp.org/sites/default/files/2016_human_development_report.pdf.

United Nations Educational, Scientific and Cultural Organization [UNESCO] (1990). Meeting basic learning needs: A vision for the 1990s. Retrieved from the ERIC database. (ED323655).

United Nations Educational, Scientific and Cultural Organization [UNESCO] (2015). Education For All 2000-2015: Achievements and challenges. Retrieved from http://en.unesco.org/gem-report/report/2015/education-all-2000-2015- achievements-and-challenges#sthash.Rnrz9uPL.dpbs.

Valli, L. & Buese, D. (2007). The changing roles of teachers in an era of high-stakes accountability. American Educational Research Journal 44 (3), 519-558.

Volante, L. (2005). Accountability, student assessment, and the need for a comprehensive approach. International Electronic Journal for Leadership in Learning, 9(6). Retrieved from the ERIC database. (EJ985390).

Voyer, D. & Voyer, S. (2014). Gender Differences in Scholastic Achievement: A Meta- Analysis. Psychological Bulletin, 140(4), 1174-1204.

Whyte, M. (1983). A short history of education in Jamaica (2nd ed.). London: Hodder and Stoughton.

128

World Bank (2004). The road to sustained growth in Jamaica: World Bank country study. Washington, D.C.: World Bank. Retrieved from https://openknowledge.worldbank.org/handle/10986/15014.

World Bank (2011). Jamaica country economic memorandum: Unlocking growth. Washington, DC: World Bank. Retrieved from https://openknowledge.worldbank.org/handle/10986/2756.

World Bank (2017). Prevalence of undernourishment (% of population). Retrieved from http://data.worldbank.org/indicator/SN.ITK.DEFC.ZS?locations=JM.

Yeh, S. S. (2005). Limiting the unintended consequences of high-stakes testing. Education Policy Analysis Archives, 13(43). Retrieved from http://epaa.asu.edu/ojs/article/view/148/274.

129

VITA

Dawn Elizabeth Elaine Miller

1987-1992 Florida Atlantic University B.S.E.E Boca Raton, FL Dec 1992

1993-2000 Software Engineer, Corporate Trainer Siemens Corporation Boca Raton, FL

2000-2001 Product/Training Manager, CTI Squared, Inc. Burlington, MA

2001-2002 Management/Technology Consultant, IBI Group Boston, MA

2002-2004 Massachusetts Institute of Technology M.B.A. Cambridge, MA June 2004

2004-2009 Professor, STEM Director (Upward Bound) Roxbury Community College Roxbury, MA

2006-2008 HGSE, Harvard University Ed.M. Cambridge, MA May 2008

2009-Present Educational Researcher/Consultant Independent

2008-2017 HGSE, Harvard University Ed.D. Cambridge, MA May 2017

2009-2013 Teaching Fellow, 2010-2015 PPE Group Facilitator HGSE, Harvard University Cambridge, MA