An Experimental Evaluation of a Tutoring Program for Struggling Readers
Total Page:16
File Type:pdf, Size:1020Kb
Journal of Research on Educational Effectiveness ISSN: 1934-5747 (Print) 1934-5739 (Online) Journal homepage: http://www.tandfonline.com/loi/uree20 Leveraging Volunteers: An Experimental Evaluation of a Tutoring Program for Struggling Readers Robin Jacob, Catherine Armstrong, A. Brooks Bowden & Yilin Pan To cite this article: Robin Jacob, Catherine Armstrong, A. Brooks Bowden & Yilin Pan (2016) Leveraging Volunteers: An Experimental Evaluation of a Tutoring Program for Struggling Readers, Journal of Research on Educational Effectiveness, 9:sup1, 67-92, DOI: 10.1080/19345747.2016.1138560 To link to this article: http://dx.doi.org/10.1080/19345747.2016.1138560 Accepted author version posted online: 06 Feb 2016. Published online: 06 Feb 2016. Submit your article to this journal Article views: 40 View related articles View Crossmark data Full Terms & Conditions of access and use can be found at http://www.tandfonline.com/action/journalInformation?journalCode=uree20 Download by: [75.177.179.204] Date: 07 October 2016, At: 07:11 JOURNAL OF RESEARCH ON EDUCATIONAL EFFECTIVENESS 2016, VOL. 9, NO. S1, 67–92 http://dx.doi.org/10.1080/19345747.2016.1138560 Leveraging Volunteers: An Experimental Evaluation of a Tutoring Program for Struggling Readers Robin Jacoba, Catherine Armstrongb, A. Brooks Bowdenc, and Yilin Panc ABSTRACT KEYWORDS This study evaluates the impacts and costs of the Reading Partners volunteers program, which uses community volunteers to provide one-on-one cost study tutoring to struggling readers in under-resourced elementary schools. randomized control trial The evaluation uses an experimental design. Students were randomly struggling readers tutors assigned within 19 different Reading Partners sites to a program or control condition to answer questions about the impact of the program on student reading proficiency. A cost study, using a subsample of six of the 19 study sites, explores the resources needed to implement the Reading Partners program as described in the evaluation. Findings indicate that the Reading Partners program has a positive and statistically significant impact on all three measures of reading proficiency assessed with an effect size equal to around 0.10. The cost study findings illustrate the potential value of the Reading Partners program from the schools’ perspective because the financial and other resources required by the schools to implement the program are low. Additionally, the study serves as an example of how evaluations can rigorously examine both the impacts and costs of a program to provide evidence regarding effectiveness. Reading skills are the key building blocks of a child’s formal education. Yet, the national sta- tistics on literacy attainment are profoundly distressing: two out of three American fourth graders are reading below grade level and almost one third of children nationwide lack even basic reading skills (National Center for Education Statistics [NCES], 2013). For children in low-income families, the numbers are even more troubling, with 80% reading below grade level (NCES, 2013). Despite several decades of educational reform efforts, only incremental progress has been made in addressing this reading crisis. From 1998 to 2013, the number of low-income fourth graders reading at a proficient level increased by only seven percentage points (NCES, 2013). Although there are a variety of interventions to help struggling ele- mentary school readers, many of the programs with the strongest evidence base are both time- and resource-intensive, and as such may not always be viable options for already under-resourced schools (Hollands et al., 2016; Simon, 2011). Since the 2007 recession, school and district policymakers have been faced with tight budget constraints (Leachman & Mai, 2014). Therefore, it is important that rigorous research provide not only evidence of CONTACT Robin Jacob [email protected] 330 Packard Street, Ann Arbor, MI 48104-2910, USA. aUniversity of Michigan, Ann Arbor, Michigan, USA bMDRC, New York, New York, USA cTeachers College, Columbia University, New York, New York, USA *Catherine Armstrong is now a graduate student at the Harvard Graduate School of Education. © 2016 MDRC effects, but also information about costs, so that decision makers can enact reform as effi- ciently as possible. From the perspective of a school, tutoring by volunteers has the potential to be a low-cost option for addressing children’s literacy needs because most of the costs are borne by volunteers. To date, only limited evidence exists regarding the efficacy of this approach on a wide scale. This study evaluates the impacts and costs of the Reading Partners program, which uses community volunteers to provide one-on-one tutoring to struggling readers in under- resourced elementary schools. Reading Partners currently serves more than 8,500 students in over 160 schools throughout California, Colorado, Maryland, New York, Oklahoma, South Carolina, Texas, Washington, and Washington, DC. The evaluation uses an experi- mental design, in which students were randomly assigned within 19 different Reading Part- ners sites to a program or control condition, to answer questions about the impact of the program on student reading proficiency. In addition, data were collected to provide informa- tion about the context in which the program was implemented and about the degree to which the program was implemented with fidelity. The evaluation also includes a cost study, using a subsample of six of the 19 study sites, to explore the resources needed to implement the Reading Partners program as described in the evaluation. Background Several national initiatives (including the Reading Excellence Act of 1997 and Reading First, which was established as part of the No Child Left Behind Act of 2001) have been instituted in recent years in an attempt to provide schools with better resources to support struggling readers in elementary school. In addition, a number of states have either enacted or are con- sidering enacting legislation regarding third-grade reading proficiency, with at least 14 states and Washington, DC, having adopted laws that require students to be retained if they are not reading at grade level by the end of third grade (Rose, 2012). The Elementary and Secondary Education Act (ESEA) of 2002 requires that schools pro- vide low-income families with extra academic assistance or supplemental education services if their children are attending a Title I school that is in Program Improvement.1 These sup- plemental education services come at no cost to families and can include tutoring or reme- dial help in subjects such as reading, language arts, and math, as long as they are aligned with state content standards and grounded in high-quality research that provides evidence of their effectiveness (Heinrich, Meyer, & Whitten, 2010). Often, external programs and ven- dors provide this extra help (U.S. Department of Education, 2012). Many supplemental education services use Response to Intervention (RtI) models, which provide a means of identifying and supporting struggling students. Funding for RtI was authorized under the Individuals with Disabilities Education Improvement Act of 2004. Under this model, all students are monitored and assessed for placement into a multitiered system of intervention. Tier I, usually classroom instruction for all students, is considered the first level of intervention, with Tier II and Tier III being progressively more intense (Fuchs & Fuchs, 2006). Daily one-on-one work with a specially trained reading 1 Title I schools are those that receive federal funds based on the number of low-income children in attendance. A school in Program Improvement has been designated by the state to be in need of improvement for more than one year (U.S. Depart- ment of Education, 2012). 68 R. JACOB ET AL. interventionist, for example, is considered a Tier III intervention. Reading Partners is typi- cally considered a Tier II intervention for students needing some additional support and is usually one among multiple Tier II interventions available for struggling readers at any given school.2 Prior Research on Supplemental Literacy Programs A variety of supplemental interventions exist to help elementary school students who are struggling to learn to read, including small-group work inside or outside the classroom, instructional technology, and one-on-one tutoring led by teachers, paraprofessionals, or vol- unteers. Among these, one-on-one tutoring by trained teachers has been shown to be among the most effective. In their meta-analysis of early reading interventions, Slavin, Lake, Davis, and Madden (2011) report a weighted mean effect size among 20 qualifying studies of 0.39, and a recent large-scale evaluation of Reading Recovery, one of the most widely known of these tutoring programs, found effects sizes between 0.42 and 0.68 standard deviations for the first graders participating in the program (May et al., 2013; May et al., 2015). Slavin et al. also find that tutoring by paraprofessionals and small-group tutorials, in which struggling readers are provided with supplementary instruction in small groups daily for 30– 45 minutes, are both effective approaches to reading intervention as well, with weighted effect sizes of 0.38 and 0.31, respectively. Despite their effectiveness, such interventions are typically expensive for schools to imple- ment. For example, reports on Reading Recovery indicate that the total cost of that program is about $4,360 to $9,180 per student, depending on the implementation.3 Small-group tuto- rials such as Corrective Reading and Wilson Reading System have been shown to cost $10,640 and $7,050 per student, respectively (Hollands et al., 2016; Simon, 2011). Tutoring by paraprofessionals is less expensive. For example, a cost study of the Sound Partners pro- gram, which served kindergarten students once a day for 30 minutes a day, four days a week was estimated to cost $791 per student. However, in this study the paraprofessionals were only able to serve a handful of students in each school because they have other duties at the school to which they have to attend.