(VOCAB): First Grade Follow-Up Impact Report and Exploratory Analyses of Kindergarten Impacts
Total Page:16
File Type:pdf, Size:1020Kb
NCEE 2012-4009 U. S. DEPARTMENT OF EDUCATION Effectiveness of a Program to Accelerate Vocabulary Development in Kindergarten (VOCAB): First Grade Follow-up Impact Report and Exploratory Analyses of Kindergarten Impacts Effectiveness of a Program to Accelerate Vocabulary Development in Kindergarten (VOCAB): First Grade Follow-up Impact Report and Exploratory Analyses of Kindergarten Impacts Final Report December 2011 Authors: Barbara Goodson, Principal Investigator Abt Associates Inc. Anne Wolf, Director of Evaluation Abt Associates Inc. Steve Bell, Task 2 Methodological Leader Abt Associates Inc. Herb Turner, Technical Consultant ANALYTICA Pamela B. Finney, Research Management Leader Regional Education Laboratory Southeast Project Officer: Sandra Garcia Institute of Education Sciences NCEE 2012–4009 U.S. Department of Education U.S. Department of Education Arne Duncan Secretary Institute of Education Sciences John Q. Easton Director National Center for Education Evaluation and Regional Assistance Rebecca A. Maynard Commissioner December 2011 This report was prepared for the National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, under contract ED–06-CO–0028. IES evaluation reports present objective information on the conditions of implementation and impacts of the programs being evaluated. IES evaluation reports do not include conclusions or recommendations or views with regard to actions policymakers or practitioners should take in light of the findings in the report. This report is in the public domain. Authorization to reproduce it in whole or in part is granted. While permission to reprint this publication is not necessary, the citation should read: Goodson, B., Wolf, A., Bell, S., Turner, H., & Finney, P. B. (2011). Effectiveness of a Program to Accelerate Vocabulary Development in Kindergarten (VOCAB): First Grade Follow-up Impact Report and Exploratory Analyses of Kindergarten Impacts (NCEE 2012–4009). Washington, DC: National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, U.S. Department of Education. This report is available on the Institute of Education Sciences website at http://ncee.ed.gov and the Regional Educational Laboratory Program website at http://edlabs.ed.gov. Alternate Formats. Upon request, this report is available in alternate formats, such as Braille, large print, audiotape, or computer diskette. For more information, please contact the Department’s Alternate Format Center at 202-260-9895 or 202-205-8113. DISCLOSURE OF POTENTIAL CONFLICT OF INTEREST1 None of the authors or other staff involved in the study from the Regional Educational Laboratory Southeast at the SERVE Center at the University of North Carolina at Greensboro, Abt Associates Inc., ANALYTICA, or Empirical Education Inc. has financial interests that could be affected by the content of this report. 1 Contractors carrying out research and evaluation projects for IES frequently need to obtain expert advice and technical assistance from individuals and entities whose other professional work may not be entirely independent of or separable from the tasks they are carrying out for the IES contractor. Contractors endeavor not to put such individuals or entities in positions in which they could bias the analysis and reporting of results, and their potential conflicts of interest are disclosed. iii ACKNOWLEDGEMENTS Many individuals and organizations contributed to the evaluation of the K-PAVE vocabulary intervention. We want to thank Dr. Hank Bounds, former State Superintendent of Education for Mississippi, and the other staff in the Mississippi Department of Education for their support in allowing us to conduct a random assignment evaluation and for communicating with district superintendents and school principals to facilitate recruitment of districts and schools for the study. In addition, we greatly appreciate the many school district superintendents, school principals, and teachers who were willing to open their classrooms and share their time with us. Many people from the SERVE Center at the University of North Carolina at Greensboro, the prime contractor for the Regional Educational Laboratory-Southeast that funded the study, were critical to the success of the project. Frankie Walton White, Alexa Edwards, Melissa Williams, Fain Barker, Lynn Amwake, Lucy Wynn, De Fuller, Glyn Brown, Teri White, Beth Thrift, and Teresa Lewis recruited participants, coordinated the intervention training and materials, trained data collectors, collected data, and identified first grade students for the follow up study. David Schaefer was critical in the document review and editing process. In addition, we want to thank the staff from Empirical Education Inc. for their accurate database entry of student assessments and classroom observations. Staff from Abt Associates was invaluable to many aspects of the project, including training data collectors and assisting with impact analyses. Carter Epstein developed rigorous training sessions for child assessors and classroom observers, and she lead a team of excellent trainers, including Kenyon Maree, Rachel McCormick, Cay Bradley, Alyssa Rulf Fountain, Megan Tiano, Laurie Bozzi, and Rachel Luck. Carolyn Layzer assisted with pilot testing of data collection instruments. Amanda Parsad was a crucial member of the impact analysis team. Thanks also go to Amanda Parsad, David Cook, and Tom McCall for their assistance in preparing the restricted-use data release. iv CONTENTS DISCLOSURE OF POTENTIAL CONFLICT OF INTEREST ..................................................................................................... III ACKNOWLEDGEMENTS ......................................................................................................................................... IV SUMMARY ................................................................................................................................................ 1 1. INTRODUCTION AND STUDY OVERVIEW ................................................................................................ 4 ROLE OF VOCABULARY KNOWLEDGE IN READING COMPREHENSION .................................................................................4 THE K-PAVE VOCABULARY INSTRUCTION PROGRAM ...................................................................................................5 WHY K-PAVE? ................................................................................................................................................... 6 THEORY OF CHANGE FOR K-PAVE ........................................................................................................................... 7 EXPERIMENTAL AND QUASI-EXPERIMENTAL EVIDENCE OF IMPACTS OF VOCABULARY INSTRUCTION ........................................8 THE FOCUS ON THE MISSISSIPPI DELTA REGION ........................................................................................................12 PRIOR FINDINGS AND RESEARCH QUESTIONS ............................................................................................................13 Results of the kindergarten study ............................................................................................................. 14 Analysis of sustained impacts on students in grade 1 .............................................................................. 15 Exploratory analysis of different impacts on subgroups of students and schools ....................................16 Exploratory analysis of kindergarten impacts on other outcomes ...........................................................17 ORGANIZATION OF THE REPORT.............................................................................................................................18 2. STUDY DESIGN AND METHODOLOGY ................................................................................................... 19 SAMPLE RECRUITMENT AND RANDOM ASSIGNMENT ..................................................................................................20 Recruitment of eligible districts, schools, and teachers ........................................................................... 20 Random assignment within blocks ........................................................................................................... 23 Random selection of classrooms .............................................................................................................. 24 Random selection of students .................................................................................................................. 26 Characteristics of schools in final study sample........................................................................................ 27 ATTRITION AND ANALYTIC SAMPLE IN THE KINDERGARTEN STUDY .................................................................................30 Attrition of schools and classrooms .......................................................................................................... 30 Attrition of students .................................................................................................................................. 31 Nonparticipants and crossovers during the kindergarten intervention year ...........................................33 ATTRITION AND ANALYTIC SAMPLE IN THE GRADE 1 FOLLOW-UP STUDY .........................................................................33 MEASURES AND DATA COLLECTION ........................................................................................................................36