NC Pre-K Program Evaluation Project Effects of the North Carolina Pre-Kindergarten Program Findings through Pre-K of a Small-Scale RCT Study 2017-2018 Statewide Evaluation Ellen Peisner-Feinberg, PhD, Sabrina Zadrozny, PhD, Laura Kuhn, PhD, & Karen Van Manen, PhD © February 2019 by Ellen S. Peisner-Feinberg, FPG Child Development Institute, The University of North Carolina at Chapel Hill. We wish to acknowledge the members of our NC Pre-K Program Evaluation Team who provided invaluable support and effort to conduct this study. Research staff – Dr. Treshawn Anderson, Shannon Casey, Alexis Kokoska, Tom Leggett, Cristina Luna- Evans, Judith Owen, Andrea Reubens, Maria Sanchez, Stefanie Schwartz, and Gina Walker. Data collectors – Shaunna Barbee, Carol Carter, Laura Clay, Evadine Codd, Kristine Cortina, Kelly Downing, Lori Dwyer, Aaron Freeman, Isaura Godinez, Kate Hodge, Diana Knechtel, Martha Lee, Cyndee Lohr, Dr. Irina Mokrova, Rosemary McNiel, Cecilia Pepper, Jeremy Pienik, Quinny Sanchez Lopez, Karen Stewart, Tia Walt McCallum, Towanda White-Miller and Susan Wyland Witek. Statistical consultant – Dr. Margaret Burchinal. In addition, we offer our appreciation to all those who participated in and assisted with this study, including the children and families, and the teachers, administrators, and other staff in the pre-k programs, and the staff of the North Carolina Division of Child Development and Early Education, Department of Health and Human Services. Cover by Gina Harrison. Suggested citation: Peisner-Feinberg, E., Zadrozny, S., Kuhn, L., & Van Manen, K. (2019). Effects of the North Carolina Pre-Kindergarten Program: Findings through Pre-K of a Small-Scale RCT Study. Chapel Hill, NC: The University of North Carolina, FPG Child Development Institute. This study was funded by the North Carolina Division of Child Development and Early Education, Department of Health and Human Services. The opinions expressed in this report do not necessarily reflect those of the funding agency. Table of Contents List of Tables ................................................................................................................................................. 3 List of Figures ................................................................................................................................................ 4 Overview of the NC Pre-K Program .............................................................................................................. 5 Purpose of the NC Pre-K Evaluation – Randomized Controlled Trial Study .................................................. 6 Method ......................................................................................................................................................... 7 Study Design ............................................................................................................................................. 7 Feasibility Study .................................................................................................................................... 7 Random Assignment Process ................................................................................................................ 7 Participants ............................................................................................................................................... 8 Children/Families .................................................................................................................................. 8 Teachers/Classrooms ............................................................................................................................ 9 Data Collection Procedures....................................................................................................................... 9 Measures ................................................................................................................................................. 10 Child Assessments ............................................................................................................................... 10 Teacher Surveys .................................................................................................................................. 10 Parent Surveys .................................................................................................................................... 11 Classroom Observations ..................................................................................................................... 11 State Administrative Data ................................................................................................................... 11 Analysis Approach ................................................................................................................................... 12 Weighting to Account for Potential Bias ............................................................................................. 12 Statistical Comparisons ....................................................................................................................... 12 Results ......................................................................................................................................................... 14 Descriptive Analyses ............................................................................................................................... 14 Treatment Effects – Full Sample ............................................................................................................. 14 Intent-to-Treat (ITT) – Full Sample ...................................................................................................... 15 Treatment-on-Treated (TOT) – Full Sample ........................................................................................ 15 Treatment Effects – DLL Subsample ....................................................................................................... 15 ITT - DLL Subsample ............................................................................................................................ 16 TOT - DLL Subsample ........................................................................................................................... 16 Program Characteristics .......................................................................................................................... 17 Trend Analyses ........................................................................................................................................ 18 1 Conclusions ................................................................................................................................................. 19 Appendix 1 .................................................................................................................................................. 53 List of NC Pre-K Evaluation Reports ............................................................................................................ 53 Appendix 2 .................................................................................................................................................. 55 Technical Information for Analysis Approach ............................................................................................. 55 Constructions of Weights to Account for Potential Bias ........................................................................ 55 Baseline Non-equivalence & Inverse Probability of Treatment Weights (IPTW) ............................... 55 Selection Bias & Inverse Probability of Censoring Weights (IPCW) .................................................... 55 Statistical Comparison Models ............................................................................................................... 56 References .............................................................................................................................................. 60 2 List of Tables Table 1. Characteristics of Sample Teachers and Classrooms .................................................................... 21 Table 2. Child Outcome and Classroom Quality Measures ......................................................................... 22 Table 3. Child Language Proficiency Levels ................................................................................................. 23 Table 4. Outcome Scores for Full Sample ................................................................................................... 24 Table 5. Outcome Scores for DLL Subsample – English Measures ............................................................. 26 Table 6. Outcome Scores for DLL Subsample – Spanish Measures ............................................................ 27 Table 7. Classroom Quality Scores .............................................................................................................. 28 Table 8. ITT Results – Full Sample ............................................................................................................... 29 Table 9. TOT Results – Full Sample ............................................................................................................. 31 Table 10. ITT Results - DLL Subsample English Outcomes .......................................................................... 32 Table 11. ITT Results - DLL Subsample Spanish Outcomes ........................................................................