Summative Evaluation Report
Total Page:16
File Type:pdf, Size:1020Kb
Pro grams for Partnerships in Early Learning Project PROPEL Early Learning Opportunities Act (ELOA) Grant Summative Evaluation Report ** Final DRAFT** For Discussion ONLY Submitted by School Program Evaluation and Research (SPEaR) Psychology and Research in Education School of Education University of Kansas May 2008 Report prepared by: PROPEL Project Evaluation Team School Program Evaluation and Research University of Kansas Steve Lee, Principal Investigator Bruce Frey, Co-Principal Investigator Jill Lohmeier, Co-Principal Investigator Vicki Schmitt, Evaluation Coordinator Erin Bennett, Graduate Research Assistant Jessica Oeth, Graduate Research Assistant The evaluation of this project was conducted by the School Program Evaluation and Research (SPEaR); an independent group of faculty and staff at the University of Kansas who provide evaluation, measurement, and assessment services to national, state and local organizations and research the evaluation process (Lee, Lohmeier, Frey & Tollefson, 2004) . SPEaR adheres to the Program Evaluation Standards (1994) developed by The Joint Committee on Standards for Educational Evaluation in all of their evaluation work. TABLE OF CONTENTS Executive Summary 1 Introduction 2 Program Description 2 Goals & Objectives 3 Background 4 Methods 7 Design 7 Sample 7 Data Collection 8 Instruments 9 Training 11 Analysis 12 Findings 13 Teacher Outcomes 13 Teacher Survey 13 ELLCO 17 Student Outcomes 21 PALS – PreK 21 G3 (PreK DIBELS) 24 Collaboration 27 Discussion 29 Appendix A: Professional Development Goals Appendix B: Teacher Survey item results Appendix C: ELLCO item results Appendix D: Collaboration Scale Project PROPEL Summative Evaluation Report _________________________________________________________ EXECUTIVE SUMMARY The Programs for Partnerships in Early Learning (PROPEL) program provided professional development in early literacy development to early childhood classroom teachers in seven childcare centers in Douglas County, Kansas along with several home day care providers. This report provides information collected from the teachers in the seven centers participating in the program as well as the students housed in each trained teacher’s classroom. Overall, teacher data suggest that training positively influenced the early literacy environment as well as the number and types of early literacy activities taking place each week in the classroom. In general, teacher efficacy toward providing a developmentally appropriate early literacy classroom increased over time. Teacher report data can be validated by evaluator observations of the classroom environment which also showed positive changes as a result of the PROPEL program. Increases in students’ oral language, phonological awareness, alphabet awareness, and book knowledge skills were also noted when comparing baseline data to data collected one year after the intervention took place. While not all increases are statistically significant, the meaningful of such growth may be important to program stakeholders given the short timeframe. Increased collaboration among grant partners took place during the implementation of PROPEL. While the current level of collaboration as reported by stakeholder representatives did not reach their target level of collaboration, the overall increase is important to note. In addition, increased collaboration among teachers within centers was also reported as a result of PROPEL providing additional evidence that the program positively influenced those involved. 1 Project PROPEL Summative Evaluation Report _________________________________________________________ INTRODUCTION Program Description The purpose of the Programs for Partnerships in Early Learning (PROPEL) program is to improve early learning experiences in literacy acuities through a professional development approach utilizing literacy coaches in Douglas county, Kansas. Approximately 24 early childhood teachers, six coaches and program directors from seven different early childhood centers took part in training. The professional development training, which focused on early language and literacy development, began in February 2006 and comprised five specific topics (1) literacy rich environment, (2) oral language, (3) print awareness, (4) alphabet knowledge, and (5) phonological awareness. Full implementation began during the summer-fall ’06 with six literacy coaches in place at the centers providing ongoing support for teachers. Figure 1. Project PROPEL Professional Development Model 2 Project PROPEL Summative Evaluation Report _________________________________________________________ The utilization of literacy coaches is an essential component of the program as they provided support for teachers by providing site-based training specific to each early childhood center’s needs focused on strategies for improving the quality of the language and literacy environment, instruction, and activities. As a result of this unique approach, changes in (1) teacher beliefs / practices, (2) the literacy environment in the classroom, and (3) student outcomes including oral language development, phonological awareness, and pre-reading abilities were examined. Figure 1 displays the professional development approach employed by the PROPEL program. Appendix A contains a full list of professional development goals. Grant Goals & Objectives Three overall goals and six objectives guide the PROPEL program. Each is organized around the unifying goal of building a common understanding of early literacy development. Conceptually, the objectives form the basis for implementation of activities designed to improve students’ access to quality early learning experiences. Following are the grant goals and objectives of PROPEL. GOAL 1: Development and implement an early literacy program for preschool children (3-5 years old) based on scientifically based reading research that prepares children in Douglas County for school. - Objective: Increase school readiness of preschool children in oral language and literacy skills. - Objective: Increase parents’ involvement in their child’s language and literacy skills development. GOAL 2: Provide a professional development model on scientifically based reading research that early childhood teachers in participating centers provide high quality language and literacy activities. 3 Project PROPEL Summative Evaluation Report _________________________________________________________ - Objective: Improve the knowledge and skills of teachers to enable them to provide research-based practices that develop young children’s language and literacy skills. - Objective: Improve planning and collaboration among existing early learning programs. - Objective: Improve the capacity of local resources and referral agencies to provide ongoing literacy professional development to its members, caregivers, and childcare teachers. GOAL 3: Improve access to early learning opportunities for children with special needs including developmental delays by providing technical assistance to early learning programs and enduring access to intervention services in these early learning programs. - Objective: Expand the number of programs with the capacity to deliver high quality early learning opportunities for children with special needs. Background According to the National Research Council (2001), children from high risk environments who participate in quality early learning experiences have a better chance of achieving positive outcomes, including, but not limited to academic achievement (Barnett, 1995). The focus on quality literacy in the early childhood classroom has become an area of increasing interest to educators, researchers, policy makers, and program evaluators with many programs now targeting early literacy development. Efforts to improve literacy rich classroom environments often require professional development training for early care professionals focused on oral language development, phonological awareness and pre-reading skills. The ultimate worth / value is not only the impact on teacher beliefs / practice, but also the positive impact on students’ overall literacy development. Research indicates that early experiences in quality literacy environments is positively related to later reading success in school (Storch & Whitehurst, 2002) and that by focusing on emergent literacy skills during the preschool years, 4 Project PROPEL Summative Evaluation Report _________________________________________________________ the number of school-age children with reading difficulties may be reduced (Snow, Burns, &Griffin, 1998). Early childcare providers are key factors in setting up quality early learning environments that positively impact children’s social, cognitive, and language development. In order to provide high-quality early childhood programs that focus on early literacy development, teachers in such environments must be adequately trained, knowledgeable of literacy development, and receive ongoing support for their efforts. Early care professionals need the instructional support that will allow them to facilitate the literacy development of the children in their classrooms (Snow, et al, 1998). The classroom environment becomes an important factor in promoting student success. And, it is the classroom teacher whose knowledge and understanding of early childhood development guides the creation of the learning environment. While some educators question the link between early childhood teachers’ beliefs and classroom practices (Wilcox-Herzog, 2002), others argue