Summative Evaluation Report

Pro grams for Partnerships in Early Learning

Project PROPEL

Early Learning Opportunities Act (ELOA) Grant

** Final DRAFT**

For Discussion ONLY

Submitted by

School Program Evaluation and Research (SPEaR) Psychology and Research in Education School of Education University of Kansas

May 2008

Report prepared by:

PROPEL Project Evaluation Team School Program Evaluation and Research University of Kansas Steve Lee, Principal Investigator Bruce Frey, Co-Principal Investigator Jill Lohmeier, Co-Principal Investigator Vicki Schmitt, Evaluation Coordinator Erin Bennett, Graduate Research Assistant Jessica Oeth, Graduate Research Assistant

The evaluation of this project was conducted by the School Program Evaluation and Research (SPEaR); an independent group of faculty and staff at the University of Kansas who provide evaluation, measurement, and assessment services to national, state and local organizations and research the evaluation process (Lee, Lohmeier, Frey & Tollefson, 2004) . SPEaR adheres to the Program Evaluation Standards (1994) developed by The Joint Committee on Standards for Educational Evaluation in all of their evaluation work.

TABLE OF CONTENTS

Executive Summary 1

Introduction 2 Program Description 2 Goals & Objectives 3 Background 4

Methods 7 Design 7 Sample 7 Data Collection 8 Instruments 9 Training 11 Analysis 12

Findings 13 Teacher Outcomes 13 Teacher Survey 13 ELLCO 17 Student Outcomes 21 PALS – PreK 21 G3 (PreK DIBELS) 24 Collaboration 27

Discussion 29

Appendix A: Professional Development Goals

Appendix B: Teacher Survey item results

Appendix C: ELLCO item results

Appendix D: Collaboration Scale

Project PROPEL Summative Evaluation Report ______

EXECUTIVE SUMMARY

The Programs for Partnerships in Early Learning (PROPEL) program provided professional development in early literacy development to early childhood classroom teachers in seven childcare centers in Douglas County, Kansas along with several home day care providers.

This report provides information collected from the teachers in the seven centers participating in the program as well as the students housed in each trained teacher’s classroom.

Overall, teacher data suggest that training positively influenced the early literacy environment as well as the number and types of early literacy activities taking place each week in the classroom. In general, teacher efficacy toward providing a developmentally appropriate early literacy classroom increased over time. Teacher report data can be validated by evaluator observations of the classroom environment which also showed positive changes as a result of the

PROPEL program.

Increases in students’ oral language, phonological awareness, alphabet awareness, and book knowledge skills were also noted when comparing baseline data to data collected one year after the intervention took place. While not all increases are statistically significant, the meaningful of such growth may be important to program stakeholders given the short timeframe.

Increased collaboration among grant partners took place during the implementation of

PROPEL. While the current level of collaboration as reported by stakeholder representatives did not reach their target level of collaboration, the overall increase is important to note. In addition, increased collaboration among teachers within centers was also reported as a result of PROPEL providing additional evidence that the program positively influenced those involved.

Project PROPEL Summative Evaluation Report ______

INTRODUCTION

Program Description

The purpose of the Programs for Partnerships in Early Learning (PROPEL) program is to improve early learning experiences in literacy acuities through a professional development approach utilizing literacy coaches in Douglas county, Kansas. Approximately 24 early childhood teachers, six coaches and program directors from seven different early childhood centers took part in training. The professional development training, which focused on early language and literacy development, began in February 2006 and comprised five specific topics

(1) literacy rich environment, (2) oral language, (3) print awareness, (4) alphabet knowledge, and

(5) phonological awareness. Full implementation began during the summer-fall ’06 with six literacy coaches in place at the centers providing ongoing support for teachers.

Figure 1. Project PROPEL Professional Development Model

Project PROPEL Summative Evaluation Report ______

The utilization of literacy coaches is an essential component of the program as they provided support for teachers by providing site-based training specific to each early childhood center’s needs focused on strategies for improving the quality of the language and literacy environment, instruction, and activities. As a result of this unique approach, changes in (1) teacher beliefs / practices, (2) the literacy environment in the classroom, and (3) student outcomes including oral language development, phonological awareness, and pre-reading abilities were examined. Figure 1 displays the professional development approach employed by the PROPEL program. Appendix A contains a full list of professional development goals.

Grant Goals & Objectives

Three overall goals and six objectives guide the PROPEL program. Each is organized around the unifying goal of building a common understanding of early literacy development.

Conceptually, the objectives form the basis for implementation of activities designed to improve students’ access to quality early learning experiences. Following are the grant goals and objectives of PROPEL.

GOAL 1: Development and implement an early literacy program for preschool children (3-5 years old) based on scientifically based reading research that prepares children in Douglas

County for school.

- Objective: Increase school readiness of preschool children in oral language and literacy skills.

- Objective: Increase parents’ involvement in their child’s language and literacy skills development.

GOAL 2: Provide a professional development model on scientifically based reading research that early childhood teachers in participating centers provide high quality language and literacy activities.

Project PROPEL Summative Evaluation Report ______

- Objective: Improve the knowledge and skills of teachers to enable them to provide research-based

practices that develop young children’s language and literacy skills.

- Objective: Improve planning and collaboration among existing early learning programs.

- Objective: Improve the capacity of local resources and referral agencies to provide ongoing literacy

professional development to its members, caregivers, and childcare teachers.

GOAL 3: Improve access to early learning opportunities for children with special needs including developmental delays by providing technical assistance to early learning programs and enduring access to intervention services in these early learning programs.

- Objective: Expand the number of programs with the capacity to deliver high quality early learning

opportunities for children with special needs.

Background

According to the National Research Council (2001), children from high risk environments who participate in quality early learning experiences have a better chance of achieving positive outcomes, including, but not limited to academic achievement (Barnett,

1995). The focus on quality literacy in the early childhood classroom has become an area of increasing interest to educators, researchers, policy makers, and program evaluators with many programs now targeting early literacy development. Efforts to improve literacy rich classroom environments often require professional development training for early care professionals focused on oral language development, phonological awareness and pre-reading skills. The ultimate worth / value is not only the impact on teacher beliefs / practice, but also the positive impact on students’ overall literacy development. Research indicates that early experiences in quality literacy environments is positively related to later reading success in school (Storch &

Whitehurst, 2002) and that by focusing on emergent literacy skills during the preschool years,

Project PROPEL Summative Evaluation Report ______the number of school-age children with reading difficulties may be reduced (Snow, Burns,

&Griffin, 1998). Early childcare providers are key factors in setting up quality early learning environments that positively impact children’s social, cognitive, and language development. In order to provide high-quality early childhood programs that focus on early literacy development, teachers in such environments must be adequately trained, knowledgeable of literacy development, and receive ongoing support for their efforts. Early care professionals need the instructional support that will allow them to facilitate the literacy development of the children in their classrooms (Snow, et al, 1998). The classroom environment becomes an important factor in promoting student success. And, it is the classroom teacher whose knowledge and understanding of early childhood development guides the creation of the learning environment.

While some educators question the link between early childhood teachers’ beliefs and classroom practices (Wilcox-Herzog, 2002), others argue that the “set of beliefs teachers have constructed act as the lens through which they view their practices” (Bleim & Davinroy, 1997, p.1).

Understanding how teacher beliefs inform their practice as they set up the learning environment is important to those interested in measuring the impact of programs focused on teacher professional development; especially when the target population includes many children facing multiple risk factors.

Professional development that focuses on improving teacher competency to provide high quality language and literacy experiences for young children are aimed at setting high quality standards for the early childhood literacy environment. The PROPEL project focuses on one such professional development program comprising five language and literacy outcomes for young children: (1) literacy rich environment, (2) oral language, (3) print awareness, (4) alphabet knowledge, and (5) phonological awareness. Utilizing a teacher / coach professional

Project PROPEL Summative Evaluation Report ______development model teachers receive ongoing training in early literacy and language development. Ongoing professional development and in-class supports provided by trained experts results in improved teacher practices in the classroom (National Research Council,

2001). The role of the literacy coach is ultimately to provide support and provide teachers with strategies for quality language and literacy activities. While this approach to professional development appears to be aligned with what many advocate for early childhood teacher training

(Snow, Burns, &Griffin, 1998), the impact of such a training model on student outcomes warrants examination. Evaluators of this Early Learning Opportunities grant were interested in documenting the degree to which pre-school teachers’ beliefs and practices change as a result of the professional development / coaching received, and whether or not these changes could be supported based on the scores obtained from a structured classroom observation. In addition, student outcomes including oral language development, phonological awareness, and pre-reading abilities are also examined in an effort to examine how teachers’ belief and practices regarding quality learning experience impact early literacy development.

This summative evaluation report describes findings from analyses of data from spring

2006 though the fall 2007, consisting of both teacher related outcomes as a result of the program and the impact on student language and literacy achievement. Teachers’ perceptions of the early literacy professional development and the effect that training had on their beliefs and practices in the classroom are examined using a structured classroom observation protocol and teacher report. Student outcomes including oral language development, phonological awareness, and pre-reading abilities are also examined to determine the overall program impact on students’ early literacy development.

Project PROPEL Summative Evaluation Report ______

METHODS

Evaluation Design

According to Oliva (1988) evaluation is defined as the “process of obtaining and providing useful information for judging decision alternatives” (p. 476). While there are many models that exist for the purpose of evaluation, the methodology and design for PROPEL followed a quasi- experimental design and an objectives-based approach (Madau & Stufflebeam, 1988) which employed elements of the Research Evaluation Model (REM) developed by the School Program

Evaluation and Research (SPEaR) team at the University of Kansas (Lee, Lohmeier, Frey, &

Tollefson, 2004), evaluators for this project. In addition, SPEaR utilized an approximation of

Stufflebeam’s Context-Input-Process-Product (CIPP - Stufflebeam, 1971; Stufflebeam &

Shinkfield, 1985) incorporating scientifically valid evaluation procedures (Ginsberg & Rhett,

2003) while adhering to the Program Evaluation Standards (1994) developed by the Joint

Committee on Standards for Educational Evaluation. Permission from the Internal Review

Board (IRB) at the University of Kansas was obtained November, 2005 through January 2008.

Sample

Participants in this evaluation include early childhood classroom teachers receiving professional development training from seven early childhood centers in Douglas County,

Kansas as well as the students (ages 3-5 year old) housed in classrooms where teachers received training. Seven early childhood centers took part in PROPEL impacting 24 early childhood settings. While 24 classroom teachers began training in February 2006, changes in staffing that took place during the 2006-07 school year reduced the number of classroom to 23 with many new teachers receiving training in the fall 2006. Overall, permission to collect data from more than 300 students was obtained.

Project PROPEL Summative Evaluation Report ______

Data Collection

Utilizing a quasi-experimental design, data were collected from approximately 22 teachers in seven childcare centers through use of an evaluator developed survey questionnaire examining (1) efficacy in early literacy and (2) early literacy classroom practices. In addition, the Early Language and Literacy Classroom Observation (ELLCO) toolkit comprising three separate research tools (1) Literacy Environment Checklist, (2) Classroom Observation / Teacher

Interview, and (3) Literacy Activities Rating Scale (Smith, MW. & Dickinson, D.K., 2002) was used as a measure of the classroom environment. These tools provided a comprehensive approach to describing the support for language and literacy development provided in the early childhood classroom.

Student assessments were also completed on approximately 334 students whose parents gave consent for their participation in data collection. The Phonological Awareness Literacy

Screening for Preschool (PALS-Pre-K) was used to assess young children’s understanding of emergent literacy concepts such as (1) Name Writing, (2) Alphabet Awareness, (3) Beginning

Sound Awareness, (4) Print and Word Awareness, and (5) Rhyme Awareness. In addition the

Pre-K DIBELS (i.e. G3) was also used as a measure of (1) Oral Language, (2) Rhyming and (3)

Alliteration.

Baseline data were collected during the spring 2006 from teachers (i.e. teacher survey and ELLCO) prior to teacher training and the implementation of the coaching model. Student baseline assessments were also conducted at this time. Following the training and implementation of the coaching model, teacher and student data were collected again at two times (1) fall 2006 and (2) spring 2007. Additional data were collected from teachers during the fall 2007 in an effort to examine sustained classroom effects.

Project PROPEL Summative Evaluation Report ______

Instruments

Teacher Survey

A teacher survey was developed by the evaluation team in an effort to gauge teachers’ perception of (1) Efficacy toward Early Literacy and (2) Early Literacy Classroom Practices,

Data collected from approximately 22 teachers in seven childcare centers through use of an evaluator developed survey questionnaire comprising three scales (1) early literacy classroom practices, (2) efficacy toward early literacy, and (3) teacher beliefs regarding early literacy, . In addition, items pertaining to the types of early literacy activities taking place each week and teacher perceptions of the professional development were examined

(see Appendix A).

Early Language and Literacy Classroom Observation (ELLCO)

The Early Language and Literacy Classroom Observation (ELLCO) toolkit comprising three separate research tools (1) Literacy Environment Checklist, (2) Classroom Observation /

Teacher Interview, and (3) Literacy Activities Rating Scale (Smith, MW. & Dickinson, D.K.,

2002) was used as a measure of the classroom environment. These three tools provide a comprehensive approach to describing the support for language and literacy development provided in the early childhood classroom. Together, the tools can be used to develop a profile of the early literacy environment of classrooms serving children ages 3-8 years. The reliability and validity of the observation tool is ultimately based on the level of training observers receive and they understanding of early literacy development. When trained, inter-rater agreement of 81% or higher on each of the tools can be obtained.

Project PROPEL Summative Evaluation Report ______

Phonological Awareness Literacy Screening for Preschool (PALS-PreK)

In addition to these teacher data, student assessments were also completed on approximately 334 students whose parents gave consent for their participation in data collection.

[During the spring 2006 baseline data with follow-up data collected from students during the fall

2006 and spring 2007.] The Phonological Awareness Literacy Screening for Preschool (PALS-

PReK) an assessment of young children’s understanding of emergent literacy concepts such as 1)

Alphabet Awareness (upper and lower case letters), 2) Beginning Sound Awareness, 3) Print and

Word Awareness, and 4) Rhyme Awareness, was used to examine student literacy outcomes.

Efforts to document the psychometric properties of the instrument found both internal consistency and inter-rater reliability estimates to be within an acceptable range and tend to be stable across a broad representation of children. Evidence of content validity of the PALS-PreK is based on a thorough review of the literature as well as an item review conducted by an advisory board of experts in early literacy development. Construct validity evidence is supported through an exploratory factor analysis of the pilot items yielding a single factor, which suggest the PALS-PreK measures a single “emergent literacy” trait. In addition, scores obtained from the

PALS-PreK correlate with the Test of Awareness of Language Segments (TALS) r = .41; The

Child Observation Record (COR) r = .71; and the Test of Early Reading Abilities (TERA-3) r =

.67. The PALS-PreK is also significantly predictive of the performance on the PALS-K (R2 =

.84) and the PALS 1-3 (R2 = .342). It should be noted that the PALS-PreK is only appropriate for student age 4-5 years who have yet to enter kindergarten.

Get it! Got it! Go! (G3) Individual Growth and Development Indicators (IGDIs)

Because many of the children in the sample ranged in age from 3-5years, the G3 also known as the Pre-K DIBELS was used as a measure of (1) Oral Language, (2) Rhyming and (3)

Project PROPEL Summative Evaluation Report ______

Alliteration. Students younger than four received only the G3 while students 4-5 years old took part in both the G3 and PALS assessments. While the G3 consists of one and two minutes probes, it has been found to be stable over time and is sensitive to preschoolss’ growing language development. The G3 or PreK DIBELS correlates well with other standardized measures of language development including the Peabody Picture Vocabulary Test (PPVT), Preschool

Language Scale (PLS-3), and Test of Phonological Awareness (TOPA).

Collaboration Scale

In an effort to examine the change in collaboration among grant partners, evaluators used the Levels of Collaboration Scale (Frey, Lohmeier, Lee, & Tollefon; 2006). The scale is shortened version of the levels of community linkages (Hogue, 1993) discussed by Borden and

Perkins (1198, 1999). The scale consists of five stages (1) Networking, (2) Cooperation, (3)

Coordination, (4) Coalition, and (5) Collaboration. A representative from each stakeholder group (typically those attending stakeholder meetings) was asked to respond to the survey noting their agencies level of collaboration with other grant partners. Data were collected prior to full implementation and then again at two additional time points to determine if the level of collaboration among partner increased as a result of the PROPEL project.

Training

During the fall 2005, assessment specialists received training on all evaluation instruments prior to baseline data collection spring 2006. Assessment specialists included graduate students from the university’s School Psychology program who already had an understanding of standardized test administration. Follow-up trainings were conducted during the summer/ fall 2006. Because instruments such as the ELLCO called for observer judgment, periodic reliability checks were also conducted in an effort to ensure the quality of data collected.

Project PROPEL Summative Evaluation Report ______

Data Analysis

Both descriptive and quasi-experimental techniques were used in this evaluation.

Descriptive statistics were used to report teacher survey and ELLCO data as well as student assessment data. Analysis of variance techniques were used to determine changes in teacher perceptions and classroom activities changes over time as well as changes in student outcomes compared to baseline. While a within group analysis would have been the optimal method for examining student level data, the timeframe for implementation of the project made such an approach nearly impossible. With baseline data collected from students during the spring 2006, more than two-thirds of those assessed did not participate in the full implementation of the program during the 2006-07 school year as many of the 4-5 year olds moved on to Kindergarten during the academic year. Thus, a between groups analysis where student data collected at baseline acted as the comparison group allowed evaluators to examine the effects of the intervention on students 3, 4, 5 years old at two different time points. The first being fall 2006 where students in centers had teachers who had been trained and were implementing early literacy practices for approximately 6 months and then again in the spring after nearly one year of implementation. By disaggregating the data at each time point by the age of the student, effects for students within various age brackets could be examined [e.g. students 4 years of age at baseline compared to students 4 years of age during the fall 2006 (6 mth of the intervention in place) and then again to students 4 year olds in the spring 2007 (one year of implementation)].

In this way, controls for students’ language development were made as the demographics of the sample remained stable over the course of the evaluation period.

Project PROPEL Summative Evaluation Report ______

FINDINGS

Teacher Outcomes

Teacher Survey

In an effort to measure the types of literacy practices taking place in the classroom and teachers’ efficacy toward promoting early literacy, a teacher survey was administered during the spring 2006 (baseline), fall 2007 and spring 2006. As Table 1 and Figure 1show, teacher beliefs regarding the types of literacy activities taking place in their respective classrooms as well as their efficacy toward early literacy increased during the grant period (Spring 2006-Spring 2007).

In addition, the teacher survey was administered again to teachers during the fall 2007 in an effort to determine the degree to which gains may have been sustained.

Table 1. Mean scores and standard deviations for teacher survey scales (spring 2006 – fall 2007) . Sp06 (n = 22) Fa06 (n = 21) Sp07 (n = 21 Fa07 (n = 15) Scale M (sd) M (sd) M (sd) M (sd) Literacy Practices 4.04 (.48) 4.30 (.43) 4.41 (.32) 4.38 (.43) Literacy Efficacy 3.98 (.45) 4.40 (.45) 4.40 (.45) 4.54 (.31) Beliefs About Literacy* 4.63 (.41) Grand Mean 4.22 (.45) 4.32 (.44) 4.41 (.39) 4.46 (.37) *Note: An additional scale measuring Teacher Beliefs About Early Literacy was administered during baseline only. Due to the overwhelming positive response from teachers, stakeholders requested these items be removed from future versions of the survey instrument.

Overall, it appears as though the biggest increase in teacher perception took place after professional development took place in the spring 2006 and teachers began implementing early literacy strategies during the fall 2006. However, it should be noted that these data are negatively skewed; potentially a result of the social desirability of the self report measure.

Project PROPEL Summative Evaluation Report ______

Figure 1. Teachers’ report of Literacy Practices & Efficacy from the teacher survey (spring 2006 – fall 2007)

Spring 06 Fall 06 Spring 07 Fall 2007 5.00 4.54 4.41 4.40 4.40 4.50 4.30 4.38 4.04 3.98 4.00

3.50

3.00

Mean 2.50

2.00

1.50

1.00 Literacy Activities Literacy Efficacy

In addition to the literacy practices and efficacy scales, teachers also reported on the number of times per week they engaged in specific literacy activities. These activities were tied specifically to the training in early literacy development that teachers’ received prior to full implementation. Table 2 and Figures 2a-c show the growth in the number of times teachers reported engaging in specific literacy activities over the course of the grant cycle. The largest increase again appears to be between spring 2006 (baseline) and fall 2006. In addition, when teachers completed the follow-up survey in the fall 2007 data indicate that the overall impact of

PROPEL appears to be sustained as teachers continue to engage in more literacy activities each week when compared to baseline. Perhaps more interesting is the variability in the number of times per week teachers report engaging in literacy activities. Appendix B of the report contains the item level data for the teacher survey.

Project PROPEL Summative Evaluation Report ______

Table 2. Mean scores and standard deviations for teacher reported literacy activities from the teacher survey (spring 2006 – fall 2007) . Sp06 (n = 22) Fa06 (n = 21) Sp07 (n = 21 Fa07 (n = 15) Literacy Activities M (sd) M (sd) M (sd) M (sd) Phonemic awareness activities 3.86 (2.21) 5.05 (4.38) 5.43 (4.69) 5.36 (3.43) Read Alouds 6.27 (2.68) 7.90 (3.77) 7.86 (3.90) 7.14 (3.01) Alphabet awareness reading 3.82 (2.04) 4.42 (3.47) 4.36 (2.02) 4.64 (2.17) Re-reading of favorite books 3.45 (2.11) 5.05 (3.20) 4.23 (2.24) 5.00 (3.96) Alphabet awareness writing 3.32 (2.32) 4.63 (2.41) 4.29 (2.23) 4.71 (2.67) Teacher-directed writing activities 3.14 (2.51) 3.48 (1.54) 4.43 (4.70) 4.07 (2.27) Child-directed writing activities 3.36 (2.08) 5.10 (2.25) 4.46 (2.22) 4.79 (2.69) Vocabulary activities 2.73 (2.27) 4.05 (3.68) 4.42 (2.80) 4.08 (2.63) Vocabulary words presented 3.64 (2.94) 2.60 (2.04) 3.42 (1.51) 3.77 (2.45) Extension activities 3.29 (2.17) 4.44 (2.62) 4.00 (1.65) 4.15 (2.03) Grand Mean 3.69 (2.33) 4.67 (2.94) 4.69 (2.80) 4.77 (2.73)

Figure 2a. Teachers’ self-report of the number of times engaged in literacy activities each week from the teacher survey (spring 2006 – fall 2007)

Spring 06 Fall 06 Spring 07 Fall 2007 7.9 7.9 8.0 7.1 7.0 6.3

6.0 5.4 5.4 5.1 5.1 5.0 5.0 4.6 4.4 4.4 4.2 4.04 4.0 3.8

3.5 Mean

3.0

2.0

1.0 Phonemic Awareness Read Alouds Alphabet Awareness -Reading Re-reading Favorite Books

Project PROPEL Summative Evaluation Report ______

Figure 2b. Teachers’ self-report of the number of times engaged in literacy activities each week from the teacher survey (spring 2006 – fall 2007)

Spring 06 Fall 06 Spring 07 Fall 2007 8.0

7.0

6.0 5.1 4.8 5.0 4.6 4.7 4.5 4.3 4.4 4.1 4.0 3.5

Mean 4.04 3.4 3.1 3.0

2.0

1.0 Alphabet Awareness -Writing Teacher-directed Writing Child-directed Writing

Figure 2c. Teachers’ self-report of the number of times engaged in literacy activities each week from the teacher survey (spring 2006 – fall 2007)

Spring 06 Fall 06 Spring 07 Fall 2007 8.0

7.0

6.0

5.0 4.4 4.4 4.2 4.1 4.1 4.0 4.0 3.8 3.4 Mean 4.04 3.3

3.0 2.6 2.3

2.0

1.0 Vocabulary Words Vocabulary Activities Extension Activities

Project PROPEL Summative Evaluation Report ______

ELLCO

Along with the teacher survey, classroom observations were also conducted by trained assessment specialists utilizing the Early Language and Literacy Classroom Observation

(ELLCO) toolkit in an effort to determine the effects of training on the classroom literacy environment. The ELLCO toolkit contains three scales (1) Literacy Environment Checklist, (2)

Classroom Observation, and (3) Literacy Activities Rating Scale. As with the teacher survey, baseline data were again collected prior to full implementation during the spring 2006 and follow-up data collection took place during the implementation year (fall 2006 and spring 2007).

Additionally, classroom observations were also conducted during the fall 2007 to examine the sustained effects of PROPEL. The specific items associated with each scale along with teacher data for each item are reported in Appendix C.

The Literacy Environment Checklist is a 24 item scale used to examine the presences of types of literacy materials available in the classroom. There are five specific areas considered within this scale: book area, book selection, book use, writing materials, and writing around the room. Table 3a provides the number of possible points for each section as they vary greatly.

Mean scores and standard deviations at each data collection point are also listed.

Table 3a. Mean scores and standard deviations for ELLCO Literacy Environment Checklist scale compared to the number of possible points (spring 2006 – fall 2007) . #Points Sp06 (n = 24) Fa06 (n = 21) Sp07 (n = 23) Fa07 (n = 20) Literacy Environment Checklist Possible M (sd) M (sd) M (sd) M (sd) Book Area 3 2.58 (.78) 2.71 (.56) 2.70 (.63) 2.95 (.22) Book Selection 8 6.54 (1.44) 7.432 (.81) 7.17 (1.44) 7.15 (1.23) Book Use 9 2.96 (2.79) 3.76 (2.64) 4.13 (2.91) 4.60 (3.23) Writing Materials 8 6.13 (2.05) 7.00 (1.48) 7.04 (1.40) 6.50 (1.76) Writing Around the Room 13 4.42 (2.54) 5.76 (2.57) 6.26 (3.11) 6.65 (2.46) Total /Average 41 22.63 (7.14) 26.67 (6.44) 27.30 (7.27) 27.85 (7321)

Project PROPEL Summative Evaluation Report ______

The Classroom Observation scale of the ELLCO utilizesa five-point Likert-type rating system with a score of 1 = Deficient; 3 = Basic, and 5 = Exemplary. The Classroom Observation scale is comprised of two different components (1) The General Classroom Environment and (2)

Language, Literacy Curriculum. Table 3b below lists the categories contained in each component as well as the total number of points possible within each, along with the mean score and standard deviation at each data collection time point.

Table 3b. Mean scores and standard deviations for ELLCO Classroom Observation scale compared to the number of possible points (spring 2006 – fall 2007)

#Points Sp06 (n = 24) Fa06 (n = 21) Sp07 (n = 23) Fa07 (n = 20) Classroom Observation Possible M (sd) M (sd) M (sd) M (sd) General Classroom Environment 25 17.13 (2.83) 20.29 (3.13) 22.04 (3.69) 21.85 (3.86) Organization of Classroom 5 3.42 (.72) 4.14 (.73) 4.13 (.76) 4.50 (.76) Contents of Classroom 5 3.00 (.66) 3.33 (.86) 3.59 (.73) 3.90 (.64) Presence/Use of Technology* 5 2.79 (.78) 3.05 (.80) 3.13 (.81) 3.20 (.89) Opportunities for Child Choice 5 3.33 (.64) 3.48 (.60 ) 3.61 (.72) 3.85 (.59) Classroom Management 5 3.42 (.78) 3.86 (.57) 3.96 (.64) 4.00 (.86) Classroom Climate 5 3.96 (.91) 4.10 (.70) 4.17 (.83) 4.25 (.79) Language / Literacy Curriculum 40 25.42 (6.16) 29.19 (3.83) 30.13 (3.89) 60.40 (5.78) Oral Language 5 3.42 (.88) 3.57 (.60) 3.74 (.45) 4.10 (.85) Presence of Books 5 3.54 (1.06) 3.86 (.96) 3.87 (.87) 4.25 (.72) Approaches to Reading 5 3.71 (.62) 4.10 (.70) 4.35 (.49) 4.00 (.73) Approaches to Writing 5 3.09 (.90) 3.52 (.68) 3.61 (.78) 3.50 (089) Curriculum Integration 5 3.13 (1.12) 3.67 (.66) 3.83 (.72) 3.80 (.83) Recognizing Diversity 5 2.79 (1.14) 3.38 (.74) 3.48 (.59) 3.60 (.60) Home Support for Literacy 5 3.21 (1.06) 3.29 (.72) 3.30 (.63) 3.55 (.76) Approaches to Assessment 5 3.00 (.88) 3.81 (1.21) 3.83 (.89) 3.75 (1.07) Total /Average 70 45.17 (8.23) 51.38 (6.55) 53.13 (6.58) 54.10 (9.14) *Note: this item is not included in the general classroom environment sub-scale score but is added back in for the total score.

Project PROPEL Summative Evaluation Report ______

The Literacy Activities Rating Scale is the final portion of the ELLCO and is made up of nine items intended to measure book reading and writing in the classroom. In order to rate these two categories, raters must observe reading and writing activities taking place in the classroom.

Figure 3c lists the two categories, possible points for each and the means scores (standard deviations) across time points. Again, the number of points within each section varies.

Table 3c. Mean scores and standard deviations for ELLCO Literacy Activities Rating scale compared to the number of possible points (spring 2006 – fall 2007) . #Points Sp06 (n = 24) Fa06 (n = 21) Sp07 (n = 23) Fa07 (n = 20) Literacy Environment Checklist Possible M (sd) M (sd) M (sd) M (sd) Book Use 9 4.08 (1.79) 5.00 (.71) 5.17 (.89) 4.75 (1.02) Writing Materials 8 1.5 (1.91) 2.24 (1.64) 2.30 (1.69) 2.70 (1.63) Total /Average 17 5.50 (2.47) 7.24 (1.79) 7.48 (2.11) 7.45 (2.11)

Figures 3a-b display the average total scores for each time period for each of the ELLCO scales. Effect size estimates were examined by comparing spring 2006 scores to spring 2007 for:

General Classroom Environment d = 1.49 (r = .598), and Language, Literacy, Curriculum d = .92

(r = .421), both indicating a large effect. Based on the data collected during the fall 2007, the effects of the implementation of the PROPEL project on the classroom environment appears to be sustained with regard to the classroom literacy environment (see Figure 3a). In addition to the classroom observation portion of the ELLCO, the average total score for the two other subscales

(i.e. literacy environment checklist and literacy activities rating scale) can be found in Figure 3b.

It should be noted that these two scales are comprised of a variety of different items and total scores for each vary. As indicated above, the overall possible total score for the Literacy

Activities Rating scale is 17. Appendix C contains the item level data from these two subscales.

Project PROPEL Summative Evaluation Report ______

Figure 3a. Average total scores for the Classroom Observation components General Classroom Environment and Language / Literacy and Curriculum from the ELLCO (spring 2006 – fall 2007)

Spring 06 17.13 Fall 06

18.83 Spring 07 General Classroom Environment Fall 07 22.04

21.85

25.42

29.33 Language, Literacy, and Curriculum 30.13

30.4

0 5 10 15 20 25 30 35 40

Figure 3b. Average total scores for the Literacy Activities Rating Scale and Literacy Environment Subscales from the ELLCO (spring 2006 – fall 2007)

5.5

7.23 Literacy Activities Spring 06 Rating Scale 7.47 Fall 06 Spring 07 7.45 Fall 07

22.63

26.67 Literacy Environment 27.3

27.8

0 5 10 15 20 25 30 35 40

Project PROPEL Summative Evaluation Report ______

Student Outcomes

Student outcomes measured by the Phonological Awareness Literacy Screening (PALS)

Pre-K and Pre-K DIBELS indicate that students in early childhood classrooms where teachers received training may have been positively impacted by the increased focus on early literacy activities taking place as a result of the training intervention. When examining differences between spring 2006 (baseline) and spring 2007 (one year after implementation), the greatest effects are noted. This is especially evident for children 4-5 years of age. The following tables and figures provide these data disaggregated by age level (i.e. 3 yr, 4 yr, and 5 yr olds).

PALS - PreK

The PALS-PreK was administered to 4-5 year old students in classrooms of teachers who had received training and whose parents had given consent for assessment. Six subtests from the assessment were examined to determine if students in classrooms prior to training (spring 2006) differed from students in those same classrooms with teachers training specifically in early literacy development (fall 2006, spring 2007). Table 4 provides the mean scores, standard deviations, and sample size for both the four year olds and the five year olds tested during each data collection period. Figures 4a-4d show these same data graphically. While gains in alphabet awareness (i.e. upper case letters, lower case letters, letter sounds), book knowledge and phonological awareness (i.e. beginning sounds, rhyming) occurred for 4 year olds, this does not appear to be the case for 5 year olds. However, due to the variability of data collected within the four year old group, these differences were not found to be significant (p < .05). When data are examined in aggregated using a within groups analysis (fall 06 to spring 07), differences in mean scores for each of the subtests are significant (p < .01). Without comparison to baseline, there is no way to know if teacher training influenced these gains.

Project PROPEL Summative Evaluation Report ______

Table 4. Means scores, standard deviations, and total points possible on the Phonological Awareness Literacy Screening (PALS) PreK subtests by age level (spring 2006 – fall 2007)

#Points Sp06 Fa06 Sp07 PALS- PreK Possible M (sd) M (sd) M (sd) 4 yr olds (n = 75) (n = 100) (n = 57) Upper Case Letters 26 11.76 (9.60) 11.62 (9.65) 13.60 (10.43) Lower Case Letters 26 6.93 (9.17) 6.70 (9.36) 9.21 (10.38) Letter Sounds 26 2.80 (5.16) 3.21 (6.41) 4.96 (6.99) Beginning Sounds 10 5.60 (3.64) 6.91 (3.17) 7.04 (3.06) Book Knowledge 10 6.30 (2.27) 5.93 (2.55) 6.74 (2.54) Rhyming 10 5.66 (2.76) 5.57 (3.17) 6.42 (3.09) 5 yr olds (n = 64) (n = 31) (n = 73) Upper Case Letters 26 19.36 (8.33) 16.16 (8.85) 17.27 (8.43) Lower Case Letters 26 15.67 (8.96) 9.94 (10.29) 11.79 (10.25) Letter Sounds 26 9.00 (7.43) 6.58 (8.43) 7.69 (8.32) Beginning Sounds 10 7.97 (3.20) 7.19 (3.35) 8.49 (1.88) Book Knowledge 10 7.63 (2.26) 7.29 (1.88) 7.69 (1.90) Rhyming 10 7.39 (3.20) 6.87 (3.23) 7.36 (2.84)

Figure 4a. Means scores for 4 yr olds on the Phonological Awareness Literacy Screening (PALS) PreK subtests: Upper Case / Lower Case Letters and Letter Sounds (spring 2006 – spring 2007)

26 24

22 Spring 06 20 Fall 06 18 Spring 07 16 13.6 14 11.8 11.6 12 10 9.2 8 6.9 6.7 6 5.0 4 2.8 3.2 2 0 Upper Case Lower Case Letter Sounds

Project PROPEL Summative Evaluation Report ______

Figure 4b. Means scores for 4 yr olds on the Phonological Awareness Literacy Screening (PALS) PreK subtests: Beginning Sounds, Book Knowledge, and Rhyming (spring 2006 – spring 2007)

10 Spring 06 Fall 06 8 Spring 07 7.0 6.9 6.7 6.3 6.4 5.9 6 5.6 5.7 5.6

0 Beginning Sounds Book Knowledge Rhyming

Figure 4c. Means scores for 5 yr olds on the Phonological Awareness Literacy Screening (PALS) PreK subtests: Upper Case / Lower Case Letters and Letter Sounds (spring 2006 – spring 2007)

26 24 22 Spring 06 19.3 20 Fall 06 18 17.3 16.2 15.7 Spring 07 16 14 11.8 12 10 9.3 9.0 7.7 8 6.6 6 4 2 0 Upper Case Lower Case Letter Sounds

Project PROPEL Summative Evaluation Report ______

Figure 4d. Means scores for 5 yr olds on the Phonological Awareness Literacy Screening (PALS) PreK subtests: Beginning Sounds, Book Knowledge, and Rhyming (spring 2006 – spring 2007)

Spring 06 10 Fall 06 8.5 Spring 07 8.0 8 7.6 7.7 7.2 7.3 7.4 7.4 6.9

0 Beginning Sounds Book Knowledge Rhyming

G3 (PreK-DIBELS)

Due to the fact that the PALS-PreK can only be administered to students at least four years of age and the fact that there were classrooms with three year olds, an additional tool was used to gather data on oral language and phonological awareness development. This tool knownas the G3 or PreK-DIBELS consisted for three subtests given as either one or two minute probes. Table 5 and Figures 5a-c display the mean scores and standard deviations for 3, 4, and 5 year olds completing this assessment. While increases in oral language as measured by the

Picture Naming subtest are noted for each age group, only 4-5 year olds’ phonological awareness skills (Alliteration and Rhyming) appear to have increased when compared to baseline.

Sum scores for each group were examined for each group. Differences in four and five year old scores exist when comparing baseline to spring 2007 (p<.05) . However, differences in three year old scores are not significant.

Project PROPEL Summative Evaluation Report ______

Table 5. Means scores and standard deviations, for the PreK-DIBELS or G3 subtests by age level (spring 2006 – fall 2007)

Sp06 Fa06 Sp07 G3 (PreK-DIBELS) M (sd) M (sd) M (sd) 3 yr olds (n = 59) (n = 44) (n = 26) Picture Naming 12.49 (5.07) 13.80 (5.78) 15.15 (7.56) Alliteration 2.25 (3.18) .84 (1.57) .50 (1.14) Rhyming 1.09 (2.42) 1.31 (2.24) .38 (1.36) 4 yr olds (n = 123) (n = 129) (n = 66) Picture Naming 16.57 (5.49) 19.46 (6.51) 20.97 (6.66) Alliteration 3.24 (3.83) 2.82 (3.85) 3.85 (4.69) Rhyming 3.52 (4.36) 4.02 (4.57) 5.33 (5.44) 5 yr olds (n = 98) (n = 25) (n = 91) Picture Naming 18.86 (5.77) 20.56 (6.35) 27.24 (7.04) Alliteration 5.50 (5.19) 4.52 (4.26) 7.09 (4.94) Rhyming 7.16 (5.84) 5.16 (4.35) 9.41 (6.12)

Figure 5a. Means scores for 3 yr olds on the G3 / PreK-DIBELS subtests: Picture Naming, Alliteration, and Rhyming (spring 2006 – spring 2007)

26 24 22 Spring 06 20 Fall 06 18 Spring 07 16 15.2 13.8 14 12.5 12 10 8 6

4 2.3 1.1 1.3 2 0.8 0.5 0.4 0 Picture Naming Alliteration Rhyming

Project PROPEL Summative Evaluation Report ______

Figure 5b. Means scores for 4 yr olds on the G3 / PreK-DIBELS subtests: Picture Naming, Alliteration, and Rhyming (spring 2006 – spring 2007)

26 24 22 21.0 Spring 06 19.5 20 Fall 06 18 16.6 Spring 07 16 14 12 10 8 6 5.3 3.9 3.5 4.0 4 3.2 2.8 2 0 Picture Naming Alliteration Rhyming

Figure 5c. Means scores for 5 yr olds on the G3 / PreK-DIBELS subtests: Picture Naming, Alliteration, and Rhyming (spring 2006 – spring 2007)

28 27.2 26 24 Spring 06 20.6 22 Fall 06 20 18.9 Spring 07 18 16 14 12 9.4 10 8 7.1 7.2 5.5 5.2 6 4.5 4 2 0 Picture Naming Alliteration Rhyming

Project PROPEL Summative Evaluation Report ______

Collaboration

Collaboration Among Grant Partners

One of the overarching goals of PROPEL targeted increasing collaboration among grant partners in an effort to build a “common language” around early literacy development in Douglas county, Kansas. Using the Collaboration Scale (see Appendix D) developed by the School

Program Evaluation and Research (SPEaR) team, grant partners reported on their perceptions of collaboration with each other prior to the implementation of the grant (Retrospective) as well as their expected or “target” level of collaboration as a result of the program. Additionally, grant partners completed the scale at two other time points during the grant period: fall 2006, fall 2007.

These data are reported in Figure 6 below. While the implementation of the grant alone appears to have increased perceptions of collaboration among grant partners, the group did not meet their target level of collaboration.

Figure 6. Mean level of collaboration reported by grant partners (fall 2006 – fall 2007)

Fall 2007 2.06

2.18 Fall 2006

TARGET 3.52

1.75 Retrospective

0.0 1.0 2.0 3.0 4.0 5.0

Project PROPEL Summative Evaluation Report ______

Teacher Collaboration & Implementation

In addition to the collaboration reported by representatives of each participating grant partner, perceptions regarding collaboration and program implementation were also collected from participating teachers during the fall 2007. As Table 6 shows, the greatest influence of

PROPEL appears to have occurred among teachers in their respective centers collaborating, problems-solving, and sharing ideas for activities and lessons focused on early literacy. Little collaboration with teachers/staff from other centers was found.

Table 6. Mean scores and standard deviations regarding teachers’ perceptions of collaboration and implementation as a result of PROPEL (fall 2007)

As a result of the PROPEL implementation, I… M (sd)

worked with the literacy coach when planning literacy activities/lessons. 3.00 (1.00) asked the literacy coach to observe classroom literacy activities. 2.91 (.94) received feedback from the literacy coach regarding classroom literacy practices. 3.55 (1.13) collaborated with teachers in my building when developing literacy activities. 3.64 (.67) problem-solve issues with the literacy coach. 3.36 (1.12) problem-solve literacy related issues with teachers in my building. 3.36 (.81) provide feedback to teachers in my building regarding literacy activities. 3.64 (.67) work with staff from other centers to develop literacy activities/lessons. 2.00 (1.34) utilize assessment data when planning literacy instruction. 3.73 (1.19) share ideas for literacy activities/lessons with teachers in my building. 4.18 (.75) share ideas for literacy activities/lessons with teachers outside of my building. 1.80 (1.14) Grand Mean 3.20 (.98) Scale: 1 = Strongly Disagree to 5 = Strongly Agree

Project PROPEL Summative Evaluation Report ______

DISCUSSION

Overall, the PROPEL project appears to have had a positive impact in increasing teachers’ understanding of developmentally appropriate early literacy practices which may in turn lead to improved student outcomes. However, there are several issues / concerns that should be addressed. Problems such as teacher turnover, changes in leadership within centers, and varying levels of collaboration among partners need to be discussed in reference to outcomes.

As with many other early childhood settings, teacher turnover for this particular project was somewhat high. During the spring 2006, twenty-four classroom teachers and seven program directors took part in training. By the fall 2006, approximately 38% of teacher left there assignments. One became a literacy coach at another center due to a reduction in classes at one center leaving eight new classroom teachers in need of training. By the fall 2007, fifty percent of teachers beginning the project remained in the classroom. In addition, changes in leadership at three of the centers took place during the project period.

Changes in teaching staff and leadership coupled with the number of students moving on to kindergarten after baseline data collection in the fall should be taken into consideration when drawing interpretations about program impact. Given the hurdles that need to be addressed during the grant cycle, the positive teacher outcomes as measured by the ELLCO (i.e. changes in the literacy environment) provide one piece of evidence that PROPEL had a positive influence on teachers who participated. Continued focus on the literacy environment as measured by the fall 2007 ELLCO help to show the sustained effects of the program and, in turn, improve students’ early literacy outcomes.

Appendix A

PROPEL Professional Development Goals

Instrumentation

Teacher Goals to be Evaluated:

Project PROPEL 1hr.)

(approx.

Survey

ELLCO Teacher CLASS MANAGEMENT & DEVELOPING RELATIONSHIPS Teacher beliefs - Do beliefs match practices? - children X - teaching and learning - parents - literacy Relationships - Relationships with ALL children (Special needs, ELL, etc.) X X - How are behaviors dealt with in the classroom? Class Rules & Routines X X - What are the established classroom routines during activities & transitions? Groupings - Per week…the number of: - Whole-class activities (eg. Read alouds) X - Small group activities (6 or fewer students) - One-on-one time with students (individual activities) Structure of Activities - Percent of daily schedule committed to (hr per activity/total hours): - teacher-led activities X - Centers or table top activities - Free play (outdoor play included) ORAL LANGUAGE Daily Whole-Class Read Alouds - Before: Prior knowledge of book; Concepts of print (title, author, illustrator, etc.) X X - During: Questioning – CAR (Conceptual/concrete; Abstract; Related to child’s life) - After: Discussion; Follow-up with AR questions (Abstract; Related to child’s life) Repeated Readings - Number of books read more than once each week X - Number of times read for each repeated reading Vocabulary - 5-10 new words per week based on readings/theme X - Words posted with picture/object Questioning (CAR) - Number of Conceptual/concrete questions - Number of Abstract/decontextualized questions X X - Number of Related to children’s lives questions - Pre-planned (sticky note…2,2,2) Small Groups - 6 or fewer students (3x a week for 5 day program) X - Number of times per week small group activities are scheduled/implemented Extension Activities - used to support vocabulary acquisition and concepts X - Number of extension activities per week Daily Talk with Children - Talk with every child individually several times a week (at least 5 minutes) X X - Talk with children on their level (kneel/bend down) - Talk times planned as part of daily routine (eg. Arrival, meals, playtime, etc.)

PRINT AWARENESS Concepts of Print - During read-alouds / book sharing times teacher models/questions: - Front to back X - Top/bottom - Title, author, illustrator, dedication page, title page, etc. - Cues for print direction (uses finger, pointing stick, etc.) Environmental Print - Number of teacher developed opportunities for environmental print in classroom (eg. Sign-in sheets, helper lists, calendar) - Number of child created environmental print artifacts in classroom (art/writing) X X - Placement of environmental print (Is it at the child’s level?) - Number of store bought vs. teacher/child created - Store bought print thematically related? Do these change monthly? Books - Books in Centers - Thematic books in centers? - How many? In what center? - Do these change monthly? X X - Class Library - Cozy area for reading? - 5-8 books per child? - Variety of books? (narrative, expository, magazines, class-made, poetry, wordless picture books, etc.) WRITING Teacher writes for children on regular basis - Number of teacher-directed writing activities (eg. Morning messages, anecdotal X X notes, language experience activities, writing poems/letters/stories) Children write on regular basis - Writing materials in all centers? - Making class books X X - Journal writing - Writing during structured and unstructured times? Writing Center - Variety of writing utensils and paper (number/type of each) – stamps, markers, pencils, X X crayons, envelopes, stickers, scissors, picture cards, etc. ALPHABET KNOWLEDGE Letter of the Week - Introduction of one letter (upper/lower case) each week - Focus is on: X - Upper and lower case - Letter sounds - Letter names Alphabet awareness through writing activities X - Teacher directed/teacher made during teacher directed writing activities. Alphabet awareness through reading activities X - Child initiated during center writing activities. PHONOLOGICAL AWARENESS Whole-class PA activities (eg. rhyming/sound games) - Number of whole-class PA activities planned each week. X - Is phonological awareness incorporated into the whole-class read aloud? Individual/Small-group PA activities (eg. rhyming/sound games) X - Number of small group PA activities planned each week.

Appendix B

Teacher Survey Item Data

Project PROPEL Teacher Survey of Classroom Literacy Baseline Results – Spring 2006 Presented to Partners June 12, 2006

Literacy Activities ( 905

n = 19 Almost Almost Never Always When I think about literacy activities in my classroom I 1 2 3 4 5 X make sure that I… (%) (%) (%) (%) (%) Find time to talk with each child in my classroom several times per week. 15.8 52.6 31.6 4.16 Incorporate “planned talk” times into my daily routine. 5.3 10.5 36.8 26.3 21.1 3.47 Re-read books on a regular basis. 5.3 15.8 36.8 42.1 4.16 Communicate with parents regarding their child’s literacy development. 5.3 31.6 31.6 31.6 3.89 Introduce a new “letter of the week” each week. 21.1 10.5 15.8 15.8 36.8 3.37 Utilize classroom management strategies that promote literacy development. 5.3 21.1 36.8 36.8 4.00 Develop class routines that promote learning. 5.3 26.3 68.4 4.63 Provide several chances for children to write. 21.1 26.3 52.6 4.32 Plan activities to extend children’s vocabulary development. 21.1 26.3 52.6 4.32 Structure the day so that children have a variety of literacy experiences. 10.5 47.4 42.1 4.32 Plan daily whole-group read alouds. 5.3 15.8 78.9 4.74 Introduce new vocabulary words each week. 5.3 15.8 31.6 26.3 21.1 3.42 Balance teacher-directed with child-directed literacy activities. 31.6 52.6 15.8 3.84 Ask concrete questions with children. 5.3 63.2 31.6 4.26 Ask abstract questions with children. 5.3 63.2 31.6 4.26 Ask questions that are related to children’s lives. 5.3 31.6 63.2 4.58 Introduce children to concepts of print. 26.3 31.6 42.1 4.16 Model writing for children. 5.3 21.1 36.8 36.8 4.05 Plan small-group literacy activities. 5.3 26.3 26.3 42.1 4.05 Encourage children to produce their own environmental print. 15.8 42.1 21.1 21.1 3.47 Fill the classroom library with a variety of different books. 5.3 10.5 36.8 47.4 4.26 Plan whole-group activities to promote phonological awareness. 44.4 33.3 22.2 3.78 Plan small-group activities to promote phonological awareness. 5.6 38.9 22.2 33.3 3.83 Focus on alphabet awareness through reading activities. 21.1 52.6 26.3 4.05 Focus on alphabet awareness through writing activities. 27.8 50.0 22.2 3.94 Introduce upper-case and lower-case letters. 10.5 5.3 10.5 57.9 15.8 3.63 Encourage child-initiated writing. 10.5 21.1 26.3 42.1 4.00

Literacy Efficacy ( 870

n = 22

Not Highly Confident Confident How confident are you that your classroom… 1 2 3 4 5 X (%) (%) (%) (%) (%) Provides every child the chance to engage in many types of literacy activities. 22.7 54.5 22.7 4.00 Helps children build relationships with their peers. 4.5 54.5 40.9 4.36 Helps children build relationships with adults. 9.1 54.5 36.4 4.27 Assists children in following classroom rules. 4.5 45.5 50.0 4.45 Contains opportunities for one-on-one literacy activities with adults. 9.1 36.4 40.9 13.6 3.59 Provides many opportunities for child-initiated writing. 9.5 38.1 33.3 19.0 3.62 Engages children in environmental print activities on a daily basis. 4.5 18.2 36.4 36.4 4.5 3.18 Promotes children’s oral language development. 4.8 71.4 23.8 4.19 Promotes children’s listening skills. 4.5 59.1 36.4 4.32 Encourages children to take the perspective of others. 27.3 59.1 13.6 3.86 Assists children in communicating their needs. 9.1 50.0 40.9 4.32 Extends children’s understanding of phonological awareness 4.8 38.1 42.9 14.3 3.67 Promotes alphabet awareness. 4.5 22.7 45.5 27.3 3.95 Provide opportunities for whole-group, teacher-led activities 13.6 22.7 63.6 4.50 Provide opportunities for small-group, teacher-led activities 22.7 27.3 50.0 4.27 Promotes literacy in all centers in the classroom. 18.2 40.9 13.6 27.3 3.50 Engages children in routines that promote classroom management. 9.1 50.0 40.9 4.32 Encourages parents to participate in literacy activities with their child. 4.5 9.1 45.5 18.2 22.7 3.45 Extends children’s thinking through the use of abstract questions. 18.2 22.7 45.5 13.6 3.55 Extends children’s thinking through the use of questions related to their lives. 18.2 40.9 40.9 4.23 Promotes literacy through use of an integrated curriculum. 9.1 18.2 50.0 22.7 3.86 Provides children with support for emerging literacy. 4.5 27.3 45.5 22.7 3.86 Encourages children’s oral language development. 13.6 45.5 40.9 4.27 Engages children in small group literacy activities. 4.5 13.6 50.0 31.8 4.09 Provides children opportunities for daily talk with adults. 13.6 36.4 50.0 4.36 Promotes literacy through play. 4.5 9.1 63.6 22.7 4.05 Extends literacy beyond the classroom. 4.5 4.5 36.4 40.9 13.6 3.55

Beliefs about Literacy ( 945

n = 20

Strongly Strongly Disagree Agree I believe that the following activities are an important part 1 2 3 4 5 X of promoting children’s literacy development… (%) (%) (%) (%) (%) Communicating with parents. 10.0 90.0 4.90 Planning classroom management strategies. 5.0 15.0 80.0 4.75 Developing class rules and routines. 5.0 20.0 75.0 4.70 Implementing whole-group read-alouds. 5.0 15.0 80.0 4.75 Reading books more than once each week. 5.0 15.0 80.0 4.75 Implementing small-group literacy activities. 5.0 5.0 90.0 4.85 Asking children concrete questions. 10.0 10.0 80.0 4.70 Asking children abstract questions. 5.0 15.0 80.0 4.75 Asking children questions that help them relate materials to their own lives. 5.0 20.0 75.0 4.70 Introducing new vocabulary words with each theme. 5.0 10.0 10.0 75.0 4.55 Using extension activities to promote concept acquisition. 25.0 75.0 4.75 Talking with children every day. 100.0 5.00 Implementing a “Letter of the Week” approach. 10.0 40.0 20.0 30.0 3.70 Providing children the opportunity to write on a regular basis. 10.0 15.0 75.0 4.65 Placing environmental print around the room. 5.0 10.0 35.0 50.0 4.30 Allowing children to create environmental print. 5.0 15.0 25.0 55.0 4.30 Introducing concepts of print (eg. front/back, directions of print, title, author, etc.) 5.0 15.0 20.0 60.0 4.35 Planning teacher-directed writing activities 15.0 20.0 65.0 4.50 Implementing phonological awareness activities 10.5 21.1 68.4 4.58 Providing children with a cozy place for book sharing 10.0 10.0 80.0 4.70 Ensuring children have the chance to experience a variety of books 5.0 95.0 4.95 Placing books in centers throughout the room 5.0 15.0 20.0 60.0 4.35 Providing opportunities for writing in centers throughout the room 15.0 15.0 70.0 4.55 Planning for a variety of different writing activities 15.0 5.0 80.0 4.65 Incorporating alphabet awareness in reading activities 25.0 75.0 4.75 Incorporating alphabet awareness in writing activities 5.0 25.0 70.0 4.65 Planning activities that are developmentally appropriate for ALL children. 5.0 95.0 4.95

Describe a Typical Week in Your Classroom:

n = 22

Please indicate the number of times each week the following take place in your classroom:

Minimum Maximum X (SD)

Phonemic awareness activities (rhyming/sound games) 1 10 3.86 (2.21) Read Alouds 2 12 6.27 (2.68) Alphabet awareness writing activities 0 8 3.32 (2.32) Alphabet awareness reading activities 1 8 3.82 (2.04) Vocabulary activities 0 10 2.73 (2.27) Vocabulary words presented 0 12 3.64 (2.94) Re-reading of favorite books 1 10 3.45 (2.11) Teacher-directed writing activities 0 10 3.14 (2.51) Child-directed writing activities 0 8 3.36 (2.08) Extension activities 1 10 3.29 (2.17)

Professional Development

Q#1: To what extent do you believe you are currently implementing classroom literacy practices?

Q#2: To what extent do you believe the professional development received through Project PROPEL will impact your classroom literacy practices?

Q#3: To what extent do you plan to utilize the literacy coach to assist you with your classroom literacy practices?

Professional Development Items--Mean Scores

6.00 5.05 5.14

5.00 4.22

4.00

Mean 3.00

2.00

1.00 Q #1 Q #2 Q #3 Questions

(Scale: Very Little = 1 to Great Deal = 6)

Utilization of Literacy Coaches

In what ways do you plan to utilize the literacy coach at your center? (eg. planning, assisting, observing, etc.)

all of the above-planning specifically co-teach, provide materials, ideas creating materials, assisting lesson plans, observe feedback/suggestions, help w/ materials help w/ literacy use, help w/ planning ideas for ESL, ideas to include parents, literature selection improvement, feedback, material help lead activities ,small group/individual, org/plan, strategize observing, plan activities, help w/ resources/materials observing, supervising kids, provide release time personal feedback, help plan activities plan activities, observe, help w/ improving literacy plan more phonemic awareness activities planning, purchasing materials planning, ideas, feedback, help to allow 1-1 interaction planning, model, materials, help w/ small groups planning, new ideas/style, SG assisting, encouragement planning, observing, feedback small group, large group help to free teacher for 1-1 literacy, org. time managing, activity planning

Project PROPEL Teacher Survey of Classroom Literacy Results – Fall 2006

Literacy Activities ( 905

n = 21 Almost Almost Never Always When I think about literacy activities in my classroom I 1 2 3 4 5 X make sure that I… (%) (%) (%) (%) (%) Find time to talk with each child in my classroom several times per week. 9.5 23.8 66.7 4.57 Incorporate “planned talk” times into my daily routine. 25.0 30.0 45.0 4.20 Re-read books on a regular basis. 9.5 14.3 76.2 4.67 Communicate with parents regarding their child’s literacy development. 42.9 38.1 19.0 3.76 Introduce a new “letter of the week” each week. 23.8 4.8 9.5 14.3 47.6 3.57 Utilize classroom management strategies that promote literacy development. 4.8 4.8 38.1 52.4 4.38 Develop class routines that promote learning. 4.8 23.8 71.4 4.67 Provide several chances for children to write. 15.0 30.0 55.0 4.40 Plan activities to extend children’s vocabulary development. 19.0 28.6 52.4 4.33 Structure the day so that children have a variety of literacy experiences. 9.5 19.0 71.4 4.62 Plan daily whole-group read alouds. 4.8 14.3 81.0 4.76 Introduce new vocabulary words each week. 14.3 28.6 14.3 42.9 3.86 Balance teacher-directed with child-directed literacy activities. 4.8 47.6 47.6 4.43 Ask concrete questions with children. 9.5 42.9 47.6 4.38 Ask abstract questions with children. 19.0 52.4 28.6 4.10 Ask questions that are related to children’s lives. 33.3 66.7 4.67 Introduce children to concepts of print. 38.1 61.9 4.62 Model writing for children. 4.8 47.6 47.6 4.43 Plan small-group literacy activities. 23.8 19.0 57.1 4.33 Encourage children to produce their own environmental print. 9.5 19.0 38.1 33.3 3.95 Fill the classroom library with a variety of different books. 4.8 23.8 71.4 4.67 Plan whole-group activities to promote phonological awareness. 23.8 38.1 38.1 4.14 Plan small-group activities to promote phonological awareness. 4.8 28.6 42.9 23.8 3.86 Focus on alphabet awareness through reading activities. 23.8 38.1 38.1 4.14 Focus on alphabet awareness through writing activities. 27.6 42.9 28.6 4.00 Introduce upper-case and lower-case letters. 5.0 20.0 20.0 55.0 4.25 Encourage child-initiated writing. 9.5 28.6 61.9 4.52

Literacy Efficacy ( 870

n = 21

Not Highly Confident Confident How confident are you that your classroom… 1 2 3 4 5 X (%) (%) (%) (%) (%) Provides every child the chance to engage in many types of literacy activities. 4.8 42.9 52.4 4.43 Helps children build relationships with their peers. 9.5 90.5 4.90 Helps children build relationships with adults. 9.5 90.5 4.90 Assists children in following classroom rules. 19.0 81.0 4.81 Contains opportunities for one-on-one literacy activities with adults. 23.8 38.1 38.1 4.14 Provides many opportunities for child-initiated writing. 19.0 33.3 47.6 4.29 Engages children in environmental print activities on a daily basis. 4.8 4.8 28.6 38.1 23.8 3.71 Promotes children’s oral language development. 38.1 61.9 4.62 Promotes children’s listening skills. 33.3 66.7 4.67 Encourages children to take the perspective of others. 14.3 47.6 38.1 4.24 Assists children in communicating their needs. 5.0 20.0 75.0 4.70 Extends children’s understanding of phonological awareness 9.5 52.4 38.1 4.29 Promotes alphabet awareness. 14.3 33.3 52.4 4.38 Provide opportunities for whole-group, teacher-led activities 4.8 33.3 61.9 4.57 Provide opportunities for small-group, teacher-led activities 19.0 28.6 52.4 4.33 Promotes literacy in all centers in the classroom. 4.8 33.3 28.6 33.3 3.90 Engages children in routines that promote classroom management. 9.5 38.1 52.4 4.43 Encourages parents to participate in literacy activities with their child. 4.8 4.8 19.0 33.3 38.1 4.00 Extends children’s thinking through the use of abstract questions. 4.8 4.8 14.3 47.6 28.6 3.90 Extends children’s thinking through the use of questions related to their lives. 4.8 33.3 61.9 4.57 Promotes literacy through use of an integrated curriculum. 4.8 14.3 23.8 57.1 4.33 Provides children with support for emerging literacy. 14.3 33.3 52.4 4.38 Encourages children’s oral language development. 19.0 81.0 4.81 Engages children in small group literacy activities. 14.3 28.6 57.1 4.43 Provides children opportunities for daily talk with adults. 4.8 33.3 61.9 4.57 Promotes literacy through play. 33.3 66.7 4.67 Extends literacy beyond the classroom. 4.8 19.0 47.6 28.6 3.95

Describe a Typical Week in Your Classroom:

n = 18-21

Please indicate the number of times each week the following take place in your classroom:

Minimum Maximum X (SD)

Phonemic awareness activities (rhyming/sound games) 0 20 5.05 (4.38) Read Alouds 3 15 7.90 (3.77) Alphabet awareness writing activities 1 10 4.63 (2.41) Alphabet awareness reading activities 0 15 4.42 (3.47) Vocabulary activities 0 15 4.05 (3.68) Vocabulary words presented 0 6 2.60 (2.04) Re-reading of favorite books 2 15 5.05 (3.20) Teacher-directed writing activities 1 6 3.48 (1.54) Child-directed writing activities 3 12 5.10 (2.25) Extension activities 1 12 4.44 (2.62)

I. Professional Development

Q#1: To what extent do you believe you are currently implementing classroom literacy practices?

Q#2: To what extent do you believe the professional development received through Project PROPEL will impact your classroom literacy practices?

Q#3: To what extent do you plan to utilize the literacy coach to assist you with your classroom literacy practices?

Professional Development Items--Mean Scores

6.00 4.71 4.52 5.00 4.24

4.00

Mean 3.00

2.00

1.00 Q #1 Q #2 Q #3 Questions

(Scale: Very Little = 1 to Great Deal = 6)

Utilization of Literacy Coaches

In what ways do you plan to utilize the literacy coach at your center? (eg. planning, assisting, observing, etc.)

planning, observing, modeling, team teaching, making and implementing new activities all of the above and materials purchasing weekly planning time for support and resources brainstorming with her, planning, co-teaching, assisting, and prepping planning, assisting, observing literacy coach helps with writing activities, alphabet, name recognition, etc. assisting, overall on a scale 1-6;3 assisting planning and observing, some assisting assisting, materials, planning observing, utilizing literacy props and materials she brings in assists in teaching Spanish, materials planning, assisting, implementing new ideas observing, planning

Project PROPEL Teacher Survey of Classroom Literacy Results –Spring 2007

I. Literacy Activities ( 905

n = 15 Almost Almost Never Always When I think about literacy activities in my classroom I 1 2 3 4 5 X make sure that I… (%) (%) (%) (%) (%) Find time to talk with each child in my classroom several times per week. 6.7 33.3 60.0 4.53 Incorporate “planned talk” times into my daily routine. 6.7 20.0 40.0 33.3 4.00 Re-read books on a regular basis. 6.7 6.7 26.7 60.0 4.40 Communicate with parents regarding their child’s literacy development. 6.7 20.0 26.7 46.7 4.13 Introduce a new “letter of the week” each week. 6.7 20.0 20.0 53.3 4.13 Utilize classroom management strategies that promote literacy development. 40.0 60.0 4.60 Develop class routines that promote learning. 20.0 80.0 4.80 Provide several chances for children to write. 33.3 66.7 4.67 Plan activities to extend children’s vocabulary development. 6.7 46.7 46.7 4.40 Structure the day so that children have a variety of literacy experiences. 6.7 33.3 60.0 4.53 Plan daily whole-group read alouds. 6.7 93.3 4.93 Introduce new vocabulary words each week. 20.0 46.7 33.3 4.13 Balance teacher-directed with child-directed literacy activities. 13.3 33.3 53.3 4.40 Ask concrete questions with children. 33.3 66.7 4.67 Ask abstract questions with children. 6.7 66.7 26.7 4.20 Ask questions that are related to children’s lives. 40.0 60.0 4.60 Introduce children to concepts of print. 13.3 40.0 46.7 4.33 Model writing for children. 6.7 40.0 53.3 4.47 Plan small-group literacy activities. 13.3 33.3 53.3 4.40 Encourage children to produce their own environmental print. 6.7 6.7 60.0 26.7 4.07 Fill the classroom library with a variety of different books. 6.7 20.0 73.3 4.67 Plan whole-group activities to promote phonological awareness. 6.7 40.0 53.3 4.47 Plan small-group activities to promote phonological awareness. 20.0 33.3 46.7 4.27 Focus on alphabet awareness through reading activities. 6.7 53.3 40.0 4.33 Focus on alphabet awareness through writing activities. 6.7 46.7 46.7 4.40 Introduce upper-case and lower-case letters. 26.7 46.7 26.7 4.00 Encourage child-initiated writing. 40.0 60.0 4.60

Literacy Efficacy ( 870

n = 15

Not Highly Confident Confident How confident are you that your classroom… 1 2 3 4 5 X (%) (%) (%) (%) (%) Provides every child the chance to engage in many types of literacy activities. 53.3 46.7 4.47 Helps children build relationships with their peers. 13.3 86.7 4.87 Helps children build relationships with adults. 20.0 80.0 4.80 Assists children in following classroom rules. 20.0 80.0 4.80 Contains opportunities for one-on-one literacy activities with adults. 13.3 40.0 46.7 4.33 Provides many opportunities for child-initiated writing. 40.0 60.0 4.60 Engages children in environmental print activities on a daily basis. 40.0 33.3 26.7 3.87 Promotes children’s oral language development. 13.3 86.7 4.87 Promotes children’s listening skills. 13.3 86.7 4.87 Encourages children to take the perspective of others. 13.3 46.7 40.0 4.27 Assists children in communicating their needs. 13.3 86.7 4.87 Extends children’s understanding of phonological awareness 6.7 26.7 66.7 4.60 Promotes alphabet awareness. 26.7 73.3 4.73 Provide opportunities for whole-group, teacher-led activities 20.0 80.0 4.80 Provide opportunities for small-group, teacher-led activities 20.0 80.0 4.80 Promotes literacy in all centers in the classroom. 20.0 26.7 53.3 4.33 Engages children in routines that promote classroom management. 26.7 73.3 4.73 Encourages parents to participate in literacy activities with their child. 26.7 40.0 33.3 4.07 Extends children’s thinking through the use of abstract questions. 13.3 66.7 20.0 4.07 Extends children’s thinking through the use of questions related to their lives. 40.0 60.0 4.60 Promotes literacy through use of an integrated curriculum. 46.7 53.3 4.53 Provides children with support for emerging literacy. 6.7 26.7 66.7 4.60 Encourages children’s oral language development. 20.0 80.0 4.80 Engages children in small group literacy activities. 6.7 33.3 60.0 4.53 Provides children opportunities for daily talk with adults. 6.7 26.7 66.7 4.60 Promotes literacy through play. 6.7 20.0 73.3 4.67 Extends literacy beyond the classroom. 60.0 40.0 4.40

Describe a Typical Week in Your Classroom:

n = 12-14

Please indicate the number of times each week the following take place in your classroom:

Minimum Maximum X (SD)

Phonemic awareness activities (rhyming/sound games) 2 20 5.43 (4.69) Read Alouds 3 15 7.86 (3.90) Alphabet awareness writing activities 2 10 4.29 (2.23) Alphabet awareness reading activities 2 10 4.36 (2.02) Vocabulary activities 1 10 4.42 (2.80) Vocabulary words presented 1 5 3.42 (1.51) Re-reading of favorite books 1 10 4.23 (2.24) Teacher-directed writing activities 1 20 4.43 (4.70) Child-directed writing activities 2 10 4.46 (2.22) Extension activities 2 8 4.00 (1.65)

I. Professional Development

Q#1: To what extent do you believe you are currently implementing classroom literacy practices?

Q#2: To what extent do you believe the professional development received through Project PROPEL will impact your classroom literacy practices?

Q#3: To what extent do you plan to utilize the literacy coach to assist you with your classroom literacy practices?

Professional Development Items--Mean Scores

6.00 5.13

5.00 4.2 3.73 4.00

Mean 3.00

2.00

1.00 Q #1 Q #2 Q #3 Questions

(Scale: Very Little = 1 to Great Deal = 6)

Utilization of Literacy Coaches

In what ways do you plan to utilize the literacy coach at your center? (eg. planning, assisting, observing, etc.)

planning, assisting, observing, team teaching, helping make materials Not much at the moment. We had a great deal more input from her first semester than as of late. planning mostly as a second pair of eyes to see how things can be improved, small group teaching and planning provides an occasional activity to the class and vocabulary words--will begin small group pull outs assisting and planning Planning different activities, helping come up with new fun way, observing to see how to better teach planning and ideas observing, assisting in finding books for storytime, assisting in planning lessons assisting, material prep. help with planning activities and getting materials ready Our coach spends very little time with us, does not interact with the kids at all, and.. Planning and observing. She gives great ideas and lets me know what I need to implement. Observing. Very little feedback or comments from the literacy coach. Very little. She has been forced to be in another classroom as of right now.

Appendix C

ELLCO Item Data

Project PROPEL Early Language & Literacy Classroom Observation (ELLCO) Baseline Results – Spring 2006

Literacy Environment Checklist (n = 24)

Book Area Yes Area set aside for book 91.7% reading Book area orderly and 87.5% inviting Book area has soft materials 79.2%

Book Selection Yes Books range in difficult level 100.0% 3+ books related to theme 54.2%

16-25 26+ M = 49.64 25.0% 75.0% # Books available to children (SD = 34.45)

0 1-2 3-5 6+ M = 10.84 12.5% 8.3% 20.8% 58.4% # Factual books (SD = 10.32)

Book Use Yes

Place for children to listen to books/stories 45.8% 0 1-3 4+ # Books in science area 75.0% 4.2% 20.8 # Books in dramatic play 75.0% 4.2% 20.8% # Books in block area 75.0% 12.5% 12.5% # Books available in other areas 37.5% 20.8% 41.7%

Writing Materials Yes Distinct area set up for writing 54.2%

Alphabet visible 62.5%

Word cards 54.2% Template/tools for forming letters 87.5% 0 1-2 3+ Variety of paper available 4.2% 20.8* 75.0% Variety writing tools available 4.2% 95.8%

Classroom Observation (n = 24)

Scale Used:

5 4 3 2 1 Exemplary Basic Deficient (strong evidence) (some evidence) (minimal evidence)

Ratings are applied to fourteen specific literacy related targets utilizing a rubric based on the amount of evidence observed in the classroom to support the presence of each literacy target. Each item/target is scored on a continuum of strong evidence to minimal evidence based on the rater’s observations. It is important to note that the rubric at least three descriptors/pieces of evidence supporting rating of Exemplary, Basic and Deficient categories. The descriptors allow raters to identify both strengths and weaknesses associated with the specific literacy target under observation.

General Classroom Environment Items:

General Classroom Environment X sd Organization of the Classroom 3.42 (0.72) Contents of the Classroom 3.00 (0.66) Presence/Use of Technology 2.79 (0.78) Opportunities for Child Choice and 3.33 (0.64) Initiative Classroom Management Strategies 3.42 (0.78) Classroom Climate 3.96 (0.91)

Language, Literacy & Curriculum Items:

Language Literacy & Curriculum X sd Oral Language Facilitation 3.42 (0.88) Presence of Books 3.58 (1.02) Approaches to Book Reading 3.71 (0.62) Approaches to Children's Writing 3.09 (0.90) Approaches to Curriculum Integration 3.13 (1.12) Recognizing Diversity in the Classroom 2.79 (1.14) Facilitating Home Support for Literacy 3.21 (1.06) Approaches to Assessment 3.00 (0.88)

Literacy Activities Rating Scale (n = 24)

Book Reading Yes Time for children to read 95.8% w/friend Adult reading w/children 37.5%

0 1 More # Full-group reading 25.0% 58.3% 16.7% sessions # Books read during full- 29.2% 50.0% 20.8% group

< 5 5-10 10+ # Minutes in full-group 29.2% 41.7% 29.2% reading

Writing Yes Children writing in their 33.3% play Children attempting 37.5% letters/words Adult model writing 37.5%

0 1-2 More # Time adult(s) helped child 75.0% 8.3% 16.7% write

Project PROPEL Early Language & Literacy Classroom Observation (ELLCO) Results – Fall 2006

Literacy Environment Checklist (n = 21)

Book Area Yes Area set aside for book reading 95.2% Book area orderly and inviting 90.5% Book area has soft materials 85.7%

Book Selection Yes Books range in difficulty 100.0% level 3+ books related to theme 76.2%

16-25 26+ # Books available to M = 70.67 9.5% 90.5% children (SD = 51.25) 0 1-2 3-5 6+ # Factual books 0% 9.5% 9.5% 81%

Book Use Yes

Place for children to listen to books/stories 45.0% 0 1-3 4+ # Books in science area 57.1% 9.5% 33.3% Books in dramatic play 66.7% 14.3% 19.0% # Books in block area 57.1% 14.3% 28.6% # Books available in other areas 14.3% 38.1% 47.6%

Writing Materials Yes

Distinct area set up for writing 85.7%

Alphabet visible 90.0%

Word cards 85.7%

Template/tools for forming letters 85.7% 0 1-2 3+ Variety of paper available 0% 23.8% 76.2% Variety writing tools available 0% 19.0% 81.0%

Classroom Observation (n = 21)

Scale Used:

5 4 3 2 1 Exemplary Basic Deficient (strong evidence) (some evidence) (minimal evidence)

General Classroom Environment Items:

General Classroom Environment X sd Organization of the Classroom 4.14 (0.73) Contents of the Classroom 3.33 (0.86) Presence/Use of Technology 3.05 (0.83) Opportunities for Child Choice and 3.48 (0.60) Initiative Classroom Management Strategies 3.86 (0.57) Classroom Climate 4.10 (0.70)

Language, Literacy & Curriculum Items:

Language Literacy & Curriculum X sd Oral Language Facilitation 3.57 (0.60) Presence of Books 3.86 (0.96) Approaches to Book Reading 4.10 (0.70) Approaches to Children's Writing 3.52 (0.68) Approaches to Curriculum Integration 3.67 (0.66) Recognizing Diversity in the Classroom 3.40 (0.75) Facilitating Home Support for Literacy 3.25 (0.72) Approaches to Assessment 3.62 (1.32)

Literacy Activities Rating Scale (n = 21)

Book Reading Yes Time for children to read 100% w/friend Adult reading w/children 47.6%

0 1 More # Full-group reading 0% 95.2% 4.8% sessions # Books read during full- 0% 95.2% 4.8% group

< 5 5-10 10+ # Minutes in full-group 0% 57.1% 42.9% reading

Writing Yes Children writing in their 47.6% play Children attempting 47.6% letters/words Adult model writing 61.9%

0 1-2 More # Time adult(s) helped child 47.6% 38.1% 14.3% write

Project PROPEL Early Language & Literacy Classroom Observation (ELLCO) Results – Spring 2007

Literacy Environment Checklist (n = 23)

Book Area Yes Area set aside for book reading 91.3% Book area orderly and inviting 91.3% Book area has soft materials 87.0%

Book Selection Yes Books range in difficulty level 100.0% 3+ books related to theme 65.2%

<15 16-25 26+ M = 82.17 8.7% 4.3% 87.0% # Books available to children (SD = 51.69)

0 1-2 3-5 6+ # Factual books 0% 8.7% 8.7% 82.6%

Book Use Yes

Place for children to listen to books/stories 43.5.0% 0 1-3 4+ # Books in science area 52.2% 0% 47.8% Books in dramatic play 56.5% 13.0% 30.4% # Books in block area 56.5% 0% 43.5% # Books available in other areas 26.1% 30.4% 43.5%

Writing Materials Yes

Distinct area set up for writing 82.6%

Alphabet visible 95.7%

Word cards 82.6%

Template/tools for forming letters 91.3% 0 1-2 3+ Variety of paper available 4.3% 30.4% 65.2% Variety writing tools available 0% 8.7% 91.3%

Classroom Observation (n = 23)

Scale Used:

5 4 3 2 1 Exemplary Basic Deficient (strong evidence) (some evidence) (minimal evidence)

General Classroom Environment Items:

General Classroom Environment X sd Organization of the Classroom 4.13 (0.76) Contents of the Classroom 3.59 (0.73) Presence/Use of Technology 3.13 (0.81) Opportunities for Child Choice and 3.60 (0.72) Initiative Classroom Management Strategies 3.96 (0.64) Classroom Climate 4.17 (0.83)

Language, Literacy & Curriculum Items:

Language Literacy & Curriculum X sd Oral Language Facilitation 3.74 (0.45) Presence of Books 3.87 (0.87) Approaches to Book Reading 4.35 (0.49) Approaches to Children's Writing 3.61 (0.78) Approaches to Curriculum Integration 3.83 (0.72) Recognizing Diversity in the Classroom 3.48 (0.59) Facilitating Home Support for Literacy 3.30 (0.63) Approaches to Assessment 3.83 (0.89)

Literacy Activities Rating Scale (n = 23)

Book Reading Yes Time for children to read 100% w/friend Adult reading w/children 34.8%

0 1 More # Full-group reading 0% 95.7% 4.3% sessions # Books read during full- 0% 78.3% 21.7% group

< 5 5-10 10+ # Minutes in full-group 0% 47.8% 52.2% reading

Writing Yes Children writing in their 69.6% play Children attempting 56.5% letters/words Adult model writing 65.2%

0 1-2 More # Time adult(s) helped child 69.6% 21.7% 8.7% write

Appendix D

Collaboration Scale(s)

Collaboration Before the Grant Began

Name ______Date ______

Organization______

Levels of Collaboration Scale

This form is designed for those who work in one of the organizations or programs that are partners in the PROPEL initiative. Please review these descriptions of different levels of collaboration. On the response section at the bottom of the page, please circle the name of the organization or group with which you are associated. Using the scale provided, please indicate the extent to which you interacted with each other partner before the PROPEL Grant began (October 2005). (Skip your own row.)

Five Levels of Collaboration and Their Characteristics Networking Cooperation Coordination Coalition Collaboration 1 2 3 4 5 Relationship -Aware of -Provide information -Share information -Share ideas -Members belong to one Characteristics organization to each other and resources -Share resources system -Loosely defined - Somewhat defined -Defined roles -Frequent and prioritized -Frequent communication roles roles -Frequent communication is characterized by mutual -Little -Formal communication -All members have a vote trust communication communication -Some shared in decision making -Consensus is reached on -All decisions are -All decisions are decision making all decisions made independently made independently

No Networking Cooperation Coordination Coalition Collaboration Partners Interaction at All 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5

TARGET Collaboration Name ______Date ______

Organization______

Levels of Collaboration Scale This form is designed for those who work in one of the organizations or programs that are partners in the PROPEL initiative. Please review these descriptions of different levels of collaboration. On the response section at the bottom of the page, please circle the name of the organization or group with which you are associated. Using the scale provided, please indicate your TARGET level of collaboration with each other partner as a result of PROPEL. (Skip your own row.)

Five Levels of Collaboration and Their Characteristics Networking Cooperation Coordination Coalition Collaboration 1 2 3 4 5 Relationship -Aware of -Provide -Share information -Share ideas -Members belong to Characteristics organization information to each and resources -Share resources one system -Loosely defined other -Defined roles -Frequent and -Frequent roles - Somewhat defined -Frequent prioritized communication is -Little roles communication communication characterized by mutual communication -Formal -Some shared -All members have a trust -All decisions are communication decision making vote in decision -Consensus is reached made -All decisions are making on all decisions independently made independently No Networking Cooperation Coordination Coalition Collaboration Partners Interaction at All 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5

Current Collaboration Name ______Date ______

Organization______

Levels of Collaboration Scale This form is designed for those who work in one of the organizations or programs that are partners in the PROPEL initiative. Please review these descriptions of different levels of collaboration. On the response section at the bottom of the page, please circle the name of the organization or group with which you are associated. Using the scale provided, please indicate the extent to which you currently interact with each other partner. (Skip your own row.)