ELOA Grant Evaluation Report

ELOA Grant Evaluation Report 1

ELOA Grant Evaluation Report Executive Summary

Kenneth Maurice Tyler, Ph.D. External Evaluator, ELOA Grant

The Early Learning Opportunities grant was developed by Community Action Council for Lexington-Fayette, Bourbon, Harrison, and Nicholas counties on behalf of a community collaborative led by Success by Six, a United Way initiative. The primary collaborators on the project were Community Action Council for Lexington-Fayette, Bourbon, Harrison, and Nicholas counties, United Way of the Bluegrass/Success by Six, Child Care Council of Kentucky along with Fayette County Public Schools and other early childhood entities. Community Action Council was awarded $921,494.00 to carry out initiatives stated in the grant proposal. The ELOA project was located in and served Lexington- Fayette County, Kentucky.

This report highlights the findings of the ELOA grant initiatives and activities designed to meet five objectives pertaining to the enhancement of literacy outcomes for early childhood students and the overall literacy context of 31 early child care centers in Fayette County, Lexington, Kentucky. To meet these objectives, which are stated in the introduction of the report, several institutions and entities, including the Community Action Council for Lexington-Fayette, Bourbon, Harrison, and Nicholas Counties, United Way of the Bluegrass/Success by Six and the Child Care Council of Kentucky met and collaborated over the twenty-three months of the grant period.

Primary ELOA staff included Monica Hall, Project Coordinator, Stacy Keys, Debbie Link, Lori Bone, Ginger Perkins, Olivia Thompson and Mae Smalley. Each provided or assisted in the provision of technical and financial assistance, curriculum module training, parent training and community resource awareness and training to parents and early childcare providers across the thirty sites. Their purpose and hence, the purpose of the ELOA initiatives were to investigate whether the provision of several literature and research-based strategies, activities, and resources would improve literacy outcomes for both students and the early child care centers they attend. These outcomes included statistical examination of participating students’ scores on the DIAL-3 pre and post-intervention assessments and pre and post-intervention assessment of multiple early childhood literacy observation forms. Also, descriptive information (e.g., percentages) was used to assess any increase in the number of sites meeting minimal standardized early childcare center requirements, including overall childcare atmosphere and physical infrastructure. ELOA Grant Evaluation Report 2

The collaborative efforts of the above councils resulted in the recruitment of thirty-two early childcare centers for participation in grant initiatives. Each participating early childhood center required licensure from the state of Kentucky. Also, each center was required to have children eligible for preschool (e.g., preschool aged children). With these requirements, one center was not allowed to participate because it no longer operated as a licensed child care center and another was eliminated from the sample of participating centers as it did not have preschool eligible children. The thirty participating early childcare centers had each received funding and ELOA intervention, both of which were used to enhance the literacy environment of the centers and ultimately, increase literacy outcomes for participating students.

Over sixty early childcare providers had been trained on several ELOA interventions designed to enhance literacy outcomes and environments of students at the different childcare sites. Many of the intervention components designed to assess and enhance literacy outcomes are also used by Fayette County Public Schools in their early childhood curriculum. These interventions included the “creative curriculum”, LEAP programming, and several early language and literacy observation tools. These same materials were used in the ELOA interventions to provide a smooth transition for participating students into kindergarten. Early childcare providers at the participating sites also received training on enhancing the literacy environment. In particular, specific room arrangements, structures and modules along with appropriate developmental practices were part of the training early childcare providers received.

In addition, toolkits containing books, literacy activities, hands-on materials were provided to the early childcare centers for parental usage. Specifically, parents were encouraged by early childcare center staff and providers to “check-out” various materials designed to reinforce the literacy activities their children were exposed to at the center and thus, promote literacy development at home. Also, a monthly resource calendar was provided to each early childcare center and was complete with specific dates highlighting specific literacy-based activities, community resources to facilitate literacy development at home and opportunities for parents to become more directly involved with some literacy-promoting activities at their children’s respective sites. While participation in pre and post-intervention testing was limited to a smaller number of pre-school age children, well over 1,000 children were present across the 30 early child care sites and thus, were exposed to ELOA intervention initiatives, particularly early childcare providers receiving additional training on literacy promotion.

To evaluate the success of ELOA interventions in increasing the number of early childcare sites achieving minimal standard requirements, all ELOA Grant Evaluation Report 3 early childhood centers, were asked to participate in the STAR for KIDS NOW childcare rating system. STAR for KIDS NOW is a voluntary quality rating system for licensed Type I and Type II child care centers and also certified family child care homes in the state of Kentucky. This system uses a scale of one through four STAR to identify levels of quality. All STAR levels surpass the minimum licensing requirements that all programs must meet. However, many early childcare centers operate without STAR ratings. Various areas assessed in each early childcare center included curriculum, parent involvement, training/education of staff, regulatory compliance, and personnel practices* Prior to the grant, thirteen of the 31(42%) centers had achieved and participated in the STAR Program. Within the duration of the grant, twenty-seven of the 31 (87%) achieved a STAR rating.

Three major research questions were developed to guide the evaluation of ELOA activities created to enhance the literacy environments and outcomes of preschool age children. To begin, several instruments and measures were created or retrieved from the literature on literacy development in order to assess the utility of ELOA program modules and activities on literacy outcomes at both the student and classroom level. Also, ELOA staff provided center directors and parents with monthly training in an effort to improve their current literacy-fostering practices.

Data collection took place in the Fall of 2006 and again in the Spring of 2007. In particular, data on preschool students’ pre and post- intervention performance using standard literacy assessment tools were collected and analyzed for statistically significant differences. Also, data on the classroom climate and the presence of activities and structures that foster literacy development were assessed at two points in time (pre and post-intervention) and were examined for statistical significance. Thirty sites participated in the initiative, however, only 130 student pairs of pre and post-intervention standardized literacy outcomes scores were available for analysis. Similarly, only 66 parent pairs and 21 site pairs of pre and post-intervention were retrieved and subsequently, statistically analyzed.

Over the course of one year, ELOA staff researched and implemented several parent and director-centered strategies to promote literacy development across 30 early child care centers in Fayette County, Kentucky. The findings of the effects of ELOA programming on student and site-based literacy outcomes were positive. Paired t-test statistical examinations revealed that on average, DIAL-3 motor, concept, language and total scores for literacy improved significantly since the implementation of ELOA programming. Also, the scores on the ELLCO literacy environment observation tool were significantly improved at posttest. Only one set of pre/posttest averaged scores of the three additional classroom observational tools proved statistically significant ELOA Grant Evaluation Report 4

scores (FDCRS). Also, since the implementation of ELOA programming and the provision and monitoring of various materials by ELOA staff, parents across the different sites reported statistically significant increases in the amount of time they spent helping their children write letters and also meeting with their child’s teacher.

Statistical findings, along with descriptive information are provided in the Results section of this report. Discussion of the findings and future research directions are also presented in the section following presentation of the study results.

Introduction

The purpose of this report is to present the results of an initiative designed to improve urban child care centers located in Fayette County, Kentucky. Specifically, federal Early Learning Opportunities Act funds were provided to the Community Action Council-Lexington-Fayette and other counties to support ELOA-based literacy interventions implemented in 30 early child care centers in Lexington- Fayette County, Kentucky. Under this initiative were five major objectives to be reached throughout the 23-month duration of federal funding. While this report presents statistical findings on three major objectives, other ELOA grant objectives and their outcomes are listed below.

Objective 1: To improve communication, collaboration and unified planning and vision between governance groups providing oversight to a broad spectrum of early childhood development in the community.

Outcome: The Child Care Council of Kentucky, United Way/Success by Six, and the Community Action Council representing six different counties in Kentucky had collaboratively partnered throughout the duration of the grant. Members of each of these councils met monthly to make available to licensed early childcare centers and home-based childcare providers in Fayette County information and resources designed to promote literacy development and overall positive childcare interactions between parent and childcare provider, children and childcare provider and parent and child.

In particular, the Child Care Council of Kentucky has added a one-stop shop website containing various resources and links of other community agencies housing information about early childhood development, literacy, parenting and related topics. While access to the site, along with its construction and content were the results of the collaboration among the different councils, website information was also available to parents, early childhood professionals, and others interested in the latest information, resources about early childhood issues, school readiness, and quality childcare. In addition, part of the ELOA intervention including significant outreach to parents concerning literacy development, community resources, and parent involvement strategies. Each of ELOA Grant Evaluation Report 5 these intervention strategies were sanctioned by the collaborative efforts of the aforementioned councils.

The collaboration also resulted in the recruitment of thirty-one early childcare centers with directors that volunteered to participate in ELOA intervention over the 23 months. Child Care Council of Kentucky, in agreement with United Way/Success by Six and Community Action Council allowed early childcare center directors to become members of the Child Care Council throughout the grant duration. This strategy was designed to enable childcare center directors with the ability to co-construct new strategies for effective childcare and literacy development, while simultaneously empowering them by allowing their voices on early childcare provision to be communicated to the various councils. Objective 2: To achieve an agreed upon definition and supports for pre-school- to-school transition between the Fayette County Public Schools and early childhood development providers and improve the systems to measure school readiness and child education success.

Outcome: The Fayette County Early Childhood Interagency Community Agreement for all children prenatal to six was completed. This was developed by the Fayette County Early Childhood Interagency Transition Team and the Kentucky Early Childhood Transition Project. Some participants in the ELOA grant were members of and thus, worked on the goals. The transition plan resulted in the following goals below for all families prenatal to six.

Goal 1: Increase awareness of social emotional development and transition skills Goal 2: Explore providing a learning opportunity for families and professionals to include conflict resolution strategies, effective home visiting techniques, and time management. Goal 3: Support parent child interaction for families (English and Spanish) by providing an alternative to reading for families using Parent Guide topic/information. Goal 4: Parents report at entry of preschool and kindergarten using activities from Parent Guides Goal 5: Agencies will identify critical transition skill/information to be transferred to receiving agencies Goal 6: Agencies will provide support to families using current resources.

The remaining objectives for the ELOA grant initiative were assessed using statistical analyses to determine significant differences in outcomes prior to and after ELOA intervention strategies. Those objectives are as follows:

 Objective 3: To facilitate communication with parents through outreach and education that help parents engage and build interest in getting more involved with their child’s education and assists them to identify and access further parent involvement opportunities. ELOA Grant Evaluation Report 6

 Objective 4: To improve the quality of early learning opportunities for children attending independent child care facilities.

 Objective 5: To improve the quality of and promote pre-literacy activities in early childcare centers and achieve greater school readiness in the area of literacy.

The early child care centers presented in this report are located in areas where feeder elementary schools are classified as Title I schools (e.g., a school with at least 75% of students eligible for free or reduced lunch). Each Title I school receives federal dollars to augment remedial education and learning enhancement activities. Students in these schools, particularly in Fayette County, average 15 points below other county elementary schools in reading and other literacy activities (Fayette County Public Schools, 2003). In addition, young children attending these early child care centers and eventually transition into one of the 13 Title I schools have, for decades, experienced significant disciplinary and behavioral problems at school, have performed significantly worse on Kentucky’s standardized school performance indices (i.e., CATS testing), and have been reported to leave school before graduation significantly more and earlier than their more advantaged counterparts.

To address these gaps in achievement and school completion rates, several have argued for the need to improve the quality of child care centers and early childhood development systems. According to the literature (Cunningham & Stanovich, 1997), it is anticipated that greater exposure to enhanced literacy instruction, an enhanced environment and enhanced literacy activities—both at home and at the center—will improve low-income preschool students’ preparedness for kindergarten and thus, facilitate their academic progress and achievement throughout elementary and secondary education. Along with this, in 2006, several small and independent child care center directors were interviewed to determine what the specific needs of their center were and how meeting these needs could help prepare preschool children for literacy activities in kindergarten. Several issues were reported and included 1) increasing access to quality improvement expertise and financial resources, particularly for literacy and 2) increasing parent involvement in center and literacy activities. Literacy has been defined as "an individual's ability to read, write, speak in English, compute and solve problems at levels of proficiency necessary to function on the job, in the family of the individual, and in society." (www.kentuckyliteracy.org).

With these issues in mind, members of the ELOA grant initiative team planned the implementation and assessment of several programs and activities through an interagency agreement with Community Action Council, Lexington- Fayette, Bourbon, Harrison and Nicholas Counties, United Way of the Bluegrass/Success by Six, and Child Care Council of Kentucky. Three major research questions were developed to examine the impact of these programs and activities on preschool student and child care center literacy outcomes. ELOA Grant Evaluation Report 7

Research Questions

1. How effective is the ELOA intervention in augmenting literacy outcomes for preschool age children? This question addresses Objective 5. 2. How effective is the ELOA intervention in enhancing the literacy environments that preschool age children are exposed to, specifically at home and at preschool service sites? This question addresses Objectives 3 & 4. 3. What parent, teacher and preschool site-based characteristics are associated with literacy post-ELOA intervention outcomes? This question explores relationships between post-intervention literacy environments, teacher demographics (i.e., age, years of experience) parent literacy practices and demographics (i.e., age, number of children at home) and student literacy outcomes (i.e., DIAL-3 posttest scores). Thus, this question addresses each objective listed for the ELOA grant.

Methodology

Sample

Early child care students

Thirty small (N<50) and independent (not subsidized by local, state or federal funds) early childhood education/daycare centers were identified for participation in the data collection process. Two hundred early child care students were included in the data analysis. Forty one percent were White, while 41% was African American. Eleven percent of the students was Latino or Hispanic and close to 8% was reported by site personnel as other. Fifty four percent of the sample was girls while 46% were boys (N=108 and 92, respectively). The majority of the sample was four years of age (N=119; 60% of the sample) and average age was 3 years and 8 months. While the literacy scores of 200 students were assessed at pre-intervention, only 130 students’ post- intervention literacy scores were collected and matched for comparison purposes with pre-intervention scores.

Parents of early child care students One hundred and twenty-three parents of early child care students participated in the initiative. Average age of the parents was 32 years. All participating parents were mothers of early child care students. Fifty-four percent of the participating parents were White, while a third (33.3) were African American. Roughly 7% of the parents were Latino and six did not indicate ethnicity. Average number of children in the home was two, with thirty-one percent of the parents having only one child, 37% having two children at home, 22% reporting three children, close to 6% reporting four children at home and 4% reporting five or more children at home. The majority of parents (43%) indicated ELOA Grant Evaluation Report 8 that high school was their highest level of education obtained, while 21%, 24% and 7% reported having an associate’s, bachelor’s or master’s degree, respectfully.

Teachers of early child care students

Sixty early child care teacher/educators participated in the initiative. That is, a total of Sixty early child care teachers/providers were exposed to the ELOA grant initiatives. It should be noted, however, that teacher demographic data was reported by early childcare center and not based on the stand-alone teacher data. Thus, each center reported will either have averaged teacher demographic data (where applicable) or the reports of one early child care provider. Given that the number of reporting child care centers was 30, data on thirty teachers were used. Teacher demographic variables included race, educational degree, years of experiences and number of teachers in the classroom. In centers where there were two child care providers, the teacher with the highest number of years of experience was considered the primary teacher of interest to be included in the analyses. Thus, her remaining demographic information was entered into the primary statistical database and analyzed.

All early child care providers were female and averaged 12 years of teaching experience. Across the 30 early child care sites, there was at least one child care provider (50%, N = 17), while 35% (N = 12) of the sites reported having two educators/teachers in the preschool classroom delivering ELOA programming. Four teachers (12%) reported earning a high school diploma as the terminal degree, 10 (29%) reported have a CDA license as the terminal degree, 4 (12%) reported earning an associate’s degree, 10 (29%) had reported earning a bachelor’s degree and only one teacher (3%) had earned a master’s degree. Nineteen of the thirty teachers (56%) were African American while nine (27%) were White. Remaining percentages were not reported.

Instrumentation

Intervention materials

Language Enrichment Activities Program (LEAP) (Carvell & Peters, 2003) curriculum guides served as the primary intervention used by ELOA staff throughout this initiative. LEAP curriculum guides are instructional guides that provide teachers with daily literacy activities to carry out with students enrolled in early child care programming across the 30 sites. The Language Enrichment Activities Program (LEAP), for instance, is an early literacy program with a scientific research base. LEAP is a multi-sensory, enriched language program which provides daily lesson plans to be used throughout the year. The teacher guides are key components of LEAP. The guides, containing daily lesson plans, emphasize the areas of receptive and expressive language, phonological awareness, knowledge of the letters of the alphabet, basic concepts, pre-writing fine motor skills, and math and science concepts, thereby increasing the children's chances for success in kindergarten. This unique program was ELOA Grant Evaluation Report 9 developed to be used with three to five year old children. The specific components included in the teacher guides are:

Language with Stories (listening with a purpose) Language with Words (expressive language) Language with Sounds (phonological awareness) Language with Letters (alphabet) Language with Ideas (concept development) Language with Motor Development (prewriting fine motor development) Language with Math and Science (predict, explore, conclude) (Please see http://www.leapsandbounds.org/curriculum.htm for additional information on the LEAP curriculum)

Another major set of materials offered to center directors to facilitate ELOA initiatives was the Resource Guide. The guide provided each center with different information regarding community resources that would be useful to parents and directors, particularly with respect to facilitating the child’s literacy development (i.e., a parent asks how I enroll my children into kindergarten, the resource guide would have all Fayette County Elementary Schools contact information)

Another set of materials included the Transition DVD, which was designed to inform parent on documents needed to enter their child in school and what their child is expected to know on entry. In particular, the Transition DVD included the Transition Packet (i.e., examples of health documents needed, school contact information, list of things needed to know before child is ready for kindergarten) and Transition Booklets (i.e., 99 Tips for School Readiness, On Track to Kindergarten)

A third set of materials included the Born Learning program. Born Learning is a public engagement campaign that seeks to help parents, caregivers, and communities create early learning opportunities for young children. The Born Learning program was designed to support a parent in their critical role as a child’s first teacher. Born Learning educational materials were made available through the efforts of United Way, United Way Success by Six, and Civitas.

Finally, toolkits containing books, literacy activities, hands-on materials for parents to check out from the center to help increase parent/child interactions at home to reinforce what was being taught in the classroom were offered to the centers’ directors. Additionally, Monthly Resource Calendar provided parent/child activities, community parent involvement opportunities, and community resources, given out monthly.

Assessment materials:

Student literacy outcomes Developmental Indicators for the Assessment of Learning (DIAL-3): The DIAL-3 ELOA Grant Evaluation Report 10

(Mardell-Czudnowski & Goldenberg, 1998) is an assessment tool used for screening preschool and kindergarten children in the area of literacy development and competence. This assessment tool and the protocol used with it are partitioned into three separate subscales. The protocol allows literacy instructors to provide a positive, non-threatening environment where children can participate in literacy competence exams in ways deemed fun as well as age appropriate and appealing to young children. Literacy-based stimuli are presented one at a time using manipulatives and other materials (e.g., pictures, drawing, etc.). The three subscales on the DIAL-3 include items to assess motor development, conceptual development and language development. Scores for each subscale are computed and then totaled to create a DIAL-3 total or composite score. Scores are derived from the teacher’s assessment of the number of correct responses or activities the preschool age child displays or engages in. Greatest possible score for the DIAL-3 is 81, with a possible score of 27 for each subscale. Alpha reliabilities for the different subscales and total scale are as follows: Motor = .66; Concepts = .84, Language =.77; DIAL-3 Total =.87. Typical cut off scores for each subscale is 17. According to the authors, pre-test scores below 17 are indicative of developmental delay.

The motor subscale area includes gross motor skill items such as catching, jumping, hopping, and skipping. Fine motor skill items are also included such as building with blocks, cutting, copying shapes and letters, and writing, and the popular finger-touching task from the DIAL-R. The concepts subscale includes items such as pointing to named body parts, naming or identifying colors, rote counting, counting blocks, placing a block in named positions relative to a little house, identifying concepts in a triad of pictures, and sorting shapes. The DIAL-3 also includes an item assessing automatic naming of colors as this skill has been shown to be associated with potential learning disabilities. The language subscale of the DIAL-3 includes items such as answering simple personal questions (name, age, and sex), articulation, naming (expressive) or identifying (receptive) objects and actions, plus phonemic awareness tasks such as rhyming and ‘I Spy’. More information about the DIAL-3 literacy assessment tool and corresponding protocol can be found here: (http://ags.pearsonassessments.com/group.asp?nGroupInfoID=a13700)

Classroom/environment literacy observation outcomes

The scales reported in this section were used to observe and evaluate the overall classroom environment. Along with an assessment of literacy-bound activities and structures within the classrooms of early child care sites, children’s health and safety, appropriate stimulation through language and activities and warmth and supportive interactions were observed. Indeed, many factors affect the care and education that early child care staff and teachers provide for the children enrolled in the early childhood programs. In fact, the cultural preferences or other beliefs of the adults involved, the curriculum approach used, the physical conditions of a building, finances or staffing issues are some examples of the factors that affect the effectiveness of a program, particularly ELOA Grant Evaluation Report 11 with its ability to provide the structures, resources and interactions listed on the environmental rating forms such as the Infant and Toddler Environmental Rating Scale (ITERS), Family Daycare Environmental Rating Scale (FDCRS), and Early Childhood Environmental Rating Scale (ECERS).

Because of these variables, it is unlikely that programs will meet every requirement on the different scales. Rather, the purpose of the environmental literacy observation scales are used to determine the strengths and challenges of each site and in doing so, offer ELOA staff an opportunity to consider where change is needed, determine how to create desired improvement and facilitate the implementation of resources, strategies, and activities that will foster greater literacy development. Each of the items on each subscale of the ECERS, ITERS, and FDCRS has a numerical response range on a 7-point scale continuum, with indicators for 1 (inadequate), 3 (minimal), 5 (good) and 7 (excellent). The scores for each subscale on each literacy environment observation scale were totaled and averaged to create ITERS, ECERS, and FDCRS average scores for each early child care site. These scores were included in statistical analyses. The reliability and validity of each scale has been reported elsewhere, thus, making them particularly useful tools for literacy program evaluation. (ECERS-Revised Edition 2003).

Early Childhood Environmental Rating Scale (ECERS): The ECERS is an observational tool used to rate preschool classrooms (ages 2 ½ -5) on the degree to which they foster optimal environmental that promote literacy engagement and competency. The ECERS contains seven subscales consisting of space and furnishings, personal care routines, language-reasoning, activities, interaction, program structure and parents and staff. The scale can be used by program directors for supervision and program improvement, by teaching staff for self- assessment, by agency staff for monitoring, and in teacher training programs. For the purposes of the ELOA grant, ELOA staff and program members performed observation of the classroom literacy-based activities and resources.

Infant Toddler Environmental Rating Scale (ITERS): The ITERS was used to rate infant and toddler classrooms (birth -30 months) and contains seven sub-scales consisting of space and furnishings, personal care routines, listening and talking, activities, interaction, program structure and parents and staff. The scale can be used by program directors for supervision and program improvement, by teaching staff for self-assessment, by agency staff for monitoring, and in teacher training programs. For the purposes of the ELOA grant, ELOA staff and program members performed observation of the classroom literacy-based activities and resources.

Family Day Care Environmental Rating Scale (FDCRS): The FDCRS is used to rate family day care centers (birth-5 years old). The FDCRS contains five subscales (space and furnishings, basic care, language-reasoning, learning activities and social development). The scale can be used by program directors for supervision and program improvement, by teaching staff for self-assessment, ELOA Grant Evaluation Report 12 by agency staff for monitoring, and in teacher training programs. For the purposes of the ELOA grant, ELOA staff and program members performed observation of the classroom literacy-based activities and resources.

Early Language and Literacy Classroom Observation (ELLCO) (Smith, Dickinson, Sangeorge & Anastasopoulos, 2002): The ELLCO is a three-part classroom observation tool that enables schools to assess and strengthen classroom quality, particularly by fostering better literacy programs and activities and thus, providing children with an early and concentrated focus on language and literacy development.

According to the website, www.brookespublishing.com, ELLCO is divided into three parts, the Literacy Environment Checklist (15–20-minute orientation to the classroom), the Classroom Observation and Teacher Interview (20–45- minute observation; 10-minute interview) and the Literacy Activities Rating Scale (10-minute book reading and writing summary). Administration of the ELLCO in one classroom, particularly in early child care centers averages about 90 minutes. The following information is taken directly from www.brookespublishing.com and describes the different subscales present in the ELLCOs.

The Literacy Environment Checklist allows users to prepare for the more detailed Classroom Observation by familiarizing themselves with the classroom environment. In about 15–20 minutes, trained observers examine the classroom’s layout and contents through 25 items that measure availability, content, and diversity of reading, writing, and listening materials.

The Classroom Observation and Teacher Interview is the second component of the ELLCO and takes about 20–45 minutes to complete. Users observe teachers interacting with children and the classroom environment, have a brief conversation with the teacher, and rate the quality of classroom supports for literacy through 14 age-specific observation elements. These 14 items cover two broad areas: general classroom environment (including organization, contents, technology, and classroom climate and management) and language, literacy, and curriculum (including reading and writing instruction, oral language use, cultural sensitivity, and assessment approaches). After the observation is complete, the third component of the ELLCO, the Teacher Interview, takes less than 10 minutes to help users clarify aspects of the observation.

The Literacy Activities Rating Scale asks observers to record how many times and for how long nine literacy behaviors occurred in two categories, Book Reading and Writing.

Across 200 preschool classrooms, the authors report inter-rater reliability at 88% for the Literacy Environment subscale, 90% for the Classroom Observation subscale and 81% for the Literacy Activities Rating Scale. The total possible score for the Literacy Environment is 41, 30 for the Classroom Observation, 40 for the Language, Literacy and Curriculum subscale, and 13 for ELOA Grant Evaluation Report 13 the Literacy Activities rating subscale. For the purpose of ELOA initiatives, ELOA staff and assistants were trained with the ELLCO’s toolkit and thus, observed early child care classrooms with the ELLCO during the intervention. Three major subscales used in statistical analyses below include the literacy environment, the combined classroom observation and language, literacy and curriculum subscales, and the literacy activities ratings scale.

In addition to observation assessment of the classrooms, a measure was created to assess the degree to which parents maintain activities that foster literacy development and competence at home. Specific, the Parent Literacy Activity Questionnaire (PLAQ) was designed to quantitative measure—through parent report—how often parents carried out specific literacy based activities.

The scale was partitioned into two subscales, the first assessing the parents’ literacy activities with their children and the second assessing the presence of reading and writing materials in the home. In the early childhood education literature (Ormrod, 2007), each of these activities was found to be linked to positive literacy development and enjoyment for reading. They include volunteering at the child’s school or in the classroom, reading to the child, helping child identify and write letters, visiting the library, and meeting with the child’s teacher. Item responses for the first subscale assessing frequency of parent literacy activity were 1=never, 2=1-2 times a week, 3=3-4 times a week, and 4=5 or more times a week. For the second subscale assessing presence of reading and writing materials, 1=none, 2=some, 3=a lot.

Procedures

ELOA personnel provided several workshops and resources throughout the duration of the initiative to parents of preschool children enrolled in early child care programs across the 30 sites. The purpose of these resources and workshops was to foster greater literacy development beyond the classroom contexts, specifically in the household of preschool children. Referred to as Parent Engagement, the purpose of the parent component was to promote and encourage parent’s involvement in their child’s education through classroom activities, parent meetings and community activities.

The parent engagement specialists informed the centers’ directors about the following programs and supplied appropriate resources to the centers for them to participate. The first was called Starting Strong-a partnership between director, teacher, and parents as a collaborative effort to form a project based on individual center needs. The centers received a specified dollar amount to assist in completing their project. Each center chooses different projects. Ex: One center made a parent information board with accessible resources. Once the group at each site worked together they were able to receive dollars to enhance their center. The overall purpose was to promote collaboration between parents and childcare staff, as well as parents becoming more knowledge of the child center’s needs. Building Lasting Relationships with Families was the second ELOA Grant Evaluation Report 14 program. Training here was intended to aid center directors in breaking down communication boundaries with their children’s parents and received helpful tips in how to get their parents more involved in their child’s education.

Regarding site-based training with center directors and staff for literacy promotion, training and follow-up assistance/implementation was provided each month to center directors using the following: Language with Stories, Language with Words, Language with Sounds, Language with Letters, Language with Ideas, Language with Motor Development, Language with Math and Language with Science. In particular, ELOA-based Literacy Coordinators conducted the following list trainings once a month during the summer of 2006 with child center directors and personnel to educate them on the LEAP curriculum, particularly its implementation. The trainings were held during the summer and when the school year started the ELOA-based Literacy Coordinators went to each center twice a month to observe LEAP activities and provide feedback both positive and negative about implementation. Coordinators were available to answer question and give technical assistance even when they were not at the center.

Results

The following section reports the research questions for the initiative, the variables examined in statistical analyses, the data analytic plan for the variables, descriptive information about each of the variables, statistically significant findings and the type of analyses used to determine such and explanation of the findings.

Research Questions

1. How effective is the ELOA intervention in augmenting literacy outcomes for preschool age children? 2. How effective is the ELOA intervention in enhancing the literacy environments that preschool age children are exposed to, specifically at home and at preschool service sites? 3. What parent, teacher and preschool site-based characteristics are associated with literacy post-ELOA intervention outcomes?

Variable List

Preschool children’s literacy outcomes  Dial-3: Developmental Indicators for the Assessment of Learning (motor, concept, language and total scale scores)

Preschool-based literacy environment outcomes  ECERS: Early Childhood Environmental Rating Scale  ELLCO: Early Language and Literacy Classroom Observation ELOA Grant Evaluation Report 15

 FDCERS: Family Day Care Environmental Rating Scale  ITERS: Infant Toddler Environmental Rating Scale

Parent/home literacy activities and demographic characteristics  Volunteering at respective daycare site  Writing materials present at home  Reading materials at home  Frequency of reading to child at home  Frequency of assisting child with alphabet letter identification  Frequency of assisting child with alphabet letter writing  Frequency of visiting library with child  Frequency of meeting with child’s daycare instructor  Frequency of attending parent workshops for literacy resources  Race/ethnicity  Number of children at home  Age Teacher/site characteristics  Teacher race  Number of teachers in a classroom  Years of teaching/daycare experience  Education

Student characteristics  Race/ethnicity  Gender  Age

Data Analytic Plan

To address the above research questions, the data gathered from this initiative were entered into the Statistical Package for the Social Sciences (SPSS, Version 12.0) software. Using SPSS, research question 1 was examined by a series of paired t-tests to examine whether differences between Dial-3 pre and post–intervention were statistically significant. Student scores across all sites were analyzed for statistical significance separately. No site by site mean score differences were carried out as this was not germane to the initial research question. By examining individual student means scores as a way to determine statistically significant differences between pre and post test scores, weighting of the means per site was not necessary (Tabachnick & Fidell, 2004). Conventional alpha probability value of .05 was used to determine statistical significance with the paired t-test, as this analysis was performed only once per data type (all students scores and all per site averaged student scores). (Please note that a legend explaining the statistical jargon throughout this manuscript is located in Appendix A).

ELOA Grant Evaluation Report 16

For research question 2, the three subscales of the ELLCOs observations along with scores on the ECERS, FDCRS, and the ITERS scales were analyzed to detect statistically significant differences between pre and post-intervention scores. Another paired t-test was performed to detect differences in scores as a function of ELOA intervention. Here, paired t-tests were conducted with each of the three ELLCOs subscales (literacy environment, classroom observation & language, literacy, and curriculum subscale and literacy activities), along with one paired t-test for each additional literacy environment observation scale (ECERS, FDCRS, and ITERS).

Average ELLCO scores, along with average ECERS, FDCRS, and ITERS scores were computed and analyzed as the majority of the pre-test information for the four different observation tools was presented in an ‘averaged’ form, thus, not allowing for evaluators to create and compute variables that would represent each site’s subscale literacy observation score independently. Conventional alpha probability value of .05 was used to determine statistical significance with the paired t-test, as this analysis was performed only once per data type (all five subscales of the ELLCOs along with pre and post test data for each additional literacy environment observation scale). In addition to these analyses, paired t- test analyses were used to determine the literacy activities and practices in the homes of preschool children vis-à-vis, examination of statistical difference between pre and posttest means for the two subscales of the PLAQ.

For research question 3, a correlation matrix was computed to examine interrelationships between all parent, teacher, site characteristics and student performance outcomes. Conventional alpha probability value of .05 was used to determine statistically significant correlations.

Findings:

How effective is the ELOA intervention in augmenting literacy outcomes for preschool age children?

Pre and post-intervention DIAL-3 scores were averaged for 130 students and examination of statistically significant difference between the two sets of scores was executed with paired t-tests. Analyses were performed for each subscale of the DIAL-3 (i.e., motor, concept, language and DIAL-3 total). Figure 1 presents the means for the each DIAL-3 assessment while Table 1 lists the descriptive information for each DIAL-3 subscale.

Paired t-test analyses reveal statistically significant differences between the DIAL-3 pre-intervention (herein referred to as pre-test) scores and DIAL-3 post-intervention (herein referred to as post-test) scores for each subscale. In particular, the mean score for DIAL-3 motor pre-test was 12.02 and post-test was 17.40. Corresponding t-test analyses revealed a statistically significant difference between these two sets of DIAL-3 motor means (t=-11.28, p<.001; N=129). For ELOA Grant Evaluation Report 17 the DIAL-3 concept pre and post-test scores, t-test analyses revealed statistically significant differences between the two sets of scores for students across the 30 sites (DIAL-3 pre = 12.67 and DIAL-3 post = 17.01; t=-12.19, p<.001; N=130).

For the DIAL-3 language pre and post-test scores, t-test analyses revealed statistically significant differences between the two sets of scores for students across the 30 sites (DIAL-3 pre = 11.93 and DIAL-3 post = 16.96; t=-17.29, p<.001; N=130). Finally, for the DIAL-3 total pre and post-test scores, t-test analyses revealed statistically significant differences between the two sets of scores for students across the 30 sites (DIAL-3 pre = 36.49 and DIAL-3 post = 50.90; t=-15.01, p<.001; N=130).

Table 1: Descriptive information for the DIAL-3 pre and post test mean scores.

Scale D3- D3- D3- D3- Descriptives motor concept langua total ge Pre-test 12.02 12.67 11.93 36.49 Mean Posttest 17.40 17.01 16.96 50.90 mean N 130 130 130 130

t-test -11.28 -12.19 -17.29 -15.01 statistic

Figure 1: Averaged student means for DIAL-3 subscales and DIAL-3 total

40 DIAL3-motor 30 DIAL3-concept DIAL3-language 20 DIAL3-total 10

0 pre post ELOA Grant Evaluation Report 18

Here, statistically significant improvement had, on average, occurred for the 130 children assessed at the thirty early child care sites. In particular, examination of the mean differences for the DIAL-3 total indicate that, since the beginning of ELOA intervention using LEAP programming and other components designed to enhance literacy competency among preschool age children, total literacy outcomes increased by 14 points. Given that the total possible DIAL-3 score was 81 (27 total score x 3 subscale scores) and a combined subscale score of 54 (17 score indicative of no developmental delay x 3 subscale scores) suggests that students are scoring above the cut off score indicative of developmental delay, the total score of 50.90 or 51 reveals that students in the current initiative made significant improvement across the sites.

How effective is the ELOA intervention in enhancing the literacy environments that preschool age children are exposed to, specifically at home and at preschool service sites?

In this section, the mean pre and posttest scores for the ELLCOs subscales along with the total pre and posttest scores for the ECERS, ITERS, and FDCRS observation tools are presented. Again, each of the items on each subscale of the ECERS, ITERS, and FDCRS has a numerical response range on a 7-point scale continuum, with indicators for 1 (inadequate), 3 (minimal), 5 (good) and 7 (excellent). It should be noted that total and subscale scores were provided for the early child care sites and thus, analyses will examine statistically significant means for each scale across the 30 sites that provided pre and posttest data. However, given the age appropriation for the ECERS, ITERS, and FDCRS observation forms, each of these will be presented separately as different sites had different observational forms. Also, note that only twenty-one of the 30 initial early child care sites had completed both pre and posttest literacy observation (FDCRS, ITERS, and ECERS) forms. Figure 2 presents the means for the ECERS, ITERS, and FDCRS averaged scale score, while Figure 3 presents the means for each subscale of the ELLCOs observational tool.

ECERS (N=7)

Paired t-test analyses revealed no statistically significant difference between the pre (2.84) and posttest (3.25) ECERS averaged scores for the seven sites (t=-1.67, p=.15). While the means are observably distinct from one another, it is likely that the small sample size (N=7) coupled with the relatively small difference in the two sets of scores yielded insufficient power to obtain a significant difference at the conventional .05 alpha level (Grimm, 1993).

ITERS (N=6) ELOA Grant Evaluation Report 19

Paired t-test analyses revealed no statistically significant difference between the pre (3.23) and posttest (3.49) ITERS averaged scores for the six sites (t=-.90, p=.41). While the means are somewhat observably distinct from one another, it is likely that the small sample size (N=6) coupled with the relatively small difference in the two sets of scores yielded insufficient power to obtain a significant difference at the conventional .05 alpha level (Grimm, 1993).

FDCRS (N=8)

Paired t-test analyses revealed a statistically significant difference between the pre (3.92) and posttest (4.88) FDCRS averaged scores for the eight sites (t=- 3.12, p=.01). Here, those sites completing FDCRS observations yielded close to a one-point increase in literacy structures and activities and resources since the time of ELOA intervention. While most early child care centers approached, on average, a rating of minimal at posttest, only those completing the FDCRS approached, on average, a rating of good.

Table 2: ECERS, ITERS, & FDCRS pre and post descriptives (overall)

Scale ECERS ITERS FDCRS Pre-test mean 2.84 3.23 3.92 Post-test mean 3.25 3.49 4.88 N 7 6 8 t-test statistic -1.67 -.90 -3.12

Figure 2: Pre and post test means for ECERS, ITERS, and FDCRS total scale scores (averaged).

5 4.5 4 3.5 3 ECERS 2.5 ITERS 2 FDCRS 1.5 1 0.5 0 pre post ELOA Grant Evaluation Report 20

ELLCOs (N=30)

Data for the ELLCOs observations forms were provided for all 30 sites and thus, each site’s pre and posttest scores were examined using paired t-test statistical analyses. The first subscale, literacy environment, had a total possible score of 41. Pretest scores were averaged across the 30 early child care centers and a mean of 17.80 was obtained, while a mean of 33.53 was yielded at posttest. Paired t-test analyses revealed that the difference between these two means was statistically significant (t=-8.19, p<.001). Here, ELOA raters observed significantly more literacy-based activities and resources at posttest than at pretest.

For the literacy, language and curriculum and classroom observation scales, ELLCO observation forms provide subtotals for each subscale. These are combined to create a total score, which was averaged across the 30 sites at pre and posttest and then examined for statistical significance via paired t-tests. At pretest, the average score was 36.93 out of a possible 70. At posttest, the mean score for the 30 early child care centers was 55.70 out of 70. Paired t-test analyses revealed a statistically significant difference between averaged scores at pre and posttest, with posttest scores increasing, on average, almost 20 points (t=-6.40, p<.001).

Finally, for the literacy activities ratings, pretest means across the 30 sites averaged 4.27 while posttest means were computed at 8.37. The difference between the pre and posttest means for these ratings was statistically significant (t=-5.43, p<.001).

Table 3: Descriptive statistics for ELLCOs subscales

Scale Literacy Subtotal Literacy Activities Environment (classroom ob & language & Curriculum) Pre-test 17.80 36.93 4.27 Post-test 33.53 55.70 8.37 N 30 30 30 t-statistic -8.19 -6.40 -5.43

Given that the Parents Literacy Activities Questionnaire (PLAQ) was developed solely for the purpose of assessing literacy activities of parents ELOA Grant Evaluation Report 21 involved in this initiative, alpha reliabilities were computed for the two subscales of the measure. The first subscale (6 items), which assessed the degree to which parents maintained specific activities that foster literacy development and competence, yielded an alpha coefficient of .60 at pretest (N=115) and .73 at posttest (N = 66). The second subscale (2 items) assessed the presence of reading and writing materials at home and yielded alpha coefficients of .72 and .76 at pretest and posttest, respectively. While these coefficients indicate low to moderate levels of reliability for the PLAQ subscales and thus, enable researchers to combine per item scores to create a total score, it was desired by ELOA staff to. Figure 3: Means for ELLCOs subscales 60

40 literacy environment 30 CO/LLC subtotal

20 literacy activities 10

0 pre post examine pre and posttest differences for each parent reported literacy activity separately Using this method, ELOA staff members were able to gain a greater understanding of what specific activities they should strengthen in their workshops and resource provision to parents.

As a result, several paired t-test analyses were executed with the six items of the first PLAQ subscale and also with the two items that created the second PLAQ subscale. The parent sample that completed both pre and posttest PLAQ were included in the analyses (N=66). Again, item responses for the first subscale assessing frequency of parent literacy activity were 1=never, 2=1-2 times a week, 3=3-4 times a week, and 4=5 or more times a week. For the second subscale assessing presence of reading and writing materials, 1=none, 2=some, 3=a lot.

Only one question of the eight questions reached statistical significance in its pretest and posttest mean differences. Another was reported as marginally ELOA Grant Evaluation Report 22

significant. In particular, for the question, “how many times do you help your child write letters”, parents at pretest reported a mean of 2.07, while at posttest, the same parents reported 2.37 (t=-3.40, p<.001). The marginally significant difference was found for the pretest and posttest means reporting on how many times parents met with their child’s teacher. Here, pretest means were 2.59, while posttest were 2.80 (t=-1.81, p=.07). All other item mean differences were not statistically significant. Table 4 reports the descriptive statistics for the PLAQ findings, while Figure 4 charts the descriptives pictorially.

Table 4: Descriptives for PLAQ items-both subscales

Scale volunteer Read Identify Write Visit Meet Writing Reading to letters letters library with materials materials child teacher Pre- 1.45 3.10 2.78 2.07 1.36 2.59 2.88 2.80 test Post- 1.55 3.20 2.96 2.37 1.48 2.80 2.84 2.84 test N 66 66 66 66 66 66 66 66 t- -1.03 -1.02 -1.62 -3.40 -1.53 -1.81 .77 -1.14 statistic

Figure 4: PLAQ pre posttest item means (significant means differences only)

1.5 write letters meet w/teacher 1

0 pre post ELOA Grant Evaluation Report 23

What parent, teacher and preschool site-based characteristics are associated with literacy post-ELOA intervention outcomes?

In this section, the significant associations between parent, teacher, and site-based characteristics were examined. In particular, characteristics and reported practices from each set of participants were examined for significant correlation with posttest scores on each literacy outcome for students (i.e., DIAL- 3 subscales), their sites (i.e., ELLCOs, FDCRS, ECERS, & ITERS), and their home environment maintained by their parents (i.e., PLAQ subscale items). The correlation data are organized in three sections. Significant correlations were determined by Pearson correlation coefficients that were statistically different from zero (e.g., statistically significant).

Significant correlations are noted with an asterisk and are bolded. The first section presents correlations between literacy outcomes and parent characteristics and practices and the second between literacy outcomes and teacher characteristics and the last section discusses associations between literacy outcomes and student characteristics. Significant correlations are discussed within each section.

Parent associations

Of great interest to ELOA staff members was whether parents’ literacy activities at home and other activities were associated with their children’s literacy performance outcomes. Examination of one of the correlation matrices used for this study found that older parents, on average, had children that scored significantly lower on the DIAL-3 concept posttest (r=-.47, p=.04). This finding emerged when examining overall Additional associations include the findings that older parents tend to read to their children more often (r=.31. p=.01) and meet with their child’s teacher less often (r=-.26, p<.04). Regarding the average number of children at home, this variable was negatively associated with the reported number of reading materials in the home (r=-.42, p<.001) and the number of times parents read to their children (r=-.42, p<.001). Thus, the more children in the home, the less often children were read to by their parents and the less materials present to read at home. It must be noted however, that none of these variables were significantly related to DIAL-3 literacy performance for children.

Parent race was negatively correlated with the average number of writing materials present in the home (r=-.26, p=.03). Examination of the means for the number of writing materials at home for the three ethnic groups present in the parent sample (White, African American and Hispanic) revealed that White parents had, on average, reported significantly more writing materials (2.91, N=44) at home than did Hispanic parents (2.50, N=4). The difference between White and African American parents (2.72, N=18) and African American and Hispanic parents was not significant. Finally, the more education parents obtained, the more writing and reading materials present in the home along with ELOA Grant Evaluation Report 24 more reading to the child (r=.44, p<.001, r=.35, p=.003, & r=.38, p=.001, respectively).

Student associations

Examination in the student correlation matrix was demographic variables such as race, gender, and age (as provided on the DIAL-3 assessment form) and DIAL-3 sub and total scale performance outcomes. Only child age was significantly associated with literacy outcomes, with a positive and statistically significant correlation coefficient being produced for the motor, concept, language subscale and the total DIAL-3 scale (r=.54, p<001, r=.32, p<.001, r=.37, p<.001, & r=.45, p<.001, respectively). As would be expected, older children produced higher scores on the DIAL-3 at posttest.

Teacher and site associations

Regarding teacher characteristics and classroom practices, several significant associations emerged from the data. For instance, for the ELLCO literacy environment subscale, teachers with more education and teachers that had assistance in their classroom (i.e., another teacher) had higher ELLCO literacy environment ratings (r=.42, p=.02 & r=.65, p=.001). Also, the number of teachers in a classroom was positively correlated with the classroom observation and language, literacy and curriculum subtotal and the literacy activities rating of the ELLCO (r=.50, p=.01 & r=.40, p=.03). Regarding average DIAL-3 performance outcomes yielded by students, correlation computations found that higher teacher degrees was significantly related to DIAL-3 motor subscale score, language subscale and total score performance (r=.53, p=.03, r=.60, p=.03, & r=.62, p=.001, respectively). Also, the number of teachers in a classroom was positively associated with DIAL-3 motor subscale performance and total score performance (r=.65, p<.001 & r=.60, p=.001).

Regarding site-based characteristics, it was shown that higher literacy environment posttest observations were significantly associated with all DIAL-3 posttest subscale scores and the DIAL-3 average total score (r=.47, p<.03, r=.41, p=.06; r=.60, p=.003; r=.56, p=.006, respectively). Similarly, the combined classroom observation and language, literacy and curriculum subscales were significantly and positively related to DIAL-3 language posttest scores. None of the FDCRS, ITERS, or ECERS scores were related to DIAL-3 posttest outcomes. However, this is likely to insufficient statistical power to detect a significant correlation among these data.

Discussion and Future Directions

According to Biancarosa & Snow (2004), there are 15 essential elements to an effective literacy program. These include professional development, formative assessment, and summative assessment and an established literacy program in ELOA Grant Evaluation Report 25 place to assess (Biancarosa & Snow, 2004). The current report highlights the results of an initiative designed to promote literacy development and readiness in classrooms vis-à-vis these stated elements.

Over the course of one year, ELOA staff researched and implemented several parent and director-centered strategies to promote literacy development across 30 early child care centers in Lexington- Fayette County, Kentucky. The findings of the effects of ELOA programming on student and site-based literacy outcomes were positive. Paired t-test statistical examinations revealed that on average, DIAL-3 motor, concept, language and total scores for literacy improved significantly since the implementation of ELOA programming. Also, the scores on the ELLCO literacy environment observation tool were significantly improved at posttest. Only one set of pre/posttest averaged scores of the three additional classroom observational tools proved statistically significant scores (FDCRS). Also, since the implementation of ELOA programming and the provision and monitoring of various materials by ELOA staff, parents across the different sites reported statistically significant increases in the amount of time they spent helping their children write letters and also meeting with their child’s teacher.

These findings are in alignment with additional studies that have suggested that increased parent involvement and a more direct, teacher-centered focus on literacy development throughout early child care facilitate literacy development (Cunningham & Stanovich, 1997; Juel, et.al., 2003). For example, Juel and colleagues (2003) found that although teachers varied in their delivery of literacy instruction, the exposure to such instruction coupled with a classroom environment enriched with activities and structures that foster literacy development enhanced students’ literacy outcomes.

Future directions with this line of inquiry focusing on the effects of literacy interventions on literacy context and performance outcomes ought to include a more systematized data collection process examining the fidelity of program initiatives carried out across the multiple child care sites. In particular, Zvoch, Letourneau, & Parker (2007) note that without an idea of how well program initiatives were carried out, evaluators and in this case, ELOA staff can only rely on anecdotal evidence to support and conclude how effective program initiatives were in improving literacy outcomes at the contextual and individual performance levels. ELOA Grant Evaluation Report 26

References

Biancarosa, G., & Snow, C. E. (2004). Reading next—A vision for action and research in middle and high school literacy: A report to Carnegie Corporation of New York. Washington, DC: Alliance for Excellent Education.

Carvell, N. & Peters, C. (2003). Language Enrichment Activities Program. Southern Methodist University Learning Therapy Program, Dallas, TX.

Cunningham, A., & Stanovich, K. (1997). Early reading acquisition and its relation to reading experience and ability 10 years later. Developmental Psychology, 33, 934-945.

Grimm, L.G. (1993). Statistical applications for the behavioral sciences. Wiley & Sons, Inc: New York, NY

Juel, C., Biancarosa, G., Coker, D., & Deffes, R. (2003). Walking with Rosie: A cautionary tale of early reading instruction. Educational Leadership, 60(7).

Ormrod, J.E. (2007). Educational Psychology: Developing Learners. Upper Saddle River, New Jersey: Prentice Hall.

Tabachnick, B.G. & Fidell, L.S. (2001). Using multivariate statistics (4th Eds.). Allyn & Bacon: Boston, MA.

Zvoch, K., Letourneau, L.E., & Parker, R.P. (2007). A multilevel multisite outcomes-by-implementation evaluation of an early childhood literacy model. American Journal of Evaluation, 28(2), 132-150. ELOA Grant Evaluation Report 27

Appendix A: Statistics glossary

 Alpha coefficient= a numerical score (coefficient) from 0 – 1 that indicates the degree to which items on a scale are “reliable” in their measurement of a given construct. The higher the coefficient, the stronger the reliability of the items on a scale. Coefficients that approximate 1 (i.e., 95 and higher) indicate the likelihood that the same question is being asked, despite the presence of multiple questions.  Alpha probability= a numerical score used to determine whether a computed statistic for mean differences or correlations is statistically significant. Conventional alpha probability is typically set at .05. If the computed statistic has an alpha level that is at or below the conventional statistic offered at .05 alpha level, then there is statistical significance with the finding.  Correlation coefficient= a numerical score (coefficient) from 0-1 that indicates the degree to which scores from two different scales are associated. The higher the coefficient, the stronger the association. Directionality is also considered in correlation, wherein positive coefficients indicate an alignment between scores (i.e., as one increases or decreases, the other does the same) and negative coefficients indicate an inverse relationship (i.e., as one increases, the other score decreases).  Error variance= person and situational factors that can explain anticipated results and in doing so, rival the explanation of results offered by researchers/evaluators’ hypotheses.  Paired t-test= a statistical examination of difference between two averages produced by the same entities, bodies, etc. What is paired are the error variance associated with those persons, entities, etc. T-test analyses invariably examine differences between two sets of means. Alpha probability and the probability corresponding to the computed t-statistic are compared. If the latter is less than or equal to the former, then statistical significance in the observed mean difference has been reached. ELOA Grant Evaluation Report 28