Leveled Literacy Intervention

Fountas and Pinnell Leveled Literacy Intervention Spring 2011 Program Evaluation

Leveled Literacy Intervention 2

Fountas and Pinnell Leveled Literacy Intervention Evaluation Report

Table of Contents

I. Executive Summary pg. 3

II. Introduction to the Report pg. 7

a. Purpose of the Evaluation pg. 7

b. Audiences for the evaluation report pg. 7

c. Limitations of the evaluation pg. 8

d. Overview of report contents pg. 8

III. Focus of the Evaluation pg. 8

a. Description of the evaluation objects pg. 8

b. Evaluative questions used to focus the study pg. 9

c. Information needed to complete the evaluation pg. 9

IV. Brief Overview of evaluation plan and procedures pg. 10

V. Presentation of evaluation results pg. 12

a. Summary of evaluation findings pg. 13

b. Interpretation of evaluation findings pg. 13

VI. Conclusions and recommendations pg. 14

VII. Appendices pg. 15

Leveled Literacy Intervention 2 3

Executive Summary

Purpose

The Fountas and Pinnell Leveled Literacy Program began at Lyman Hall Elementary

School three years ago. The LLI Program is available for students in kindergarten through fifth grade. The Leveled Literacy Intervention System is a small-group, supplementary intervention designed for children who find reading and writing difficult. LLI is designed to bring children quickly up to grade-level competency—in 14 to 18 weeks on average. LLI serves those students who need intensive support to achieve grade-level competency.

These children are the lowest achieving children in the classroom who are not receiving another supplementary intervention. Each lesson in the LLI system also provides specific suggestions for supporting English language learners who are selected for the program.

An evaluation of the Leveled Literacy Intervention Program was conducted for a few reasons. The program has been in place for three years and has never been evaluated. Doing an evaluation was a good way to decide what is working with the program, what may need to be changed, or what is not working with the program. Several questions were addressed in the evaluation. The main questions addressed and researched for this program evaluation is as follows: How many students recover before 14 weeks, during the 14-18 week period or do not recover at the completion of 18 weeks. What are the parent, teacher and staff perceptions about the Leveled Literacy System?

Information Collection

When completing the LLI Program evaluation, vital information was gathered. The primary source of collecting data for this evaluation was through formal DRA’s (directed reading assessments) and informal DRA’s known as cold reads. The information gathered

Leveled Literacy Intervention 4 through these assessments were then used to answer evaluation questions related to whether students are reading below grade level or reading on grade level. Interviews were also conducted with parents, teachers, and staff members to better understand the perceptions about the LLI Program.

Students had to answer the following questions when completing the DRA and cold reads. As well as answering these questions students receive a percentage on their reading ability. (see appendices DRA)

 Look at the title and the pictures and tell me what you think is going to happen in

the book? (Prediction)

 Start at the beginning and tell me what happened in the book? (Retelling)

 What was your favorite part and why?

 What connection did you make while reading the book?

By having students answer these questions many conclusions were made. Below is a list of findings. A more detailed list with explanations can be found in the full evaluation report.

Findings

 Out of 5 students evaluated for the LLI Program 0 of the students were on grade

level before 14 weeks.

 Out of the 5 students evaluated for the LLI Program 0 of the students were on grade

at 14 weeks.

 Out of the 5 students evaluated for the LLI Program 0 of the students were on grade

level after the 18 week completion of the program.

The findings from the parent, teacher and staff surveys were unanimous. The main concern expressed through the survey was that the program benefits middle class society as well as

Leveled Literacy Intervention 4 5 those students with a good English vocabulary. The students within this program are students who speak Spanish as their first language and English as their second. Their families are below the poverty line, they are considered migrant students who move for work and they do not receive help from home. The teachers and staff suggested piloting a program that is specifically geared toward low end, second language learners.

Judgments

Overall the students and staff were very willing to help in data collection. The evaluation indicates that the program is not recovering students to grade level reading within the specified time, 14-18 weeks. The evaluation also shows that students are becoming frustrated with their reading skills/abilities and are then choosing not to read.

Recommendations

After viewing all of the data, a list of recommendations has been made. Please refer to the entire report for more information and explanations.

 Continue to use the LLI structure of even days reading books and odd days

writing about the books, 5 minutes reread, 5 minutes phonics work, 15

minutes new book or writing depending on day and 5 minutes word work.

Seeing Star boxes can add to classroom word work to allow for

differentiation, more exposure and use of nonsense words (more

explanation is given under recommendations later in the report).

 Instead of using LLI books use Lyman Hall Leveled Books developed for English

Language Learners on their independent reading level and instructional reading

level.

 Before taking a warm read or formal DRA allow the student to read a warm

read book (independent level) to get their confidence built and then

administer the cold read or formal DRA.

The LLI Program at Lyman Hall Elementary School is not working well and needs to be improved. From the feedback and data collected it is obvious that the program is not recovering students to grade level within the 14-18 week time line. With the recommendations provided, the LLI Program can in turn reach below grade level students and have them continue to grow in their reading levels.

Please review the full report for more information and data that reflects the information stated in the Executive Summary.

Introduction

Purpose of Evaluation

The main purpose of the Leveled Literacy System evaluation was to determine whether the program was meeting the objectives that had been outlined in the description of the program. This evaluation evaluates if the students receive the amount of support needed in order to benefit from the system’s objectives. The main question that arises during this evaluation was, are the students being able to close the reading gap after completing the program? The evaluation of the Leveled Literacy System was conducted to research this question to determine if this program was benefiting the student’s reading level. Data was collected from administration and staff members who have first-hand experience with this literacy program. This data gave us a picture of how this program was working within the real world setting.

Audiences of the Report

This evaluation can help all staff members that are involved with the Leveled Literacy

System. Teachers who use this intervention system to improve their student’s reading level are the targeted audience who will benefit the most from the outcome of this evaluation. Other groups of people like parents and other school systems can use this evaluation as an insight as to how the program works and how effective the program is. Each person who reviews this

Leveled Literacy Intervention 8 evaluation will be able to see the positives and negatives, as well as the areas in which improvements can be made.

Limitations of the Evaluation

This evaluation only had several limitations. The main limitation was the teachers did not have an universal recording system where all administrators could see the process of the students who participated in the leveled literacy intervention. The teachers track their students on their own and keep their recordings for their records. If there was a progress monitoring sheet for each teacher to record the students’ progress then the administrators and other teachers could easily keep track of each student and his progress within the system. This progress monitoring sheet would be saved on the school drive and would be created to use for the recording of the Level Literacy System’s outcomes.

Overview of the Report Contents

This report includes a description of the Fountas and Pinnell Leveled Literacy System and how it is used to increase the reading levels of struggling students. The audience of this evaluation will learn what questions were important when evaluating the system. Included is also a description of the evaluation plan and procedures which were used to evaluate the system. A summary and interpretation of the findings from the research of the level literacy system are also included as a part of the evaluation. In the final section, recommendations and conclusions are made for this intervention program.

FOCUS OF THE ELVAUATION

Description of the evaluation object:

The evaluation focuses on the Fountas and Pinnell Leveled Literacy Intervention System.

This intervention program is designed for a small group setup. The system is not a reading

Leveled Literacy Intervention 8 9 program, but a supplementary program to be used in addition to the reading program that is already in place for that school system. The small group who received the supplementary program are those students who need intense support to reach grade level competency.

Therefore, the students who receive the leveled literacy intervention are low-achieving children who are not receiving other forms of supplementary intervention.

The leveled literacy intervention is a short term program that has been proven to bring students up to grade level competency in reading in fourteen to eighteen weeks. It is broken into three systems that support different levels of learners. Students are tested using the running record testing program to determine which system (orange, green, or blue) that the child needs to begin the appropriate support instruction.

Each system contains a collection of at least seventy books. For each book, there is a lesson for the teacher to follow. Within each lesson, there is an emphasis on reading skills as well as writing skills. The specific components of each lesson are listed below:

 Comprehension strategies

 Attention to fiction and nonfiction texts

 Specific work on sounds, letters, and words (and how words “work”)

 Vocabulary practice

 Fluency and phrased reading

 Writing about reading to practice writing strategies

Evaluative questions used to focus the study:

 How many students recover before 14-18 weeks, during the 14-18 week period

or not recover at the completion of the 18 week?

 What are the parent, teacher, and staff perceptions about the leveled literacy

system?

Information needed to complete the evaluation:

Before beginning the evaluation information had to be collected on the Leveled Literacy

Intervention Program. Individual student’s formal and informal DRA assessments were collected to evaluate their progress throughout the program. This is the greatest source of information for us to use as we measured the effectiveness of the program. The formal DRA assessment was given twice during the program and the informal DRA assessment was given every two weeks with a total of sixteen assessments. Depending upon the score on these assessments determined whether the student would then revisit the current level or progress to a new level and which level they would progress to.

Brief Overview of Evaluation Plan and Procedures

Before this evaluation began, there were extensive steps to complete the background research for the Leveled Literacy System. Research was completed to gain information about the Leveled Literacy System. There were documents that were collected to understand the objectives and goals of the program. Documented outcomes of students who completed the program were analyzed to determine exactly how the program is working in a particular school system. The main focus of the evaluation was discussed based on items collected. The evaluation questions were formed after these items had been analyzed.

The second step of the evaluation was to complete an evaluability assessment. This assessment is a tool that serves a communication between the stakeholders and evaluators to make sure the program is worthy of the evaluation. Everyone that was involved in this evaluation was a part of the evaluability assessment. Teachers and parents were able to Leveled Literacy Intervention 10 11 share concerns that were taken into consideration for the assessment. Including all persons involved allowed the evaluators to see if the program was meeting the needs of all.

After reviewing the evaluability assessment, the decision was made that the evaluation would be completed. The idea was discussed that the outcome of this evaluation would help school systems across the state to determine if the program would work based on the specific needs. Documentation has been withheld from testing, parent and teacher surveys, and student achievement to provide the information needed throughout the process.

After the evaulability assessment was complete, the next step of the evaluation was to create the standards and criteria. Each standard was used to evaluate the success of each question in the evaluation. The standards and criteria that were produced followed along with the Georgia Performance Standards for the specific grade level that it was being used in. The standards also align with the evaluation questions that were produced at the beginning of the evaluation.

A spreadsheet was created to organize all information that summarized the evaluation plan. Within this spreadsheet, one could determine the evaluation questions, information required, design, information source, method of collecting information, sampling, information collection procedures, schedule, and analysis procedures. It was determined that the data collected would come from teacher information about the success of students within a specific grade level.

Finally, the plan was placed into action when the data collection process began.

Students were monitored as they completed the Leveled Literacy System with their homeroom teacher. This tool was used to generate a specific percentage of students who were closing the reading gap before the 14 week period, during the 14 – 18 week period, or

Leveled Literacy Intervention 12 not closing the gap at all. We compared these results with the results of the teacher surveys to obtain any relation between the program outcomes and the administers.

Presentation of Results

Summary of Findings

Conclusions were drawn based on test scores from Running Records and Developmental

Reading Assessments. Feedback from parents and teachers were also used to make conclusions. Zero percent of students between the 14-18 weeks of intervention, closed the reading gap to meet grade level peers and grade level standards according to the test results.

When conducting the online survey, we had lack of parent response, and ninety-five percent of teachers responded. Ninety-five percent of teachers felt that the Fountas and Pinnell program was not an efficient reading intervention program. As mentioned earlier the report the program is too repetitive and students lack motivation for moving up levels. It was also stated that students were frustrated and bored with the Leveled Literacy Intervention program causing lack of motivation to build fluency and comprehension. A first grade teacher stated that students need hands-on learning manipulatives and visuals to manipulate diphthongs, diagraphs and letter sounds to build fluency. Another first grade teacher, expressed concerns with the lack of background knowledge and vocabulary for second grade content due to lack of parent involvement. Zero percent of parents participated in the online survey of the Fountas and

Pinnell Literacy Program.

Interpretations of the Findings

The evaluation indicates that the program is not helping students close gaps in order to meet grade level reading competency. Because students are bored with the repetitive nature of the program, they are unable to view themselves as successful grade level readers. It is also mentioned that the program is designed for middle class students who speak only English. The main concern expressed through the survey was that the program benefits middle class society as well as those students with a good English vocabulary. The students within this program are students who speak Spanish as their first language and English as their second.

Another concern is that when using the program, teachers do not have the opportunity to make the learning meaningful. Research states that when students internalize and make connections to their reading they are more likely to build fluency and comprehension. Due to the students’ lack of exposure to reading content and vocabulary, they are unable to make connections. These students’ first need to understand why it’s important to be able to read, then they will be more likely to internalize and connect to the content; lack of parent involvement creates a huge barrier for these students.

Conclusions and Recommendations

After reading the data, a list of recommendations with explanations has been made. Since teachers believe the root of the problem is the focus on students who speak one language and the repetitive nature of the program, suggestions have been made to incorporate more hands on, visual learning.

1. Teachers can incorporate graphic organizers to aid in teaching comprehension strategies. Use Lyman Hall Leveled Books developed for English Language Learners on their independent reading level and instructional reading level; then have students create graphic organizers in small groups to connect to the content.

Comprehension strategies can be taught through making hands on material that can help students visualize and connect to the content. For students to find the plot of the story, they could use paper to create a house. At the top of the house, they could write or draw about what happened in the beginning of the story, in the middle of the house they could write or draw about the middle of the story and at the bottom of the house they could write or draw about what happened at the end of the story; students have now created a graphic organizer.

Before assessing comprehension by using a cold read or formal DRA allow the student to read a warm read book (independent level) to get their confidence built and then administer the cold read or formal DRA.

2. Teachers could incorporate Seeing Stars boxes into daily reading rotations. The teacher holds up a nonsense word (not a real word) from the box. She then has students use letter tiles to make the word. A student reads the word and then the teacher asks the student to delete or add letters to create a new word. At first, the goal is for the teacher to have the students create words they can easily recognize to build self-esteem then, the teacher has the students build more difficult words as they become more comfortable. Seeing star boxes allow teachers to individualize instruction for phonemic awareness, sight words and spelling, which also builds grade level fluency. On Monday, the teacher would get a baseline and on Friday, teachers will assess the phonemic awareness skill worked on for the week. For example if the teacher worked on long vowels she would get a baseline on Monday to see what the students already know.

Throughout the week, the teacher would focus on long vowels using Seeing Star Boxes during reading rotations. On Fridays, she would assess words the students have never seen to get the results and help guide her instruction for the next week. If the students needed more work on long vowels, she would continue to work long vowels and create new words for the following week to assess.

Appendices

DRA short form for recording reading accuracy and comprehension.

Leveled Literacy Intervention