DOCUMENT RESUME ED 427 320 CS 216 580 AUTHOR Hill, Bonnie TITLE Oregon Statewide Writing Assessment: Results, Analysis, and Sample Student Writings, 1993-1997. Grades 8,10, 11. INSTITUTION Oregon State Dept. of Education, Office of Assessment and Evaluation, Salem. PUB DATE 1998-02-00 NOTE 119p. AVAILABLE FROM Oregon Department of Education, 255 Capitol St., N.E., Salem, OR 97310-0203. PUB TYPE Reports Research (143) EDRS PRICE MF01/PC05 Plus Postage. DESCRIPTORS *Evaluation Methods; Grade 10; Grade 11; Grade 8; Secondary Education; State Standards; Student Evaluation; Student Writing Models; *Writing Achievement; *Writing Evaluation; Writing Instruction IDENTIFIERS *Oregon Statewide Assessment Program ABSTRACT Noting that statewide writing assessment shows the achievements of Oregon students over the last decade, this booklet explains the results for the 1993-1997 assessments and provides papers illustrating the work that Oregon students in grades 8,10, and 11 produce on the state test. The booklet's chapters are:(1) The History of Writing Assessment in Oregon;(2) The Analytic Models and Highlights of Assessment Results;(3) The Classroom Connection; and (4) Statewide Results and Sample Student Papers. Appendixes contain: a writing scoring guide; mode scoring guide; sample prompts, grades 8,10, and 11; guides to revision, grades 8 and 10; student writer's report; requirements for collection of student work samples; conversion tables from 5- to 6-point scoring scale; inter-rater reliability 1996-1997; writing content panel members; and a 19-item annotated bibliography of material on writing assessment and instruction. (CR) ******************************************************************************* Reproductions supplied by EDRS are the best that can be made from the original document. *******************************************************************************' Oregori C5 STATEWIDE ASSESSMENT RESULTS, ANALYSIS, and SAMPLE STUDENT WRITINGS 1993-1997 GRADES 8, 10, 11 U.S. DEPARTMENT OF EDUCATION Office of Educational Research and Improvement PERMISSION TO REPRODUCE AND EDUCATIONAL RESOURCES INFORMATION DISSEMINATE THIS MATERIAL HAS CENTER (ERIC) BEEN GRANTED BY 'K. This document has been reproduced as received from the person or organization originating it. ii24/1-) 0 Minor changes have been made to improve reproduction quality. TO THE EDUCATIONAL RESOURCES o Points of view or opinions stated in this INFORMATION CENTER (ERIC) document do not necessarily represent official OERI position or policy. Norma Paulus, State Superintendent of Public Instruction Oregon Department of Education, Salem, Oregon 2 Oregon Statewide Writing Assessment 1993 to 1997 with Student Writings Grades 8, 10 and 11 This report was prepared under the direction of Barbara Wolfe Assessment Coordinator Office of Assessment and Evaluation Oregon Department of Education Public Service Building 255 Capitol Street NE Salem, Oregon 97301-0203 February 1998 3 It is the policy of the State Board of Education and a priority of the Oregon Department of Education that there will be no discrimination or harassment on the grounds of race, color, sex, marital status, religion, national origin, age or handicap in any educational programs, activities, or employment. Persons having questions about equal opportunity and nondiscrimination should contact the State Superintendent of Public Instruction at the Oregon Department of Education. Cover Page Layout and Design Credit: Deon Foulger Advanced Graphics Class North Salem High School Salem, Oregon This document was written by Bonnie Hill, Alsea School District Many thanks to the Scoring Directors and Assistant Scoring Directors for the Oregon Statewide Writing Assessment who forwarded copies of student papers for use in this report. This booklet is dedicated to Oregon's students and teachers, who work so hard and with such good spirit to ensure that all students can write effectively. Produced by the Office of Assessment and Evaluation Oregon Department of Education Salem, Oregon 97310-0203 Complimentary copies have been sent to all Oregon public schools and school districts. Additional copies are available for $8.00 each. To order, please send payment to the Publications Sales Clerk at the Oregon Department of Education, 255 Capitol St. NE, Salem, OR 97310-0203. All or any part of this document may be photocopied for educational purposes and distributed for the cost of reproduction. Any such use for other than educational purposes requires the written permission of the Oregon Department of Education. Foreword Oregon's statewide writing assessment shows the achievements of Oregon students over the last decade. It also gives us a preview of how students are likely to do as they attempt to reach the new higher performance standards adopted as part of Oregon's school improvement efforts. The writing assessment has been conducted yearly since 1991 (biennially from 1985- 1991) and the pattern of results over the years is gratifying. This document will explain the results for the 1993 1997 assessments and provide papers illustrating the work that Oregon students produce on our state test.I am pleased to point out that average scores for students have shown improvement in almost every trait from year to year. Oregon is a pioneer in developing an assessment that requires actual student writing and allows students sufficient testing time to produce and revise a fairly lengthy essay. The state is also a leader in providing information to students about their particular strengths and weaknesses on that essay by scoring the writing for six important areas, or traits, of writing. Oregon schoolsuse the summary information about their students to plan curriculum and instruction improvements. More and more Oregon teachers are making the state's Analytic Trait Writing Assessment part of their daily classroom activity. This report is useful as a summary of state level assessment results and as a staff development tool for teachers. It is a source of examples to share with students and parents in teaching writing or explaining the state test. I commend Oregon students and teachers for their fine efforts and achievements in the critically important area of writing. And I encourage them to continue and strengthen those effortsso that all students will be able to achieve the writing standards established by the State Board of Education for the Certificate of Initial Mastery and the benchmarks leading to it. Noyrna Paulus Superintendent of Public Instruction Table of Contents Chapter 1: The History of Writing Assessment in Oregon 1 Chapter 2: The Analytic Models and Highlights of Assessment Results 5 The Six-Trait Analytic Model Summary 5 Mode Summary 7 Trends in Writing Scores 1991-1997 9 Highlights of Results of the Writing Assessment, 1993-1997 11 Chapter 3: The Classroom Connection 15 The Value of the Scoring Guide 15 Using the Scoring Guide as an Instructional Tool 15 Integrating the Statewide Assessment into the Classroom (and Tips for Helping Students Do Their Best) 19 Chapter 4: Statewide Results and Sample Student Papers 21 Grades 10 and 11 22-52 Ideas and Content 22 Organization 27 Voice 32 Word Choice 36 Sentence Fluency 41 Conventions 46 Grade 8 53-75 Ideas and Content 53 Organization 57 Voice 61 Word Choice 64 Sentence Fluency 67 Conventions 71 Appendices: A. Writing Scoring Guide: Analytical Traits (including CitingSources) 78 B. Mode Scoring Guide 85 C. Sample Prompts, Grades 8, 10 and 11 91 D. Guides to Revision, Grades 8 and 10 94 E. Student Writer's Report 95 F.Requirements for Collection of Student Work Samples 96 G. Conversion Tables from Five to Six-point ScoringScale 97 H. Inter-rater Reliability: 1996 and 1997 98 I. Writing Content Panel Members 99 J.Bibliography: Writing Assessment and Instruction 101 6 Grades 8, 10 and 11 Chapter 1 The History of Writing Assessment in Oregon The Background: From Indirect to Direct Assessment Writing assessment has changed significantly during the past twenty years in Oregon and in other parts of the United States. As educators gained new insights about defining and teaching writing, they recognized the value of assessing student writing directly rather than indirectly. For decades, educators attempted to assess student writing through the use of standardized multiple choice tests. They recognized, however, the limited scope of information provided by those indirect assessments; multiple choice items could reveal students' understanding of some issues such as grammar and usage, but effective writing is characterized by qualities that multiple choice items simply cannot address. A richer, more meaningful approach emerged: students were provided with carefully designed, open-ended writing prompts and asked to write in response to those prompts; the writing that resulted was then assessed. Oregon's First Direct Assessments: 1978-1982 In 1978, Oregon students were asked for the first time to actually produce a sample of original writing to be assessed by trained professionals beyond their school districts. A representative sample of students throughout the state in grades 4, 7, and 11 was assessed; the sample was based on school size, organizational structure, and geographic location. The student writing samples were assessed holistically, meaning that each paper received a single score based on its overall effectiveness. In 1982, another sample of students in grades 4, 7, and 11 participated in a holistic assessment. Emergence of the Analytic Trait Model: Early 1980's Teachers of