DOCUMENT RESUME ED 332 223 CS 212 855 TITLE Alaska Writing Assessment Pilot Survey 1988-89. INSTITUTION Alaska State Dept. of Education, Juneau. PUb DATE 89 NOTE 106p. PUB TYPE Reports - Research/Technical (143) EDRS PRICE MF01/PC05 Plus Postage. DESCRIPTORS Correlation; Grade 10; High Schools; Holistic Evaluation; Public Schools; *Student Evaluation; Writing Ability; *Writing Evaluation; Writing Research IDENTIFIERS *Alaska; Analytical Scoring; Primary Trait Scoring ABSTRACT A study investigated what can be learned about the strengths and weaknesses of students' writing from direct assessment of writing performance and knowledge of overall student performance. In the first part of the study, all 13 school districts in Alaska participated in an interdistrict writing assessment procedure intended to measure the overall strengths and weaknesses of tenth graders' writing skills. Nine school districts participated in the second part of the study which correlated 668 tenth graders' performance in the direct writing assessment to their performance on standardized tests and to their grade point average. The subjects responded to a writing prompt concerning friendship and were given two 50-minute periods to write and revise their responses. The strengths and weaknesses of the subjects' writing abilities were analyzed for each of six writing traits: ideas and content; organization; voice; word cho:i.ce; sentence structure; and conventions. Results indicated that: (1) the relationships among the six writing traits were higher than the relationship with the standardized test result; (2) the strongest associations between writing traits and test scores occur with "sentence structure" and "conventions"; and (3) the weakest association between writing traits and test scoras occurs with "ideas and content" and "voice." Findings suggest that while the traits are strongly interrelated, they are in fact measuring different components of writing. (Sixteen tables of data are included; appendixes include a list of participating districts, the agenda for early training sessions, definitions of scoring methods, the analytical scoring guide, department of education letters and instruction, participating teachers' comments, contingency tables, and interdistrict writing assessment statistics.) (RS) *********************************************************************** Reproductions supplied by EDRS are the best that can be made from the original document. *********************************************************************** ALASKA WRITING ASSESSMENTPILOT SURVEY 1988-89 BEST COPY AVAILABLE U DEPARTMENT OF EDUCATION Office of Educational Research and lmorOvemenl EDUCATIONAL RESOURCES INFORMATION "PERMISSION TO REPRODUCE THIS Naska CENTER IERIC1 MATERIAL HAS BEEN GRANTED BY r4 This document hat been reproduced as Departrnent of received from the person or organization orqrnahno it OA 41111Education C Minor changes have Peen made to improve reproduction duality Ponta ot vew 0 OCemOns staled in this docu ment do nol necessarily represent official OE RI position or 000 TO THE EDUCATIONAL RESOURCES 4 4 INFORMATION CENTER (ERIC).- Steve Cowper GOVERNOR GDemmert: COMMISSIONElkOft:spuomON.. ... .'.. .... Toni Kaliklen4ones DIRECTOR, EDUCATIONAL PROGRAMS . Alanka.Staisiligardsadwatiga Barney Gottstein PRESIDENT Don Gray FIRST VICE PRESIDENT Janie Leask SECOND VICE PRESIDENT Jack Chenoweth Catherine Bishop Herrnsteen Cora Sakeagak Sue Wilken Lt. Col. Matthew P. Kenney Raychelle Daniel TABLE OF CONTENTS I Alaska's 1989 Direct Writing Assessment An Introduction 1 v Chapter One: A Brief History of Writing Assessment in Alaska 3 Chapter Two: The Analytical Assessment Model 5 Chapter Three: Assessment and Scoring Procedures 7 Chapter Four: The Correlational Study 10 Results and Analysis Chapter Five: Strengths and Weaknesses: A Trait by Trait Analysis 18 Ideas and Content 20 Organization 27 Voice 35 Word Choice 44 Sentence Structure 52 Writing Conventions 57 Appendix A Participating Districts and Contacts 67 Appendix B Agenda of Early Training Session 69 Appendix C Definitions of Scoring Methods 72 Appendix D Analytical Scoring Guide 75 Appendix E Department of Education Letters and Instructions 82 Appendix F Participating Teachers' Comments 88 Appendix G Contingency Tables 90 Appendix H Interdistrict Writing Assessment Statistics 98 . PILOT PROJECT STEERING COMMITTEE Judy Arter, Northwest Regional Educational Laboratory Mary Birkeland; University of Alaska Fairbanks Phil Brady, University of Alaska Fairbanks Annie Calkins, Department of Education Marlys Henderson, Fairbanks North Star Borough Bob Silverman, Department of Education Nicholas Stayrook, Fairbanks North Star Borough James Villano, Fairbanks North Star Borough PILOT PARTICIPANTS: Gayle Hammons, Sitka Borough Schools Pat Hunt, Ketchikan Gateway Borough Schools Patti Irwin, Northwest Arctic Borough Schools Lois Jones, Kenai Peninsula Borough Schools Hilarie Meadows, Valdez City Schools Carol Merritt, Fairbanks North Star Sorough Schools Deane O'Dell, Iditarod Area Schools Sandra Parker, Anchorage School District Bambi Phipps, Kuspuk School District Mike Sirofchuck, Kodiak Island Borough Schools Joe I. Slats, Lower Kuskokwim School District Arlie Swett, Railbelt School District Alaska's 1989 Direct writing Assessment An Introduction Nationwide, many states are augmenting their standardized assessments of students' basic skills in language arts with direct assessments of actual writing samples. Most often, the argument for use of a writing sample is that it provides educators with a more concrete picture of actual writing skills. That is to say, rather than having students respond in a multiple choice manner to questions about writing, we ask students to write. This method of assessment provides students the opportunity to demonstrate their skillin effective use of grammatical and other writing conventions, their organizational and persuasive powers in writing, and their fluency of expression. Furthermore, the results of direct assessment are broadly useful. They can be used for program evaluation by either validating existing program successes or by pointing to the need for curricular change. Results can be used by teachers in targeting specific instructional methods and student activities in areas where there are weaknesses. In addition, students get direct feedback regarding the strengths and/or weaknesses of their actual writing performance. New Jersey, Washington, Illinois, California, and Oregon are among the many states who have begUn to augment their traditional assessmentsof language arts with direct assessment of student writing samples. They have used a variety of writing assessment scoring procedures including holistic, primary trait, and analytical methods. Some states have conducted their writing assessment in conjunction with standardized tests of language ability while others have kept their writing assessment separate from standardized tests. Some states assess writing abilities of high school students only %..ihereas others also include elementary and middle school students. A common purpose of these assessment efforts, however, is to describe the condition of students' writing skills and to begin to monitor skill development over time. In April 1989, the Alaska Department of Education conducted its first interdistrict direct assessment of tenth graders' writing skills. The purposes for this assessment were: 1) To determine the feasibility of conducting a statewide interdistrict writing assessment; 2) To gather direct writing assessment results for participating districts; 3) To investigate the relationship between direct writing performance and standardized test performance; and 4) To gauge the interest and acceptability of doing writing assessments among Alaskan districts. Thirteen school districts from across toe state participated in this pilot study. (Participating districts and their district contacts are 1 listed in Appendix I). Given the purposes, the pilot was divided in two parts. All school districts participated in Part One, an interdistrict writing assessment intended to measure the overall strengths and weaknesses of tenth graders' writing skills. Part Two of the study, in which nine school districts participated, correlated teAth graders' performance in the direct writing assessment to their performance on a standardized test and to their grade point average (GPA). The purpose of the correlational study was to examine the relationshlp between different assessments of writing performance and knowledge with overall student performance. A crucial question for the overall study was "What can we learn about the strengths and weaknesses of a student's writing performance f,om direct writing assessment that we may not learn otherwise?" A second policy level question also considered was "What useful planning information do we gain?" In the pilot study, 668 terih graders responded to a common writing "prompt" or topic over a two day period between March 27 and April 7, 1989. Teachers reported that students generally wrote a rough draft on the first day and revised it on the second day. The writing topic selected by contact teachers from each participating district and Department of Education specialists was: We all need.friends. Describe a time in your life (either real or imaginary) when friendship really counted. The people you describe may be real or imaginary. 8 Chapter 1: A Brief History of writing Assessment in Alaska Interest in the direct assessment of student writing