NCEE 2012-4010 U. S. DEPARTMENT OF EDUCATION An Investigation of the Impact of the 6+1 Trait Writing Model on Grade 5 Student Writing Achievement An Investigation of the Impact of the 6+1 Trait Writing Model on Grade 5 Student Writing Achievement Final Report December 2011 Authors: Michael Coe Cedar Lake Research Group Makoto Hanita Education Northwest Vicki Nishioka Education Northwest Richard Smiley Education Northwest Project Officer: Ok-Choon Park Institute of Education Sciences NCEE 2012–4010 U.S. Department of Education U.S. Department of Education Arne Duncan Secretary Institute of Education Sciences John Q. Easton Director National Center for Education Evaluation and Regional Assistance Rebecca A. Maynard Commissioner December 2011 This report was prepared for the National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, under contract ED-06-CO-0016 with Regional Educational Laboratory Northwest administered by Education Northwest. IES evaluation reports present objective information on the conditions of implementation and impacts of the programs being evaluated. IES evaluation reports do not include conclusions or recommendations or views with regard to actions policymakers or practitioners should take in light of the findings in the report. This report is in the public domain. Authorization to reproduce it in whole or in part is granted. While permission to reprint this publication is not necessary, the citation should read: Coe, Michael, Hanita, Makoto, Nishioka, Vicki, and Smiley, Richard. (2011). An investigation of the impact of the 6+1 Trait Writing model on grade 5 student writing achievement (NCEE 2012–4010). Washington, DC: National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, U.S. Department of Education. This report is available on the Institute of Education Sciences website at http://ncee.ed.gov and the Regional Educational Laboratory Program website at http://edlabs.ed.gov. Alternate Formats Upon request, this report is available in alternate formats, such as Braille, large print, audiotape, or computer diskette. For more information, please contact the Department’s Alternate Format Center at 202-260-9895 or 202-205-8113. Disclosure of potential conflict of interest The design of this study along with all data collection, data analysis, and report writing were conducted by the Research and Evaluation Program of Education Northwest and by Cedar Lake Research Group, a subcontractor. None of the members of the research team have financial interests that could be affected by the content of this report. Professional development for teachers in the study was provided by the 6+1 Trait Writing Unit at Education Northwest; this unit does have financial interest that could be affected by the study but was not involved in the research activities. Contractors carrying out research and evaluation projects for IES frequently need to obtain expert advice and technical assistance from individuals and entities whose other professional work may not be entirely independent of or separable from the tasks they are carrying out for the IES contractor. Contractors endeavor not to put such individuals or entities in positions in which they could bias the analysis and reporting of results, and their potential conflicts of interest are disclosed. iii Acknowledgements The authors would like to thank everyone who helped make this study possible, including the teachers, students, and administrators in participating schools; the technical advisors, reviewers, and editors who assisted us throughout the study; staff members at the U.S. Department of Education who were associated with the Regional Educational Laboratory Northwest during this contract; the Oregon Department of Education; and staff members at Education Northwest who contributed to the project. Mike Puma and Dave Connell of Chesapeake Research Associates were invaluable advisers during all phases of the project, and provided a data management system that kept the research team blind to the origin of student essays during scoring. Andrew Jaciw of Empirical Education provided very helpful consultation on statistical issues as well as independent verification of the consistency and integrity of the data during the analytic process. Cris Price of Abt Associates helped resolve issues with the multiple imputation of missing data during the analysis. Carol Pistorino of Decision Information Resources, Inc. and her colleagues on the Analytical and Technical Support team, administered by Mathematica Policy Research, Inc., provided sound advice throughout the project. Peter Schochet of Mathematica was particularly helpful in resolving statistical modeling issues. Steve Slater at the Oregon Department of Education helped determine input parameters for the study design, based on Oregon statewide assessment data. The REL Northwest Technical Working Group provided insight and guidance along the way. This group included Ray Barnhardt, Hans Bos, Audrey Champagne, Dave Connell, Bill Demmert, Allen Glenn, Dan Goldhaber, Brian Gong, Joan Herman, Michael Kamil, LeAnne Robinson, Lynn Santelmann, and Sam Stringfield. The research team enjoyed the thorough and able support of Linda Fitch, Jennifer Klump, and Lisa Todd for information services; Matt Lewis for database management; Kathy Petersen, Tess Bridgman and Ann Rader for administrative support; and Bracken Reed and Nicole Sage for technical editing and communications. Management oversight and guidance was provided by Terri Akey, Bob Rayborn, Kim Yap, Steve Nelson, and Steve Fleischman. Art Burke and Richard Greenough provided statistical and technical advice. The student essays were scored by Linda Beith, Louise Brame, Adriane Burkhart, Dorthea Dober, Kerby Gault, Joan Hartley, Glenda Montgomery, Roger Rossman, Connie Spiegel, Ed Stites, and Marilyn Young, who were guided and coordinated by Sandra Marek. Professional development services for teachers who participated in the study were provided by Jan Littlebear and Peter Bellamy with support from Mark Workman and Kay Dameron; Jan and Peter also provided much helpful input for our descriptions of the history, development and use of the 6+1 Trait Writing model. Gary Nave also provided assistance related to the description and implementation of the professional development. Ruth Culham of the Culham Writing Company provided historical insight into the development and key features of the 6+1 Trait Writing model. iv Contents DISCLOSURE OF POTENTIAL CONFLICT OF INTEREST ..............................................................................III ACKNOWLEDGEMENTS ........................................................................................................................ IV SUMMARY ........................................................................................................................................... IX STUDY SAMPLE AND METHODS ........................................................................................................................... X SUMMARY OF FINDINGS .................................................................................................................................. XII LIMITATIONS ................................................................................................................................................ XIII 1. STUDY BACKGROUND ........................................................................................................................ 1 NEED FOR THE STUDY ...................................................................................................................................... 2 AN OVERVIEW OF 6+1 TRAIT WRITING ............................................................................................................... 4 PREVIOUS RESEARCH ON 6+1 TRAIT WRITING ...................................................................................................... 9 NEED FOR EXPERIMENTAL EVIDENCE ON 6+1 TRAIT WRITING................................................................................ 10 RESEARCH QUESTIONS ................................................................................................................................... 10 2. STUDY DESIGN AND METHODOLOGY ............................................................................................... 13 A MULTISITE CLUSTER RANDOMIZED TRIAL ......................................................................................................... 13 STUDY TIMELINE ........................................................................................................................................... 14 TARGET SAMPLE SIZE, POPULATION, AND RECRUITMENT METHODS ......................................................................... 15 INCENTIVES TO PARTICIPATE IN THE STUDY ......................................................................................................... 18 RANDOM ASSIGNMENT, STUDY PARTICIPANTS, AND ATTRITION .............................................................................. 19 BASELINE EQUIVALENCE OF TREATMENT AND CONTROL GROUPS............................................................................. 23 DATA COLLECTION INSTRUMENTS AND PROCEDURES ............................................................................................ 26 OUTCOME MEASURES ...................................................................................................................................