The Effects of Web-Based Peer Review on Student Writing
Total Page:16
File Type:pdf, Size:1020Kb
THE EFFECTS OF WEB-BASED PEER REVIEW ON STUDENT WRITING A dissertation submitted to the College of Education, Health and Human Services in partial fulfillment of the requirements of the degree of Doctor of Philosophy Kent State University by Ryan S. Wooley December 2007 Copyright © 2007 by Ryan S. Wooley All Rights Reserved ii A dissertation written by Ryan S. Wooley B.A., University of Akron, 1995 M.A.T., Kent State University, 1997 M.Ed., Kent State University, 2004 Ph.D., Kent State University, 2007 Approved by _______________________________, Director, Doctoral Dissertation Committee David W. Dalton _______________________________, Co-director, Doctoral Dissertation Committee Christopher A. Was _______________________________, Members, Doctoral Dissertation Committee Shawn M. Fitzgerald _______________________________, Christian D. Schunn Accepted by _______________________________, Chairperson, Department of Educational Awilda C. Hamilton Foundations and Special Services _______________________________, Dean, College of Education, Health and Donald L. Bubenzer Human Services iii WOOLEY, RYAN S., Ph.D., December 2007 Educational Foundations, and Special Services THE EFFECTS OF WEB-BASED PEER REVIEW ON STUDENT WRITING (206 pp.) Co-Directors of Dissertation: David W. Dalton, Ph.D. Christopher A. Was, Ph. D. Some theorists consider writing and cognition to be symbiotic if not synonymous. Writing is regarded not only a means of communicating and assessing understanding of content knowledge, but as a way of constructing knowledge. Yet, though writing is likely a beneficial activity in most, if not all, disciplines, it has been difficult to implement in content courses. Robust online peer review systems for student writing now offer solutions to many of the problems that have impeded peer review activities in the past. Research in self-explanations and reciprocal teaching has suggested that students stand to benefit cognitively by articulating explanations to self and others, but this research has been conducted primarily in math and science domains. There have been few, if any, investigations of the effects of articulating feedback for others on one’s own subsequent writing. The goal of this study was to examine the effects of reviewing on one’s subsequent writing. Further, the study sought to illuminate distinctions between different types of reviewing and reviewer preparation, namely the effects of feedback elaboration and the effects of providing prototypical examples of helpful and unhelpful feedback. Results indicate that students who provided elaborate forms of feedback, which included free-form comments, performed significantly better on their own writing than students who provided numerical ratings only. In this context, the use of examples did iv not have significant effects on reviewers’ subsequent writing quality. Also, review-first groups did not perform significantly better than write-first groups, however, the author notes that the design of the study may have inadvertently counteracted reviewing effects. v ACKNOWLEDGEMENTS I would like to thank several individuals who made substantial contributions toward the completion of this research project. First, I would like to extend my sincere appreciation to my advisor, Dave Dalton, who has been a remarkable mentor to me academically and professionally, and for his patient guidance through this dissertation process. I would also like to thank my other committee members—Dr. Chris Was, Dr. Shawn Fitzgerald, and Dr. Christian Schunn—for their generous investments of time, energy and expertise. Dr. Schunn deserves additional thanks for designing the SWoRD system, the sine qua non of this dissertation, and for reaching across institutional and geographic boundaries to participate in this project. Second, I would like to thank Dr. Kwangsu Cho, who helped to create and administer the SWoRD system. During this project, he graciously fielded the increased demands created by treatment conditions and other idiosyncrasies of the study design. I would also like to thank each of the instructors who agreed to participate in the project: Dr. Anne Morrison, Dr. Larry Froehlich, Mary Louise Allen-Huffman, and Tom Foster. These instructors were willing to take the chance of engaging in a novel form of pedagogy, knowing there would be resistance from their students. Finally, I would like to thank my wife Christina for her immense patience and support through this process. vi TABLE OF CONTENTS Page ABSTRACT...............................................................................................................iv ACKNOWLEDGEMENTS.......................................................................................vi LIST OF TABLES.....................................................................................................x CHAPTER I INTRODUCTION...............................................................................1 Background........................................................................................1 Rationale ............................................................................................8 Purpose of Study................................................................................9 Research Questions and Related Hypotheses Phase I....................................................................................10 Phase II...................................................................................10 Phase I....................................................................................10 Phase II...................................................................................10 Limitations of Study..........................................................................11 Delimitations of Study .......................................................................11 Operational Definitions......................................................................12 II REVIEW OF THE LITERATURE ...................................................14 Writing Curricula and Research.........................................................15 Articulation ........................................................................................24 Teach to Learn ...................................................................................35 Measuring Writing Quality................................................................42 Peer Review.......................................................................................50 Feedback ............................................................................................56 Models and Modeling........................................................................62 Conclusion .........................................................................................67 vii III METHODS ........................................................................................71 Phase I ................................................................................................71 Phase II...............................................................................................71 Participants.........................................................................................71 Apparatus ...........................................................................................72 Instrumentation ..................................................................................73 Procedures..........................................................................................78 Treatments..............................................................................79 Write First..................................................................80 Review First, With Examples, With Elaboration.......81 Review First, With Examples, Without Elaboration .81 Review First, Without Examples, With Elaboration.81 Review First, Without Examples, Without Elaboration.....................................................82 Experimental Design and Data Analysis ...........................................82 Research Hypotheses .............................................................82 Phase I........................................................................82 Phase II.......................................................................82 Design ....................................................................................82 Analyses.................................................................................84 IV RESULTS ..........................................................................................87 Overview............................................................................................87 Phase I—One Factor ANOVA...............................................87 Phase II—Factorial ANOVA.................................................88 Preliminary Analyses.................................................88 Hypothesis 1...............................................................89 Hypothesis 2...............................................................90 Hypothesis 3...............................................................90 Summary............................................................................................92 V DISCUSSION....................................................................................93 The Role of Context...........................................................................94 Expected Results................................................................................100 Unexpected