The Thirty-Ninth Annual Conference of the International Society for Exploring Teaching and Learning (ISETL) October 8-10, 2009 Philadelphia, PA Proceedings ISETL is dedicated to the study of instruction and principles of learning in order to implement practical, effective methods of teaching and learning; promote the application, development and evaluation of such methods; and foster the scholarship of teaching and learning among practicing post-secondary educators. Edited by Susan E. Copeland Clayton State University Copyright 2009 by the International Society for Exploring Teaching & Learning 2 BOARD OF DIRECTORS Angela Humphrey Brown, President, Piedmont College Peter E. Doolittle, President, Past President, Virginia Tech Susan E. Copeland, Clayton State University Jill L, Lane, Clayton State University Jerry W. Samples, University of Pittsburgh at Johnstown CORPORATE MEMBERS Diane Aschenbrenner, Johns-Hopkins University Miriam Diaz-Gilbert, University of the Sciences in Philadelphia Mominka Fileva, Davenport University Jill L. Lane, Clayton State University Mary Mattson, Georgia Perimeter College Julie Schrock, Meredith College Christina P. Shorall, Carlow University Marilyn Simon, University of Cincinnati OFFICERS Angela Humphrey Brown, President, Piedmont College Bruce Saulnier, Treasurer, Quinnipiac University Christina P. Shorall, Secretary, Carlow University DISTINGUISHED FELLOWS OF THE SOCIETY Samuel Postlethwait, Professor Emeritus, Purdue University Joseph E. J. Habowsky, Professor Emeritus, University of Windsor Charles Wales, West Virginian University Kenneth Klopfenstein, Colorado State University Jean E. Wold, California State University Donald Borchardt, Rutgers University Robert A. Stager, University of Windsor A. Jeanne Miller, University of Central Florida Eunice L. Krinsky, California State University – Dominguez Hills Anne H. Nardi, West Virginia University Bruce Saulnier, Quinnipiac University Jerry W. Samples, University of Pittsburgh – Johnstown Susan E. Copeland, Clayton State University 3 PRESIDENTS OF THE SOCIETY 1970-71 Samuel Postlethwait 1971-72 Dave Husband 1972-73 Ben Meleca 1973-74 Robert Hurst 1974-75 John Hinton 1975-76 Sally Short 1976-77 Stan Nelson 1977-78 John Zimmerman 1978-79 Mary Lynch 1979-80 George D. Brown 1980-81 Warren D. Dolphin 1981-82 Joseph E. J. Habowsky 1982-84 Charles E. Wales 1984-85 Blaine Carpenter 1985-86 Donald E. Borchardt 1986-87 Jean E. Wold 1987-88 James Marlin 1988-90 Kenneth Klopfenstein 1990-93 William J. Mullin 1993-95 Bonnie Johnson 1995-97 Kenneth Brown 1997-99 Shirley Rickert 1999-2001 George Watson 2001-03 Bruce Saulnier 2003-05 Alexander Crispo 2005-07 Susan E. Copeland 2007-09 Peter E. Doolittle 2009-11 Angela Humphrey Brown CONFERENCE COMMITTEE Sheila Wood, Registrar Angela Humphrey Brown, Proposal Review Chair Susan E. Copeland, Proceedings Editor Tammy V. Wiley, Pro-Cam Conference Planner and Liaison 4 A Comparison of Writing Strategies and Multiple Choice Tests on Text Comprehension and Retention Jane Abraham Virginia Tech Center for Excelllence in Undergraduate Teaching Blacksburg, Virginia 24061 [email protected] Thomas Sherman Virginia Tech School of Education Blacksburg, Virginia 24061 [email protected] Textbooks provide essential information for students to master their academic material in college courses. Assuming that students study/comprehend text assignments successfully, professors can plan class time to elaborate or enrich the information presented in texts. Two common approaches to promote completing text assignments in a timely manner and assessing comprehension are to require students to complete writing assignments such as summarizing based on their study and to administer tests covering the required reading material. Writing assignments are appealing to some professors because of the perceived benefits that writing is purported to have on learning. Writing promotes learning because more intellectual activity is required to write about a topic than to only read about it. For example, Newell and Winograd (1989) demonstrated that learners were able to generate more accurate gist of text when they wrote summaries than when they wrote outlines or answered study questions. As a result of research on writing activities, it may be possible to construct a hierarchy of the impact of writing activities on comprehension reading alone is less effective than reading and underlining which is less effective than reading and taking notes which is less effective than reading and writing summaries. The influence of tests on learning has been widely investigated with a consistent conclusion that classroom testing impacts student learning. Subsequent research has supported the powerful relationship between classroom testing and student learning (e.g., Butler, Karpicke, & Roediger, 2007; Crooks, 1988; Martinez, 1999) while seeking to identify the sources of the impact of classroom testing. In this study, three writing assignments (summary, outline, and personal statement) were compared with individual chapter multiple choice quizzes on the grades students achieved on a comprehensive course-end multiple choice examination covering all the chapters in the required text. One purpose of this study was to compare three types of writing to investigate possible differential effects on a course-end comprehensive multiple choice test on the required course text. A second purpose was to compare writing to test taking as strategies that promote text comprehension indicated by performance on a course-end comprehensive multiple choice test on the required course text. One notable feature of this study is that it was a component of actual classes in which grades were assigned with students who were studying the content to meet graduation and certification requirements. 5 Students enrolled in two sections of a required teacher education pre-service educational psychology course at a large mid-Atlantic research university served as the participants. The two sections of this course were randomly assigned to either writing or quiz conditions. Students were assigned to study the textbook, Educational Psychology: Developing Learners, 6th edition (Ormrod, 2008a) following a schedule that required them to submit an assignment on a chapter of the text each week of the term to receive credit for the assignment. Students were given a 45-item multiple choice pre-test drawn from the text author-supplied test item pool (Ormrod, 2008b) on the second day of class (3 items for each chapter of the 15 chapters in the text). Due to restrictions imposed by the institutional human subjects review board students were given the option of completing a 45-item multiple choice post-test covering the whole text (3 items for each chapter of the 15 chapters in the text) administered on the final day of class for the term. The post-test was also drawn from the author supplied test bank. Thirty-three of 39 students completed the post-test administered on the last day of class. The post-test grades were not included in calculating students’ course grades. Students in the morning class (N=18) were required to submit one of three writing assignments for each chapter in the text: outline, summary, or personal meaning. One of the three writing paper assignments was randomly applied to five of the 15 chapters of the required text. Thus, students wrote five summaries, five outlines, and five personal meaning papers. Instructions on how to write each of these papers and examples were given for each of the writing to learn assignments Students in the afternoon class (N=21) were required to complete an online 15-item multiple choice quiz for each of the 15 chapters in the textbook; the online program through which the tests were delivered was a commercial course management program (Blackboard). All items in these chapter quizzes were drawn from the item pool supplied by the text author (Ormrod, 2008b). The mean of the pretest scores of the writing section (N=15) was 25.8667 (SD=3.7391); the mean of the pretest scores of the multiple choice quiz section (N=16) was 25.6875 (SD=4.5125). A one-way ANOVA indicated no significant difference between groups (p=.9054). Thus, both classes began with the same prior knowledge about the content of the text The mean of the post-test scores of the writing section (N=16) was 32.1875 (SD=3.7727); the mean of the post-test scores of the multiple choice quiz section (N=17) was 31.2941 (SD=5.8818). A one-way ANOVA indicated no significant difference between groups (p=.6086). Thus, both classes completed the course with essentially the same understanding of the text. To investigate the extent to which students learned the material in the text, a t-test was conducted comparing pretest and post-test scores. The mean of pretest scores was 25.7742 (N=31) and of post-test scores was 31.6129 (N=31). This was significantly different (p=.000004). Thus, students demonstrated increased knowledge of the text material from the beginning to the end of the course. To examine the possibility of differential effects of the writing assignments on the quality of students’ performance in doing the writing, an ANOVA was applied to the grades students received on each type of writing assignment. The mean grade of summary papers (N=16) was 61.2650 (SD=6.3757); the mean grade of outlines (N=16) was 68.8750 (SD=4.3186); and the mean grade of personal learning papers (N=16) was 70.6875 (SD=2.7256). There was a significant difference between groups (p=.000004). Further t-tests between groups indicated a