Hankins IBIS Faculty Report

Hankins – IBIS Faculty Report

1. Provide an overview of your course and what you redesigned for the IBIS project

A foundation course for English and Liberal Arts majors in the literature concentration, English 258 introduces students to literary criticism and theory and builds skills of critical thinking and writing in English Studies. Emphasis is on the application of principles and methods of literary theory to selected texts, and examining and responding to texts from a variety of critical perspectives. My project targeted the following department-provided SLO: students will write well-developed, coherent essays that use critical theory to interpret a text. I assessed how online delivery of peer review writing workshops impacts students’ abilities to utilize critical theory in their written work. Students were placed in peer review groups during the first week of class. Students exchanged drafts of written work and commented on those drafts using Word review tools and the myCourses discussion board. During online class sessions, students used the myCourses discussion board to discuss their comments, the challenges of applying the assigned theory to a literary text, and strategies for meeting those challenges. These peer review sessions were organized around weekly response papers and two formal papers. Midterm course evaluations overwhelmingly stated that students were not finding the online sessions helpful. Following this evaluation I modified the course so that students continued to post drafts and comments online, but the discussion of those drafts took place in face to face sessions.

2. Explain the data you collected

A – Surveys I administered a survey at the beginning of the semester evaluating student attitudes toward peer review. At the end of the semester my mentor administered a survey evaluating student attitudes toward the online peer review sessions. See Appendix 1 and Appendix 2 for quantifiable survey results.

B – Grades I also compared grades for online workshops to grades for written work. See Appendix 3 for relevant grade averages.

C – Control Group I was also able to compare grades for written work from a previous semester that did not incorporate online sessions.

3. Provide a data analysis and your conclusions about the course experience. The central question is did the blend positively or negatively or not at all affect student learning? This is the core of your report.

A – Surveys The final student survey showed that students felt they received accurate feedback from their peers during online peer review, that face-to-face sessions prepared them for the online sessions, that instructor evaluation of online sessions had a positive impact on Hankins – IBIS Faculty Report the effectiveness of online sessions, and that online sessions increased their understanding of instructor expectations and assignment guidelines. Despite these positive effects, students did not find the workshops to improve their ability to write an essay using critical theory to interpret a text. The numbers were split on whether the review sessions improved the group’s ability to write essays using critical theory. Most students felt the myCourses platform was an impediment to learning— students overwhelmingly found the myCourses discussion board to be an ineffective tool for practicing the written application of critical theory. The chat room feature also proved ineffective as most students were unable to use it. These findings also reflect the information gathered by my mentor in her final evaluation of the course. The qualitative data I collected in the surveys also suggests that although students did not believe the online workshops improved their learning, they did find some benefit in receiving written comments from peers online. For example, one student stated “I preferred online because I think the comments were thought out instead of thought of on the fly.” Another student observed that “it was easier to formulate my thoughts online and I felt we never got off topic, like you can when face-to-face. You also have a record of the discussion.” The same student noted “people who have a difficult time getting their thoughts out in person or get nervous/shy get more involved online and its really helpful for them, they can get ideas across without pressure.” Another student noted that online sessions allowed for more emphasis on “information relating to improving understanding of [applying] theory to text.” This, of course, was one of my primary goals for the online sessions. These comments summarize the comments of about 7 of the surveys; for these students, online sessions allowed for more focused discussion, more articulate feedback on drafts, and equal participation. The negative comments emphasized the limitations of the myCourses platform and expressed general dislike of online conversations. The new blend I instituted midsemester seems to have addressed these concerns, as many students noted that the online comments combined with face-to-face discussions were the most helpful parts of the peer review system. One challenge I did not successfully address was expressed by students who were unhappy with their peer group’s participation. The success of these review groups depends on each member committing to the process. Although online work accounted for 15% of the final grade, clearly I need to implement policies that more effectively guarantee a positive experience for every committed student.

B – Grades Despite the students’ assessment that online peer review sessions did not improve their ability to use critical theory to interpret a text, there might be a correspondence between grades for online sessions and written work. The four lowest averages for online work (C- or below) corresponded to the four lowest scores for written work (B- or below). Students who received at least a C+ for online work received at least a B for written work.

C – Control Group Comparing this semester’s grades to the previous semester’s grades shows some improvement, especially in response paper averages. The response paper average for the Hankins – IBIS Faculty Report semester without online review sessions was 2.946; with online sessions the response paper average was 3.353. Formal paper grades showed a slight improvement. In the previous semester the average for the midterm paper was 3.0 and the average for the final paper was 2.873. In the blended semester the average for the midterm paper was 3.116 and the average for the final paper was 2.889. These improvements, however, are likely a result of my own increased ability to write more precise assignments and grading rubrics. Additionally, these improvements might simply reflect the added peer review component, not necessarily the online component. One number that I do think is a direct reflection of the online sessions is that in the previous semester, of the 17 students who completed the course, 5 students failed to complete the 4 required response papers. This semester, of the 19 students who completed the course, not a single student failed to complete all 5 required response papers. Although students were not convinced of the value of the online sessions, as an instructor, I witnessed a tremendous improvement in the responsibility students took for their work when asked to participate in online group workshops.

Conclusions Given these results, I’ve concluded that the online peer review workshops may have increased student learning, but there is not any significant quantitative data that convincingly reflects this speculation. In the future, I will continue to ask students to post drafts and comments online, but I will reserve online sessions for discussions focused on specific key content questions rather than writing assessment. I’d also like to note that I noticed a negative attitude toward online work from the ‘millennials’ when I introduced the grant project at the beginning of the semester. Some students never overcame this resistance. Some of these negative attitudes are directly linked to the positive benefits of online work that instructors observe—online peer review is more demanding (thus the resistance) than face-to-face peer review because students are forced to precisely articulate their thoughts in writing, the online record preserves their level of investment in each review session, and even the most reticent or disengaged students must participate in order to receive credit. Without discounting students’ assessment of their experience, I do continue to believe that instructors are able to observe benefits that students might not see. I need to do a better job of convincing students of these benefits.

4. Provide a section outlining your peer mentor experience. When did you meet? What guidelines did you set up? What worked well? What didn't work well?

Professor Anupama Arora and I discussed my plan over the summer via email and telephone. She also provided me with sample assessment materials. Professor Arora observed the myCourses site and administered the final survey to students at the end of the semester. She also asked students to give her their reflections of the course. Hankins – IBIS Faculty Report

Appendix 1 – Survey Administered 7 September 2012

1) On a scale of 1-5 (1 being the lowest) indicate how confident you feel in your ability to write an essay that uses critical theory to interpret at text. Average – 3.7857 2) On a scale of 1-5, indicate how confident you feel that peer review will improve your ability to write an essay that uses critical theory to interpret a text. Average - 3.666667 3) On a scale of 1-5, indicate how helpful you have found peer review writing workshops in the past. Average - 3.381 4) On a scale of 1-5, indicate how confident you feel in the feedback you receive from your peers during writing workshops. Average - 3.190476 5) On a scale of 1-5, indicate how willing you are to significantly revise your writing based on feedback from a peer. Average - 3.52381 6) How much time do you usually spend reading and commenting on a peer’s draft? (Assume the draft is about 5 pages long.) a. less than 30 minutes 4 b. 30-60 minutes 13 c. 1-2 hours 4 d. more than 2 hours 0 7) How would you describe the drafts you usually submit for peer review? a. very rough (argument undetermined, prose deteriorates into notes, less than half the required length, no research incorporated) 0 b. rough (argument attempted but vague and unfocused, fully in prose but supporting paragraphs not fully developed, no clear transitions between ideas, research inserted but not properly cited) 10 c. complete (argument is clearly stated, all supporting paragraphs are fully developed, clear transitions between ideas, research properly cited) 10 Hankins – IBIS Faculty Report

8) What do you find most helpful about peer review? 9) What do you find least helpful about peer review? Appendix 2 –Survey Administered by Mentor on 10 December 2012 1) Online peer review workshops increased my understanding of instructor expectations and assignment guidelines. Strongly agree 1 Agree 10 Disagree 5 Strongly disagree 2) Online peer review workshops improved my ability to write an essay that uses critical theory to interpret a text. Strongly agree 1 Agree 5 Disagree 10 Strongly disagree 3) Online peer review workshops improved my peer review group’s ability to write essays that use critical theory to interpret a text. Strongly agree 1 Agree 8 Disagree 7 Strongly disagree 4) Face-to-face class sessions provided the preparation I needed to offer my peers accurate feedback on their written applications of critical theory. Strongly agree 7 Agree 8 Disagree Strongly disagree 1 5) The myCourses Discussion Board was an effective tool for peer review. Strongly agree Agree 4 Disagree 11 Strongly disagree 1 6) The myCourses Discussion Board was an effective tool for practicing the written application of critical theory. Strongly agree Agree 7 Disagree 8 Strongly disagree 7) I received accurate feedback from my peers during online peer review. Strongly agree 3 Agree 12 Disagree Strongly disagree 1 8) The instructor evaluation of online peer review sessions had a positive impact on the effectiveness of those sessions. Strongly agree 1 Agree 11 Disagree 4 Strongly disagree 9) How have your attitudes toward peer review work changed over the course of the semester? 10) What was useful or not useful about the online peer review system? 11) How did your experience with online peer review sessions this semester compare to your experience with face-to-face peer review sessions in the past? Hankins – IBIS Faculty Report

12) What could you and your group members have done differently to increase the effectiveness of online peer review sessions? 13) What could the instructor have done differently to increase the effectiveness of online peer review sessions? 14) Further comments are welcome below: Appendix 3 – Relevant Grade Averages

Response paper Midterm Average for Average for average (5 total) paper Final paper written work online work 3.3 3.7 3.7 3.566666667 4 3 3 3.3 3.1 4 3.3 3 3 3.1 4 3.7 4 3.3 3.666666667 4 4 4 3 3.666666667 4 2.7 3.7 2.3 2.9 3.7 3.3 3 3 3.1 3.7 3.3 3.3 2 2.866666667 3.7 3.7 3 3.3 3.333333333 3.7 4 3.3 3.7 3.666666667 3.7 4 3.3 3 3.433333333 3.7 3.7 3 3.3 3.333333333 3.7 3.7 3.3 3.7 3.566666667 3.3 3.3 3.7 3 3.333333333 2.7 3.3 3.3 2.3 2.966666667 2.3 2.7 2.7 2.7 2.7 1.7 3.3 2.3 2 2.533333333 1.7 2.7 2.3 2.3 2.433333333 1.7 2.7 1.3 2 2 1.3 3.352631579 3.115789474 2.88947368