Memory Recall Exercises

Food for Thought: Memory Recall Exercises

Dale Hommerding, George Fust, Sean McMahon, Cole Spitzack, and Michael Wright

This paper was completed and submitted in partial fulfillment of the Master Teacher Program, a 2-year faculty professional development program conducted by the Center for Faculty Excellence, United States Military Academy, West Point, NY, 2020.

Abstract Instructors constantly seek to improve their students’ retention of learned knowledge through a variety of pedagogical techniques. Existing scholarship suggests that incorporating memory recall exercises into the course of instruction may help improve long term student retention of information by “interrupting forgetting.” Our study seeks to expand upon previous studies to the undergraduate level. It seeks to determine an empirical correlation between performance and retrieval exercises. This study was primarily motivated by the authors’ participation in the Master Teacher Program at the United States Military Academy at West Point. It involved 10 different instructors teaching 370 students over the course of a semester. Our research yielded no significant findings to support the premise that interrupting forgetting improves student performance in a classroom environment using memory recall exercises. Furthermore, our research indicates memory recall exercises may have a negative effect on performance by unintentionally contaminating student understanding of course material. We recommend instructors consider alternative methods for improving student retention.

Food for thought: memory recall exercises 2

Introduction

A perennial challenge for students at any level is the retention, recall, and application of knowledge. One method consists of memory recall exercises, which involve short in-class writing exercises designed to activate students’ memory of previously covered course material into the course of instruction. Since improved knowledge retention is a goal that any teacher seeks to achieve, testing the validity of memory recall exercises can provide valuable insights to teachers in any discipline.

Concepts discussed in the U.S. Military Academy’s Master Teacher Program provided the inspiration for this study. In particular, two chapters from Peter C. Brown, Henry L.

Roediger and Mark McDaniel’s Make it Stick1 provided our theoretical foundation. The chapter titled “To Learn, Retrieve” discusses positive aspects of repeated testing on students’ ability to retrieve information. Brown, Roediger, and McDaniel highlight how “positive effects of repeated testing has been demonstrated many times in controlled laboratory settings, but rarely in a regular classroom setting.”1 Furthermore, the Make it Stick chapter titled “Embrace

Difficulties” discusses positive benefits associated with difficult learning methods on subjects’ ability to retrieve information at a later date.2

Our study sought to test the validity of these concepts. Over the course of an academic semester, we conducted an experiment to test the effects of memory recall exercises on student retention. Throughout the semester, we incorporated several short memory recall exercises as part of a college-level sophomore American politics survey course and measured the effects these exercises had on student long-term retention, as measured by their performance on the final exam. In addition to testing the effects of such exercises on long-term retention, we also sought

Food for thought: memory recall exercises 3 to test Brown and Roediger’s theories regarding the consequences of repeated testing and the effects of “desirable” difficulties on student performance.3

We found no evidence that our exercises had a positive effect on student knowledge retention. In fact, our results indicate that, to the degree that such exercises have an effect, that effect is negative. To our knowledge, this is the first experimental evidence that tests the effects of memory recall exercises at the college level, and our findings suggest that such exercises may not be an effective tool for improving long-term knowledge retention.

Literature Review

The idea that there are benefits to testing memory is not new. Sir Francis Bacon’s recognition in 1620 that “if you read a piece of text through twenty times, you will not learn it by heart so easily as if you read it ten times while attempting to recite from time to time and consulting the text when your memory fails.”4 Modern researchers have devoted significant attention to assessing techniques meant to improve students’ ability to recall information.

Experiments have tested retention through direct methods such as recall and recognition exercises.5 In free-recall tests, individuals are asked to recall previously presented information, such as recalling a list of words or objects from memory.6 Recognition tests typically provide a list of possible choices for subjects to reference. One common area of focus when studying retention is the testing effect, otherwise known as the retrieval-practice effect, which states that

“practicing retrieval makes learning stick far better than re-exposure to the original material.”7

While the ideas behind testing memory to improve recollection dates back centuries, precise methods for improving memory through repeated testing are often either narrowly focused or yield inconclusive results.

Food for thought: memory recall exercises 4

Multiple scholars have conducted experiments related to the testing effect with mixed results. Phillip Ballard pioneered experiments in 1913 showing an increase in children’s ability to recall poetry passages through testing. Ballard concluded that testing yields positive effects such as reminiscence, where individuals recall material on future tests that was not recalled on previous tests.8 In 1974, Erdelyi and Becker expanded on Ballard’s research by testing subjects’ ability to recall large sets of words or pictures. Their experiments provided evidence of memory improvements which they labeled as hypermnesia, where individuals recalled material on subsequent tests that they could not recall on previous tests.9 Research by Henry Roediger and

Jeffrey Karpicke suggest the process of test-taking yields greater positive results than an equivalent time spent studying “even when performance on the test is far from perfect and no feedback is given on missed information.”10 Collectively, prior research provides a foundation for the positive correlations between increased testing and student performance.

However, separate experiments highlight drastically different conclusions and suggest potentially negative impacts from repeated testing. Frederick C. Bartlett’s experiments reveal an increase in forgetting and information distortion in subsequent tests after subjects learned prose passages and were tested both immediately following (15 minutes after studying) and later

(within a week of learning the passage).11 Bartlett noticed that student performance declined over time as his research subjects forgot portions of the passages, but often re-created details of the story based on their own perceptions.12 This suggests an aspect of contamination resulting from inaccurate information learned during previous tests.13 James Deese also cautions against the use of repeated testing due to information contamination. Deese notes that repeated testing

“yields very impure measures of retention after the first test, since all subsequent measures are contaminated by the practice the first test allows.”14 As opposed to previous conclusions

Food for thought: memory recall exercises 5 highlighting testing’s ability to facilitate learning through reminiscence and hypermnesia, Deese highlights negative aspects of learning through testing due to contamination. Research highlighting reduced performance from repeated testing combined with the contamination effect contributes to a broader area of study into forgetting.

In addition to contamination, other significant theories related to memory recall have been posited. Stuart Hulse and Howard Egeth discuss interference theory which is significant to the study of forgetting. Interference theory examines the relationship between a specific stimulus and a specific response (S-R). Hulse and Egeth suggest that providing more frequent stimuli, mostly through increased testing, may have either positive or negative effects on student performance.15 Evidence suggests that once a specific “S-R connection is established, it is not weakened by the passage of time; however, S-R associations interact with one another, producing behavioral changes that may be either facilitatory or inhibitory.”16 Additionally, a

2012 study by P. K. Agarwal, et. al. offers compelling evidence that interference theory is at play in middle school settings.17 This 5-year study provides the benchmark for our experiment at the undergraduate level. Thus, by creating additional stimuli through testing, one can expect to see a range of either positive or negative responses. For example, while testing could potentially provide a facilitatory benefit through hypermnesia, further stimulus-response interactions may inhibit recall by introducing information contamination. Accordingly, interference theory provides a framework for future experiments to examine the correlation between testing and retention.

Even though significant prior research has been undertaken in examining impacts of the testing effect, the preponderance of tests focused on simple free-recall tests, such as recalling details from a prose passage, poetry verse, or list of names or images. We look to expand on

Food for thought: memory recall exercises 6 existing research on the testing effect to observe the impacts of memory recall exercises on student performance targeting more complex and diverse topics from a core-level American politics course.

Methodology

Our research examines the effect of memory recall exercises on student learning. We use similar methods as previous authors but test more complex cognitive functions among college students rather than simplistic tasks among primary education students. Our research utilized a series of in-class memory recall exercises targeting major course concepts. The study included twenty-nine sections of the U.S. Military Academy’s mandatory, 200-level American Politics course. Ten instructors and 321 students participated in the project. Sections were randomly assigned to one of three groups; two treatment groups and a control group. Students were blocked by class section so that all students in a given section were assigned to the same control or treatment group. Instructors who taught multiple sections had their sections randomly assigned, while ensuring each instructor had at least one control section. During grading, individual instructors mixed exams from their respective treatment and control sections to prevent grading bias toward any one group.

Memory recall exercises entailed providing students a short prompt asking them to write everything they could recall about a specific topic previously covered in the course. Students were allowed ten minutes at the beginning of class to complete the exercise. Five exercises were completed at roughly even intervals throughout the course with each exercise covering a different aspect of course material relevant to a subsequent graded exam.

Food for thought: memory recall exercises 7

We created two treatment groups to examine impacts of repeated testing and potential contamination on student performance. Students assigned to the first treatment group, Treatment

(No Notes), were required to complete the exercise without referencing their course notes or readings. The second group, Treatment (Notes), were allowed to reference their course notes and readings. We measured treatment effects by comparing students’ overall course grades. We also recorded student performance on individual exam questions directly correlated to questions posed in the exercise prompts to offer further insights into how memory recall exercises impact final course grades.

Our study assessed student performance across the two treatment groups to test whether the inability to reference class notes during the exercise contributed to greater learning endurance by making learning more difficult. Here we hypothesis that students not referencing notes or course readings would be more likely to retain course concepts due to being forced to recall information from memory rather than by simply regurgitating notes. This type of recall should be more difficult, and therefore more likely to promote later retrieval.

Finally, we conducted an end of course survey with the respective treatment groups to glean students’ feedback about the exercises. Surveys centered around four statements pertaining to students’ perception of the memory recall exercises’ effects. Students were asked to what extent they agreed or disagreed with statements indicating the exercises improved their performance on subsequent exams, improved their overall understanding of course concepts, helped focus their exam preparation efforts, and prompted students to review course material outside of class. Surveys employed a five-point Likert scale where 1 represented disagreement with the statement, and 5 represented significant agreement. An answer indicating agreement

Food for thought: memory recall exercises 8 with the statement corresponds with the student perceiving the memory recall exercises contributed positively in some way.

Expected Results

We expected students who were assigned to either treatment group to exhibit a greater ability to retrieve and apply information as measured by stronger performance in the course.

These students should have benefitted from the opportunity to practice retrieval of course material through the memory retrieval exercises. We hypothesize that exposure to more retrieval opportunities will result in a greater ability to retain and apply course concepts. Additionally, we expect students who were not allowed to reference course notes and other materials to perform better than those who were allowed to access notes. We believe the greater difficulty associated with recalling information from memory (rather than from referenced class notes) will lead to a greater ability to recall information later.

Results

Students who were given the memory recall exercises did not demonstrate improved course performance, as measured by both final course grades and all exam grades. This is true of students in both treatment groups. A difference of means test found no significant difference between the control and either treatment group. The magnitude of the difference between means was also not substantial. Therefore, upon initial analysis, these results suggest that the memory recall exercises had no effect on student performance. However, given our access to extensive data on the students and the complexity of the variables involved we sought to leverage other empirical methods and control for other effects. We controlled for idiosyncrasies associated with

Food for thought: memory recall exercises 9 different class hours, instructors, student GPA, and different versions of the final exam on student performance by conducting Ordinary Least Squares (OLS) regressions.

Unsurprisingly, we found that student GPA is

the overwhelmingly dominant explanatory variable.

Table 1 shows a highly significant impact of a 7%

increase in final grades for every 1 point higher GPA

the student previously earned. In other words,

students with a 3.5 GPA earned a 7% higher grade on

their final exam than students with a 2.5 GPA. We

also found that the version of the final exam students

took had a significant impact of 1.1% on final grades.

Controlling for these fixed effects, as well as

anticipated differences between different class hours

and instructors, validated our initial finding of no

impact for those who received the Treatment (No

Notes). However, the Treatment (Notes) group performed significantly lower (-1.4%) on final grades. This result suggests that students who conducted the memory recall exercises with access to their notes performed worse, not better, than students who conducted the exercises without notes or did not conduct the exercises at all.

Findings and Discussion

Our results yielded no significant findings to support the premise that interrupting forgetting improves student performance in a classroom environment using memory recall exercises. Most results indicated that the exercises had no effect. Moreover, in one model the

Food for thought: memory recall exercises 10 memory recall exercises had a negative effect on student performance in the group that had access to their notes and course readings during the exercises. This outcome could be attributable to several factors.

First, the intervals between applicable lessons, memory recall exercises, and subsequent exams may have been inappropriate for achieving the desired effect. Second, we did not use any feedback mechanisms with the exercises. Providing feedback may have improved the effectiveness of the exercise.18 Conducting the exercise without reference to the correct answer may have had the unintended “contamination” effect of reinforcing the student’s “benchmarked” wrong answer.19 Third, the structure of the exercise could have been suboptimal for improving retention of the targeted topics. For instance, selecting different topics for the exercise or interrupting forgetting through different methods (e.g. oral recall, etc.) may have changed retention outcomes.

Our survey results (see Appendix B) revealed several discontinuities between the measured effect of the recall exercises and the student’s perception of the exercise’s effect on performance. First, student self-perception is that the exercise improved their performance (see

Appendix B). Although in reality this was not the case, this suggests that the exercise itself may have a positive effect beyond the scope of this study, such as on general student interest in the topic.

The second useful finding from our survey regarding student self-perception is that the exercise had no effect on their motivation to study out-of-class (see Appendix B). This is informative although out-of-class motivation was not the primary goal of the activity. It suggests that requiring the student to access previously-learned material did not have a noticeable effect on their study habits.

Food for thought: memory recall exercises 11

Taken together, these results provide several useful observations. First, instructors should be aware of the discrepancy between the student’s perceived benefit from the exercise, which was perceived as high, and the actual measured effect on performance, which was negligible. This potentially has both positive and negative effects on student learning beyond the exercise’s intended effects. On one hand, similar exercises could be used to facilitate student comfort with the material and improve their confidence in the classroom. On the other hand, this can be detrimental to learning outcomes if such an effect leads to overconfidence. The second useful observation for instructors is the unintended “contamination” effect that can result from lack of feedback. This means that providing feedback on student responses may be valuable to prevent students from “benchmarking” to incorrect understandings of concepts.

Conclusions and Future Research

We found no evidence that written memory recall exercises improved long-term knowledge retention based on a semester-long experiment in an undergraduate-level survey course. Additionally, we found evidence that, under certain conditions, such exercises may reduce student performance and long-term retention. This suggests that instructors may want to consider alternative methods for improving student retention.

Although insightful, our study leaves open the possibility for future research in several areas. Based on our survey results, one area of focus concerns the role of instructor feedback in improving knowledge retention. Many students indicated on end-of-course surveys that they thought the exercises were helpful, however, they believed they would be even more so if the students had received instructor feedback, either individually or as a group through in-class discussion. It is uncertain whether feedback would have produced different results. Scholarship on negative “benchmarking” effects that occur in the absence of feedback suggests that

Food for thought: memory recall exercises 12 providing feedback in conjunction with the retrieval exercises could have improved knowledge retention. With minimal effort, this study could be replicated to include a variety of instructor feedback approaches to evaluate potential effects.

A second area for future research concerns the level of specificity that the recall exercise targets, and its influence on overall learning outcomes. The exercises conducted in our study focused on retrieving knowledge on specific topics about one political actor per exercise (in a course that surveyed a variety of political actors). However, existing scholarship suggests that long-term retention can be improved by “interleaving” multiple concepts within a single exercise.20 Therefore, it may be beneficial to replicate this experiment with varied levels of specificity.

Another consideration for future analysis on memory recall should leverage technology.

Exercises could be required outside of class using a digital platform that may increase student contact with course content and better enable instructor feedback.

Finally, it may be that this study’s findings are influenced by the nuances inherent to the study of American politics, or to West Point’s unique learning environment and/or student population. Therefore, our findings should be tested across academic disciplines, and in civilian undergraduate institutions before judging our findings’ generalizability.

While we acknowledge memory recall for content-based courses is ultimately dependent on student effort, pedagogical techniques can be leveraged to affect outcomes in the aggregate.

Instructor techniques and course design do play a critical role in memory recall. Future experiments can further clarify the relationship of memory recall exercises to student performance.

Food for thought: memory recall exercises 13

NOTES

Food for thought: memory recall exercises 14

1 Peter C. Brown, Henry L. Roediger, and Mark A. McDaniel, Make it Stick: The Science of Successful Learning (Cambridge, Harvard University Press, 2014), 33. 2 Brown, Roediger, and McDaniel, 75-9. 3 Brown, Roediger, and McDaniel, 235-241. 4 Francis Bacon, Novum Organum, trans. Lisa Jardine and Michael Silverthorne (Cambridge, England: Cambridge University Press, 2000), 646. 5 Stewart H. Hulse, James Deese, and Howard Egeth, “The Psychology of Learning (New York, McGraw-Hill, 1975), 346. 6 Hulse, Deese, and Egeth, 346. 7 Peter C. Brown, Henry L. Roediger, and Mark A. McDaniel, Make it Stick: The Science of Successful Learning (Cambridge, Harvard University Press, 2014), 28. 8 Phillip B. Ballard, “Oblivescence and Reminiscence,” British Journal of Psychology Monograph Supplements 1 (1932), 1-82. 9 Matthew Erdelyi and J. Becker, “Hypermnesia for Pictures: Incremental Memory for Pictures but not for Words in Multiple Recall Trials,” Cognitive Psychology 6, (1974), 159-171. 10 Henry L. Roediger and Jeffrey D. Karpicke, “The Power of Testing Memory: Basic Research and Implications for Educational Practice,” Perspectives on Psychological Science 1, No 3, (September 2006), 181. 11 Mark A. Wheeler and Henry L. Roediger, “Disparate Effects of Repeated Testing: Reconstructing Ballard’s (1913) and Bartlett’s (1932) Results,” Psychological Science 3, (July 1992) 240-5. 12 Wheeler and Roediger, 240. 13 Ibid. 14 James Deese, The Psychology of Learning (New York, McGraw-Hill, 1958), 237-8. 15 Hulse, Deese, and Egeth, 340. 16 Ibid. 17P.K. Agarwal, P. M. Bain, & R. W. Chamberlain, “The value of applied research: Retrieval practice improves classroom learning and recommendations from a teacher, a principal, and a scientist.” Educational Psychology Review 24 (2012), 437– 448. 18 Brown, Roediger, and McDaniel, 235-241. 19 Argawal, Bain, and Chamberlain. 20 Brown, Roediger, and McDaniel, 47-50.

REFERENCES

Food for thought: memory recall exercises 15

Agarwal, P.K. P. M. Bain, & R. W. Chamberlain, “The value of applied research: Retrieval practice improves classroom learning and recommendations from a teacher, a principal, and a scientist.” Educational Psychology Review 24 (2012), 437– 448. Bacon, Francis. Novum Organum. Translated by Lisa Jardine and Michael Silverthorne. Cambridge, England: Cambridge University Press, 2000. Ballard, Phillip B. “Oblivescence and Reminiscence.” British Journal of Psychology Monograph Supplements 1 (1932): 1-82. Bartlett, Frederic C. Remembering: A Study in Experimental and Social Psychology. Cambridge, England, Cambridge University Press, 1932. Brown, Peter, Henry Roediger, and Mark McDaniel. Make it Stick: The Science of Successful Learning. Cambridge: Harvard University Press, 2014. Deese, James, The Psychology of Learning. New York: McGraw-Hill, 1958. Erdelyi, Matthew and J. Becker. “Hypermnesia for Pictures: Incremental Memory for Pictures but not for Words in Multiple Recall Trials.” Cognitive Psychology 6, (1974), 159-171. Hulse, Stewart H., James Deese, and Howard Egeth, The Psychology of Learning. New York, McGraw-Hill, 1975. Roediger, Henry L. and Jeffrey D. Karpicke. “The Power of Testing Memory: Basic Research and Implications for Educational Practice.” Perspectives on Psychological Science 1, No 3 (September 2006), 181-210. Wheeler, Mark A. and Henry L. Roediger. “Disparate Effects of Repeated Testing: Reconstructing Ballard’s (1913) and Bartlett’s (1932) Results.” Psychological Science 3 (July 1992): 240-5.

Appendix A: Memory Recall Exercise Prompts

Memory Recall Exercise I

Food for thought: memory recall exercises 16

Write down everything you know about Thomas Hobbes and John Locke. Some things you might consider include who these men were, what ideas did they have about human nature and government, and how did they influence our society and government today? Anything else you think relevant. Memory Recall Exercise II Write down everything you know about Congress. Some things you might consider include: What formal powers are granted to Congress in the Constitution? How might Congress check or influence the actions of the other branches of government? What motivates members of Congress? How do members of Congress conceive of their constituencies? What are some ways Congress can ensure Executive agencies implement/enforce laws according to Congress’ intent? Anything else you remember. Memory Recall Exercise III Write down everything you know about the Presidency. Some things you might consider include: What are the president’s formal powers enumerated in the Constitution? What are the president’s implied or informal powers? How might the president check the actions of the other branches of government? How might he otherwise influence them? How might the president exercise control over the bureaucracy? Memory Recall Exercise IV Write down everything you know about the Interest Groups. Some things you might consider include: What role do interest groups play in the policymaking process. What benefits do interest groups provide to members of government? What are inside and outside lobbying strategies, and under what circumstances might one type of strategy be preferred over the other? How do interest groups solve the collective action problem? Memory Recall Exercise V Write down everything you know about political parties. Some things you might consider include: What roles do political parties play in the American political system? What roles do political parties play in the policymaking process? Describe political parties as organizations. How do they relate to the electorate? How do political parties reduce the cost of voting? Why do there tend to be only two major political parties in the U.S.? Anything else you can think of!

Appendix B: End-of-Course Survey with Results

Instructor Name______Section______

Food for thought: memory recall exercises 17

1. In-class writing exercises helped improve my performance on later exams?

Strongly Disagree Neutral Agree Strongly Disagree Agree

2. In-class writing exercises helped me better understand course concepts? Strongly Disagree Neutral Agree Strongly Disagree Agree

3. In-class writing exercises prompted me to focus my exam preparation on the topics covered in the in-class exercises more than I would have had I NOT seen those topics on the in-class activities?

Strongly Disagree Neutral Agree Strongly Disagree Agree

Food for thought: memory recall exercises 18

4. In-class writing exercises prompted me to review course-related topics outside of class.

Strongly Disagree Neutral Agree Strongly Disagree Agree