“DESIRABLE DIFFICULTIES”

STUDYING

REMEMBERING

J Baucom, Landmark College, Summer Institute 2017 Zeus, , and the Nine Muses (Anton Mengs, 1761) Six Highly Effective Study Practices

Space Your Studying

Vary the Conditions of Study

Test Yourself

Write Questions

Draw Visuals

Summarize What You Learn Agenda:

u Important features of

u Why difficulties in learning are good

u 3 study strategies that build on how memory works

u Selected studies

u What insights into studying tell us about teaching

u Educational implications, questions QUIZ u 1. What was the name of your 6th grade math teacher?

u 2. Who was the 25th President of the ?

u 3. What is the name of the person you introduced yourself to at the beginning of this session? (1885)

u First to apply experimental procedures, mathematical models, and statistics to the study of learning and memory

u Conducted a series of rigorously controlled between 1879- 1884 with a single subject Ebbinghaus: 4 Major Contributions

Relearning Spacing Overlearning Curve Effect Effect

• What is lost • What is saved • How it’s • Surplus Ebbinghaus's Forgetting/Retention Curve

120%

100%

80%

60% RELEARNED 40%

20% PERCENTAGE OF INFORMATION INFORMATION OF PERCENTAGE

0% 0 19 minutes 63 minutes 8 hours 24 hours 2 days 6 days 31 days

INTERVAL BETWEEN ORIGINAL LEARNING AND MEMORY TEST A New Theory of Disuse Robert and Elizabeth Bjork (1992)

Disuse Decay Retrieval Strength Two Distinct Memory Strengths : Retrieval: How well learned How accessible

Strong Weak Strong Weak High SS/Low RS Low SS/ High RS q childhood street address q current hotel room number q Distributive law in mathematics q Ebbinghaus: what percentage of memory loss occurs in first 20 minutes after learning?

Low SS/Low RS High SS/High RS

q old hotel room number q current street address q 25th US president q how to calculate a percentage Key Design Features of Storage

Doesn’t Unlimited Diminish

Expands Latent with Use Value of Storage Strength

Ø Enhances gain of RS

Ø Slows the loss of RS

Ø Enables us to recognize or relearn info What Strengthens Storage?

Meaningfulness

Repeated Connection to Existing Practice Knowledge SS Key Design Features of Retrieval

Determines Limited, Frail Current Performance

Strengthens Changes with Difficulty Memory Determinants of Retrieval

Relative Strength Absolute Context Strength Cues RS u Who was the 25th President of the United States? u Which President was assassinated, resulting in Theodore Roosevelt becoming President? u William McKinley Adaptive Value of Disuse Theory Takeaways/Educational Implications

Forgetting is good

Good How do we teaching= boost RS? SS +++

Beware High RS “illusion of mastery” Low SS What Is A Desirable Difficulty?

u “training conditions that are difficult and appear to impede performance during training but that yield greater long-term benefits than their easier training counterparts” (Bjork, 2016)

u “…conditions that produce the fewest errors during learning (like massed practice…) can lead to very poor long-term retention” (Karpicke and Roediger 2008)

u Study harder? The Cognitive Reflection Test (Frederick 2005)

u 1. A bat and a ball cost $1.10 in total. The bat costs $1.00 more than the ball.

How much does the ball cost?

u Answer: u Alter et al (2007) administered CRT to Princeton students u Avg Score: 1.9/3

u With altered font? Avg score: 2.45 out of 3 Strategy/Desirable Difficulty 1: Space Your Studying (Distributed Practice)

u Spread out practice of material over time rather than in one large time block

u Opposite of cramming Spaced Study or Distributed Practice

Mon Tues Wed Thurs Total Fri- Study Test Day Time John 4 hours

Susan 4 hours

Fernando 4 hours Spaced Study or Distributed Practice

Mon Tues Wed Thurs Total Fri- Study Test Day Time John 4 hours 4 hours

Susan 2 hours 2 hours 4 hours

Fernando 1 hour 1 hour 1 hour 1 hour 4 hours Spaced Study or Distributed Practice

Mon Tues Wed Thurs Total Fri- Study Test Day Time John 4 hours 4 hours C-

Susan 2 hours 2 hours 4 hours B-

Fernando 1 hour 1 hour 1 hour 1 hour 4 hours A Cepeda et al. 2006

839 assessments of spaced study

317 different experiments

14,000 participants

average benefit of 15% on test performance for spaced study over massed study u Spacing between study sessions should be 10-20% of the desired retention interval (the delay between final study session and criterion task) Dunlosky et al., 2013

u To remember info 30 days later, aim for 3-6 days between study sessions Moulton, Resnick et al (2006) “Teaching Surgical Skills: What Kind of Practice Makes Perfect?”

u University of Toronto Department of Surgery u Taught complex surgical skill using different schedules of instruction:

Massed Training: 4 sessions in one day

Distributed Training: 1 session per week, over 4 weeks Immediately One Month After Training Later •Both groups •DT group equal better on performance retention and transfer Why Does Spacing Work?

Relearing Forgetting Desirable Effect Occurs Difficulty Ebbinghaus on

u …the method naturally employed in practice agrees. The schoolboy doesn’t force himself to learn his vocabularies and rules altogether at night, but knows that he must impress them again in the morning. A teacher distributes his class lesson not indifferently over the period at his disposal but reserves in advance a part of it for one or more reviews” (p. 89) Implications Beyond School and College Educational Implications/Reflection

u Writing Exercise: What are the implications for spacing in the design and implementation of our educational models?

u Writing Exercise: Knowing what you know about spacing effects, how might you apply this powerful strategy in your work? Strategy/Desirable Difficulty 2:

uVary the Conditions of Your Learning u Physical Context

u Cognitive Context (questions, visuals, summaries) u Informational Context: Interleaving Why Create Variation?

u Builds a richer network of connections and associations, “memory landmarks”

u Reduces context dependency, specificity (Tulving, 1973) Physical Context: Studying and Testing (Smith, Glenberg &Bjork (1978)

Task: Study a list of 40 words

Study Session 1 Study Session 2 Testing

Group 1 Context A Context A Context C (15.9 words, 39%)

Group 2 Context A Context B Context C (completely (24.4 words, 61%) different) Adding Variation: Interleaving u Alternate skills or concepts within a given time period, rather than studying each one in a large block u u A-A-A, B-B-B, C-C-C

A-B-C, A-B-C, A-B-C or A-B-C, B-A-C, C-B-A

u Applies to motor skills, semantic knowledge, procedural knowledge, arts, music, athletics u Kang and Pashler (2012), college students studied 40 paintings by three artists (Blencoe, Lindenberg, O’Shea) and their styles in interleaved, massed block, or spaced block conditions u Interleaved group best able to recognize painters’ styles on a transfer test Interleaving Math Instruction (Rohrer et al, 2015)

u Over a 3 month period, 126 7th graders were taught how to graph linear equations and how to find the slope of a line

u Some classes = homework with interleaved graph, blocked slope problems

u Other classes = homework with interleaved graph and blocked slope problems

Results of Rohrer 2015 90

80

70

60

50

40

30

20

10

0 I day delay 30 day delay

Interleaved Blocked Why Does Interleaving Work? Educational Implications/Reflection

u How well do we “mix up” and create varying conditions in our teaching? Are we too cautious about mixing it up?

u Do we interleave instruction or teach in blocked units?

u Interleaving is how the “real world” works!

u Writing Exercise: Knowing that varying the conditions of learning can impact retention, how might you apply this desirable difficulty in your work? Strategy/Desirable Difficulty 3: Test Yourself

u Test-enhanced learning

u “”

u One of the best study practices (Dunlosky et al. 2013) Studying vs Testing

Roediger & Karpicke 2006 90 80 70 60 50

Recalledd 40 30 20 10 0 5 Minutes 1 Week Proportion of Ideas of Proportion SSSS SSST STTT Retention Interval Why does testing improve learning?

Demands Modifies Increases Retrieval Memory SS and RS How Can Students Test Themselves?

•Summaries •Visuals

•SMART Cards •.com

•Reciprocal Teaching •Practice Tests Educational Implications/Reflection

u Is the testing effect utilized as a teaching tool often enough?

u : 1) purpose, value of testing 2) low-stakes vs high stakes testing

u Writing Exercise: Knowing what you know about the testing effect, how might you apply this powerful strategy in your work? Putting It All Together: A Study Plan

Mon Mon Mon Mon Sun Mon Mar 11 Mar 18 Mar 25 Apr 1 Apr 7 Apr 8 Test Day

Karen 1 hour 1 hour 1 hour 1 hour 1 hour A+++

Where? Context A Context B Context A Context C Context A Context A EAB108 Library EAB108 Dining Hall EAB108 EAB108

How? Study Self Test Self Test Self Test Study, night Real Test before test, Interleave Interleave Interleave Interleave then sleep Spaced Study

u Writing Exercise: What are the implications for spacing in the design and implementation of our educational models?

u Writing Exercise: Knowing what you know about spacing effects, how might you apply this powerful strategy in your work? Interleaving

u Writing Exercise: Knowing that varying the conditions of learning can impact retention, how might you apply this desirable difficulty in your work? Testing Effect

u Writing Exercise: Knowing what you know about the testing effect, how might you apply this powerful strategy in your work? Sources Cited

Alter, A.L., Oppenheimer, D.M., Epley, N., Eyre, R.N. (2007). Overcoming intuition: metacognitive difficulty activates analytic reasoning. Journal of experimental : general, 136, (4), 569-576.

Bjork, R.A. & Bjork, E.L. (1992). A new theory of disuse and an old theory stimulus fluctuation. In A. Healy, S. Kosslyn, & R. Shiffrin (Eds.) From Learning Processes to Cognitive Processes: Essays in Honor of William K. Estes, (Vol. 2, pp. 35-67). Hillsdale, NJ: Erlbaum.

Bjork, R.A. & Bjork, E.L. (2016). UCLA: Bjork learning & forgetting lab. bjorklab.psych.ucla.edu/research.html

Butler, A. C. (2010). Repeated testing produces superior transfer of learning relative to repeated studying. Journal of : Learning, memory and , 36 (5), 1118- 1133.

Cepeda, N. J., Pashler, H., Vul, E., Wixted, J. T., & Rohrer, D. (2006). Distributed practice in verbal tasks: A review and quantitative synthesis. Psychological Bulletin, 132, 354-380.

Cepeda, N.J., Coburn, N., Rohrer, D., Wixted, J.T., Mozer, M.C., & Pashler, H. (2009). Optimizing distributed practice: Theoretical analysis and practical implications. Experimental psychology, 56 (4), 236-246.

Dempster, F. N. (1988). The spacing effect: A case study in the failure to apply the results of psychological research. American psychologist, 43 (8), 627-635.

Dunlosky, J., Rawson, K.A., Marsh, E.J., Nathan, M.J., & Willingham, D.T. (2013). Improving students’ learning with effective learning techniques: promising directions from cognitive and educational psychology. Psychological science in the public interest, 14 (1), 4-58.

Ebbinghaus, Hermann (1964). Memory: A contribution to experimental psychology. (Henry A. Ruger & Clara E. Bussenius, translators.). New York: Dover Publications, Inc. (Original work published 1885)

Frederick, S. (2015). Cognitive reflection and decision making. Journal of economic perspectives, 19 (4), 25-42.

Grant, H. M., Bredahl, L. C., Clay, J., Ferrie, J., Groves, J. E., McDorman, T. A., & Dark, V. J. (1998). Context-dependent memory for meaningful material: Information for students. Applied , 12, 617-623.

Kang, S. H.K., & Pashler, H. (2012). Learning painting styles: spacing is advantageous when it promotes discriminative contrast. Applied cognitive psychology, 26 (97-103).

Karpicke, J. D., & Roediger III, H. L. (2008). The critical importance of retrieval for learning. Science, 319, 966-968. doi:10.1126/science.1152408

Kerr, R. & Booth, F. (1978). Specific and varied practice of motor skill. Perceptual and motor skills, 46 (2) 395-401.

Moulton, C-A.E., Dubrowski, A., MacRae, H., Graham, B, Grober, E. & Reznick, R. (2006). Teaching surgical skills: what kind of practice makes perfect? Annals of surgery, 244 (3), 400- 409.

Nantais, K.M., & Schellenberg, G.E. (1999). The Mozart effect. Psychological science, 10 (4), 370-373.

Pashler, H., & Carrier, M. (1996). Structures, processes, and the flow of information. In E. L. Bjork & R. A. Bjork (Eds.), Memory (2nd ed. pp. 3-29). San Diego: Academic Press.

Roediger, H. L., & Guynn, M. J. (1996). Retrieval processes. In E. L. Bjork & R. A. Bjork (Eds.), Memory (2nd ed. pp. 197-236). San Diego: Academic Press.

Roediger, H. L., & Guynn, M. J. (1996). Retrieval processes. In E. L. Bjork & R. A. Bjork (Eds.), Memory (2nd ed. pp. 225-228). San Diego: Academic Press.

Roediger, H. L., & Karpicke, J. D. (2006). The power of testing memory: Basic research and implications for educational practice. Perspectives on psychological science, 1 (3), 181-210.

Roediger, H. L., & Karpicke, J. D. (2006). Test-enhanced learning: Taking memory tests improves long term retention. Psychological science, 17 (3), 249-255.

Rohrer, D., Dedrick, R.F. & Stershic, S. (2015). Interleaved practice improves mathematics learning. Journal of educational psychology, 107 (3), 900-908.

Schellenberg, G.E. & Hallam, S. (2005). Music listening and cognitive abilities in 10- and 11- year-olds: The blur effect. Annals of the New York Academy of Sciences, 1060, 202-209.

Smith, S.M., (1985). Background music and context-dependent memory. The American Journal of Psychology, 98 (4), 342-353.

Smith, S.M., Glenberg, A., & Bjork, R.A. (1978). Environmental context and human memory. Memory & Cognition, 6 (4), 342-352.

Tulving, E., & Thompson, D. M. (1971). Retrieval processes in recognition memory: Effects of associative context. Journal of experimental psychology, 87 (1), 116-124.

Tulving, E., & Thompson, D. M. (1973). Encoding specificity and retrieval processes in . Psychological review, 80 (5), 352-373.