A Queueing Network Model for Spaced Repetition Siddharth Reddy, Igor Labutov, Siddhartha Banerjee sgr45, iil4, sbanerjee @cornell.edu { } Introduction Experiments

The ability to retain a large number of new ideas in memory is an essential component of human 0.9 30000 learning. In recent times, there has been a growing body of work that attempts to ‘engineer’ this process – creating tools that enhance the learning process by building on the scientific understanding 0.8 25000 of human memory. Flashcards are one such tool that use the idea of spaced repetition to overcome the human ‘forgetting curve’. Though they have been around for a while in the physical form, a 0.7 new generation of spaced repetition software such as SuperMemo, Anki, and Mnemosyne allow a 20000 0.6 much greater degree of control and monitoring of the process. As these software applications grow popular, there is a need for formal models for reasoning about and optimizing their operations. In 0.5 15000 this work, we use ideas from queueing theory to develop such a formal model for one of the simplest and oldest spaced repetition systems: the Leitner system. 0.4 Validation AUC 10000 0.3 Key Contributions 5000 0.2 Frequency (number of interactions) Formalize the Leitner spaced repetition system using ideas from queueing theory • 0.1 0 Validate the exponential forgetting curve on large-scale log data from the Mnemosyne flashcard 1 2 3 4 5 6 7 8 9 10 • software Deck (qij) Model 3: φ(θ β ) j − i Verify key predictions of our queueing model using a controlled experiment on Amazon Mechanical Model 5: exp ( θ d /n ) Model 10: exp ( θ d /n ) • − · ij ij − i · ij ij Turk Model 7: exp ( θ/n ) Model 12: exp ( θ /n ) − ij − i ij Model 8: exp ( θ d /q ) Model 13: exp ( θ d /q ) − · ij ij − i · ij ij Model 9: exp ( θ/q ) Model 14: exp ( θ /q ) Memory Model − ij − i ij

We use a variant on the exponential forgetting curve to model the decay of human memory over 0.025 time. The probability of a student recalling an item is as follows. Maximum Arrival Rate λext∗ Empirical  D P[recall] = exp θ 0.020 Simulated (Clocked Delay) − · s Simulated (Expected Delay) where θ is the item difficulty, D is time elapsed since previous review, and s is memory strength. 0.015

Leitner Queue Network (Items Per Second) n

λ 0.010 mastered items!

0.005 Throughput

0.000 0.00 0.02 0.04 0.06 0.08 0.10 0.12 0.14 0.16 Arrival Rate λext (Items Per Second) 1! 2! 3! 4! 5! 70 Deck 1 60 Deck 2 Deck 3 Deck 4 50 Deck 5 Mastered 40

new items! 30 Number of Items 20 New items arrive into deck 1 according to a Poisson process with rate λext. The routing probability matrix is P , where 10   P[recall s = i, ] if i < n j = i + 1 P = 1 [recall| s =· i, ] if (i > 1∧ j = i 1) i = j = 1 0 ij P 0.00 0.02 0.04 0.06 0.08 0.10 0.12  0 − | · otherwise∧ − ∨ Arrival Rate λext (Items Per Second) Items exit the system from deck n with probability P[recall s = n, ]. The service rate for deck i is indicated by µ , and the user’s work rate budget (e.g., the| maximum· number of items the user i 1.0 can review per day) is given by U. We are interested in finding µi that maximize the steady-state λ = 0.004 (Before Phase Transition) n ext P throughput of the system such that the budget constraint µi U is satisfied. Formally, we λext = 0.015 (Before Phase Transition) ≤ i=1 0.8 λext = 0.048 (After Phase Transition) must solve the following static planning problem. λext = 0.114 (After Phase Transition)

maximize λext n 0.6 X subject to µi U ≤ i=1 λext + P11λ1 + P21λ2 = λ1 0.4 P12λ1 + P32λ3 = λ2 . 0.2 Pn 1,nλn 1 + Pnnλn = λn − − µi, λi 0 i Fraction of Items Seen During Session ≥ ∀ λi < µi i ∀ 0.0 The algorithm for selecting the next item to present to the user is simple: sample deck i with 1 2 3 4 5 6 µi probability Pn , and select the item at the top of the sampled deck. Deck i=1 µi Full paper and code available at http://siddharth.io/leitnerq