DOCSLIB.ORG

Building a Formative Assessment System That Is Easy to Adopt Yet Supports Long-Term Improvement: a Review of the Literature and Design Recommendations

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Building a Formative Assessment System That Is Easy to Adopt Yet Supports Long-Term Improvement: a Review of the Literature and Design Recommendations Solaiman E, Barney J, Smith M. Building A Formative Assessment System That is Easy To Adopt Yet Supports Long-Term Improvement: A Review of the Literature and Design Recommendations. In: 9th International Conference on Computer Supported Education CSEDU. 2017, Porto, Portugal. Copyright: This is the author’s manuscript of a paper that was presented at the 9th International Conference on Computer Supported Education CSEDU, held 21-23 April 2017, Porto, Portugal. Link to Conference website: http://www.csedu.org/ Date deposited: 05/05/2017 This work is licensed under a Creative Commons Attribution-NonCommercial 3.0 Unported License Newcastle University ePrints - eprint.ncl.ac.uk Building A Formative Assessment System That is Easy To Adopt Yet Supports Long-Term Improvement: A Review of the Literature and Design Recommendations Ellis Solaiman, Joshua Barney, and Martin Smith School of Computing Science, Newcastle University, United Kingdom fellis.solaiman, j.barney1, [email protected] Keywords: Formative Assessment, Assessment Design, Review, Feedback Automation, Education Technology Abstract: Formative assessment has been shown to be an effective teaching tool, yet is infrequently used in practice. With the intent of building a formative e-assessment platform, we examine research on formative practices and supporting computer-based systems with a focus on: institutional barriers to adoption of previous systems; senses in which students and teachers can improve their practices across varying timescales; and collectible data (self-reported or otherwise) necessary or advantageous in supporting these processes. From this research we identify the minimal set of data which adequately supports these processes of improvement, arrive at a set of requirements and recommendations for an innovative system which collects, processes, and presents this data appropriately, and from these requirements design the architecture of an extensible electronic formative assessment system which balances the need for complex long-term analytics with that of accessibility. 1 Introduction by technical problems such as lack of integration with systems in use within institutions, and educational data-mining efforts fail due to the absence of tools Formative assessment refers to the use of testing which are easy for non-experts to use (Romero and to allow both students to reflect on their own learn- Ventura, 2010). A key factor inhibiting the adoption ing and teachers to determine the needs of individ- of formative assessment technologies is staff time. ual students, or groups of students. This can also Teacher time is an especially scarce resource – those be described as assessment for learning. Formative who see the value of educational technology may still assessment is often contrasted with summative as- avoid full utilization for this reason; ”These are all sessment, described as assessment of learning, which great initiatives, but I’m running out of hours in a day uses testing as a means for summarizing the perfor- ... I’m in overload!” (Hall and Hall, 2010). Research mance of students and their level of knowledge at the by (Macfadyen and Dawson, 2012) highlights that in end of a course (Looney, 2011) (Black and Wiliam, order to overcome individual and group resistance to 2009). Despite the clear benefits of formative assess- innovation and change, planning processes must cre- ment as an effective teaching tool, it is infrequently ate conditions that allow participants to both think and used in practice. Also whilst VLEs (Virtual Learn- feel positively about change; ”conditions that appeal ing Environments) are widespread in higher educa- to both the heart and the head”. Without investing re- tion, their formative abilities are often severely under- sources for effective automated data analysis and vi- utilized (Blin and Munro, 2008). Even when change sualization, large quantities of data will not compel is not resisted by faculty staff, adoption of educational social or cultural change (Kotter and Cohen, 2002); technology tends to proceed at a slow pace, and often ”increasing observability of change is an effective innovation is driven by lone individuals deciding to strategy for reducing resistance” (Rogers, 1995). use technologies without backing from above (Rus- sell, 2009). Although perception of student expec- This paper builds upon and complements previous tation can provide positive incentive, technical barri- work and literature reviews such as (Gikandi et al., ers along with a lack of institutional support are key 2011), by taking a more integrative approach. We suppressing factors (King and Boyatt, 2015). Cur- attempt through the development a holistic theoretic rently available Education technologies are hampered framework,to arrive at the design of a system suitable for active use both as a tool for student and teacher and Thompson, 2007) in that we conceive of teachers improvement, and as a platform for further empirical not just in terms of the role they can play in providing research about the impact of various formative strate- feedback to students but also as individuals who can gies. Initially, multiple high impact journals in the benefit from feedback themselves. field of educational technology were identified, and the titles and abstract of all publications from the the last 5 years were manually examined. A number of 3 Feedback Processes and Their potentially relevant papers were identified and scruti- nized in further detail, and common subjects and ref- Data Collection Requirements erences were noted. Keywords related to these sub- jects were used to find further papers using academic We examine the specifics of feedback processes search engines. The process of reading papers, dis- below, considering issues related by their similar data- covered through references or search, continued re- requirements together. This view allows us to move cursively until a clear picture of a range of formative closer to the design of a real system. and learning practices emerged. A conceptual frame- work was developed to organize these practices by 3.1 Formative Feedback Principles time and individual, and the body of collected pa- pers was re-examined in the context of this frame- Synthesizing research literature, (Nicol and work to explore the data-collection requirements of Macfarlane-Dick, 2006) present seven princi- each of the identified practices. Recommendations ples, reordered for clarity here, arguing effective are made for implementation, and from these recom- formative feedback practice: mendations, the design of an extensible system to col- 1. delivers high quality information to students lect this data is presented. about their learning. 2. helps clarify what good performance is (goals, criteria, expected standards). 3. provides opportunities to close the gap between current and desired performance. 4. facilitates the 2 Classification of Feedback development of self-assessment (reflection) in learn- by User and Duration ing. 5. encourages positive motivational beliefs and self-esteem. 6. encourages teacher and peer dialogue Multiple researchers distinguish types of forma- around learning. 7. provides information to teachers tive assessment based on their duration. Allal and that can be used to help shape teaching. Schwartz (Allal and Schwartz, 1996) use the termi- The first three principles mirror (Wiliam and nology of “Level 1” and “Level 2’ formative assess- Thompson, 2007)’s theory of formative assessment, ments, which directly benefit students who are as- which emphasizes establishing where the learners are sessed as or use data gathered to benefit future in- in their learning, where they are going, and what structional activities or new groups of students respec- needs to be done to get them there. Black and tively. Alternatively, (Wiliam, 2006) distinguishes be- William, (Black and Wiliam, 1998) warn that class- tween short-, medium-, and long-cycle formative as- room cultures focused on rewards or grades encour- sessment, which operate within and between lessons, age students to game the system, avoid difficult tasks, within and between teaching units, and across mark- and where difficulties or poor results are encountered, ing periods, semesters or even years. In addition to ”retire hurt”. By 2004, this had been strengthened acting as a helpful framework within which to clas- in (Black et al., 2004) into an explicit recommenda- sify and organize research, this viewpoint is useful tion to adopt comment-only marking practices; ”Stu- as a design tool – collecting data required for longer- dents given marks are likely to see it as a way to com- term formative processes requires action over a sus- pare themselves with others; those given only com- tained period of time, and this affects the short-term ments see it as helping them to improve. The latter experience of new users. As discussed below, a sys- group outperforms the former.” A literature review by tem must show immediate value in order to have the (Gikandi et al., 2011) provides supporting studies and best chance at being widely adopted, and balancing explicitly reaffirms the above seven principles as ”an the conflict between supporting longer-term goals and essential condition for effective formative feedback”. maintaining accessibility is a design challenge that Whilst the above principles may be sufficient to deserves thinking about explicitly. In addition to dis- guide the behavior of teachers in conversation with tinguishing processes by time we also do so by the students in
Load more

Recommended publications
  • Teaching the Science of Learning Yana Weinstein1* , Christopher R
    Weinstein et al. Cognitive Research: Principles and Implications (2018) 3:2 Cognitive Research: Principles DOI 10.1186/s41235-017-0087-y and Implications TUTORIALREVIEW Open Access Teaching the science of learning Yana Weinstein1* , Christopher R. Madan2,3 and Megan A. Sumeracki4 Abstract The science of learning has made a considerable contribution to our understanding of effective teaching and learning strategies. However, few instructors outside of the field are privy to this research. In this tutorial review, we focus on six specific cognitive strategies that have received robust support from decades of research: spaced practice, interleaving, retrieval practice, elaboration, concrete examples, and dual coding. We describe the basic research behind each strategy and relevant applied research, present examples of existing and suggested implementation, and make recommendations for further research that would broaden the reach of these strategies. Keywords: Education, Learning, Memory, Teaching Significance educators who might be interested in integrating the Education does not currently adhere to the medical strategies into their teaching practice, (b) science of learn- model of evidence-based practice (Roediger, 2013). ing researchers who are looking for open questions to help However, over the past few decades, our field has made determine future research priorities, and (c) researchers in significant advances in applying cognitive processes to other subfields who are interested in the ways that princi- education. From this work, specific recommendations ples from cognitive psychology have been applied to can be made for students to maximize their learning effi- education. ciency (Dunlosky, Rawson, Marsh, Nathan, & Willing- While the typical teacher may not be exposed to this ham, 2013; Roediger, Finn, & Weinstein, 2012).
    [Show full text]
  • Enhancing Human Learning Via Spaced Repetition Optimization
    Enhancing human learning via spaced repetition optimization Behzad Tabibiana,b,1, Utkarsh Upadhyaya, Abir Dea, Ali Zarezadea, Bernhard Scholkopf¨ b, and Manuel Gomez-Rodrigueza aNetworks Learning Group, Max Planck Institute for Software Systems, 67663 Kaiserslautern, Germany; and bEmpirical Inference Department, Max Planck Institute for Intelligent Systems, 72076 Tubingen,¨ Germany Edited by Richard M. Shiffrin, Indiana University, Bloomington, IN, and approved December 14, 2018 (received for review September 3, 2018) Spaced repetition is a technique for efficient memorization which tion algorithms. However, most of the above spaced repetition uses repeated review of content following a schedule determined algorithms are simple rule-based heuristics with a few hard- by a spaced repetition algorithm to improve long-term reten- coded parameters (8), which are unable to fulfill this promise— tion. However, current spaced repetition algorithms are simple adaptive, data-driven algorithms with provable guarantees have rule-based heuristics with a few hard-coded parameters. Here, been largely missing until very recently (14, 15). Among these we introduce a flexible representation of spaced repetition using recent notable exceptions, the work most closely related to ours the framework of marked temporal point processes and then is by Reddy et al. (15), who proposed a queueing network model address the design of spaced repetition algorithms with prov- for a particular spaced repetition method—the Leitner system able guarantees as an optimal control
    [Show full text]
  • Learning with Smartphones
    LEARNING WITH SMARTPHONES Case study of learning drug prescribing using the PharmaFrog app Inauguraldissertation zur Erlangung des Akademischen Grades eines Dr. phil., vorgelegt dem Fachbereich 02 Sozialwissenschaften, Medien und Sport der Johannes Gutenberg-Universität Mainz von Dativa Tibyampansha aus Tansania Mainz, Germany Tag des Prüfungskolloquiums: July 15, 2020 PRELIMINARY REMARKS I would like acknowledge the indispensable contributions of the following individuals and institutions: 1) This thesis’ advisors 2) Programmers 3) PharmaFrog app concept and content developers 4) The participants of the surveys and of the PharmaFrog app evaluation presented in this thesis 5) My family and friends for moral support and encouragement. The PharmaFrog app can be downloaded using the following QR codes. Android iOS i TABLE OF CONTENTS LIST OF FIGURES ................................................................................................................. vi LIST OF TABLES ................................................................................................................. viii ABBREVIATIONS .................................................................................................................. ix SUMMARY ............................................................................................................................... 1 INTRODUCTION..................................................................................................................... 3 Deficiencies in drug prescribing ....................................................................................................................
    [Show full text]
  • Senior Students
    Digital Ambition Parent Discussion Guide This sheet is intended to help you have a discussion with your child about how to study in the most effective way. Below, we have a number of useful study applications and suggested questions that you and your child can ask each other about how you both learn to do new things and manage your time. By sharing your experiences with your child, we hope that you both improve your productivity in both study and work. Each question should be answered by your child AND yourself. Applications that help you create Applications that reduce/block distractions flashcards/quizzes and study schedules One of the biggest problems for modern students is the number of online and mobile distractions that hinder effective study. Research has shown that retrieval practice is one of the most effective ways of studying. Retrieval practice is testing Many of these websites and programs have hundreds of yourself regularly about the key ideas in anything new you’re engineers designing them to figure out the best way to get learning. One common way of this is creating flashcards on people to use them and keep using them, so it’s not a surprise the key ideas as you’re reading the material for the first time that many students give into the temptation to check their and then testing yourself as a way of studying the material. phone or surf the web for a minute and then find out that they’ve wasted most of their study time. Research also shows that if you spread out what you are studying so that you revise material just before you’re about Rescuetime: to forget it, your long-term learning will be much better.
    [Show full text]
  • Teaching the Science of Learning
    Cogn Res Princ Implic. 2018 Dec; 3: 2. PMCID: PMC5780548 Published online 2018 Jan 24. doi: 10.1186/s41235-017-0087-y PMID: 29399621 Teaching the science of learning Yana Weinstein, 1 Christopher R. Madan,2,3 and Megan A. Sumeracki4 1Department of Psychology, University of Massachusetts Lowell, Lowell, MA USA 2Department of Psychology, Boston College, Chestnut Hill, MA USA 3School of Psychology, University of Nottingham, Nottingham, UK 4Department of Psychology, Rhode Island College, Providence, RI USA Yana Weinstein, Email: [email protected]. Contributor Information. Corresponding author. Received 2016 Dec 20; Accepted 2017 Dec 2. Copyright © The Author(s) 2018 Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. Abstract The science of learning has made a considerable contribution to our understanding of effective teaching and learning strategies. However, few instructors outside of the field are privy to this research. In this tutorial review, we focus on six specific cognitive strategies that have received robust support from decades of research: spaced practice, interleaving, retrieval practice, elaboration, concrete examples, and dual coding. We describe the basic research behind each strategy and relevant applied research, present examples of existing and suggested implementation, and make recommendations for further research that would broaden the reach of these strategies. Keywords: Education, Learning, Memory, Teaching Significance Education does not currently adhere to the medical model of evidence-based practice (Roediger, 2013).
    [Show full text]
  • Downloading Speed and Lack of Subject Coverage
    Teaching English in Multilingual Contexts Selections from the Fifth AINET International Conference, Hyderabad, 10-11 January 2020 Krishna Dixit Vivek Joshi Ravinarayan Chakrakodi Nadeem Khan Kuheli Mukherjee Milind Mane AINET Association of English Teachers www.theainet.net Teaching English in Multilingual Contexts Editors Krishna Dixit Vivek Joshi Ravinarayan Chakrakodi Nadeem Khan Kuheli Mukherjee Milind Mane © 2021 AINET ISBN 978-81-930684-1-0 Published by AINET Association of English Teachers The views expressed in the articles contained herein are the personal views of the respective authors and do not necessarily reflect the views of the editors or of AINET. The authors have confirmed that the articles are their original work. Foreword We are pleased to bring to you this collection of papers presented at the 5th AINET International Conference held on 10-11 January 2020 at Vasavi College of Engineering, Hyderabad (India). This is the third in the series of conference selections published by the AINET after English Language Education: Understanding Change (2015) and Exploring Learners and Learning of English (2016). Built around the theme of English in Multilingual Contexts, the conference attracted over 400 delegates and saw over 125 paper presentations and 30 workshops. The multilingual board placed at the conference venue, inviting delegates to mention their languages, showed the presence of 100+ languages – Indian and non-Indian – at the conference. The collection is noteworthy as it presents what occupies teachers’ minds and hearts – opportunities for innovation and change, constraints and challenges, and strategies to overcome them. One common theme across all papers is the concern for learners and their effective learning.
    [Show full text]
  • Boletim CSP 7
    Boletim Pedagógico - CSP (7ª Edição parte 4) Estabeleça um método de estudo! O uso adequado de um método de estudo é favorável à aprendizagem Imagem 1* Escrito por Coordenadoria Sociopedagógica em 30 de abril de 2020. ​ ​ Para ter um melhor desempenho durante os estudos, você pode adotar um método que te ajude a desenvolver seu aprendizado, auxiliando no foco e melhor compreensão dos conteúdos. Para isso, veja abaixo qual método pode servir como ferramenta na sua rotina de estudante em casa. DESTAQUE DO DIA Aprendendo sobre os métodos de estudo: Sistema de Repetição Espaçada Este método é interessante quando precisamos memorizar uma nova informação, mas estamos sempre esquecendo. Quando aprendemos algo novo e não colocamos em prática, com o tempo, teremos dificuldade de nos lembrar do que aprendemos (conforme a curva do esquecimento apresentado na Imagem 2). Isso é natural, é como se ​ ​ nosso cérebro estivesse sempre fazendo uma faxina com as informações que considera desnecessária, ou seja, ele apaga as informações que não colocamos em uso ou que nunca revemos. Imagem 2** Para memorizar uma informação nova, portanto, é necessário que ocorra uma repetição de seu estudo. O ideal é revisitarmos os conteúdos novos com maior frequência no início do aprendizado. Conforme o tempo passa, podemos diminuir a frequência de seu estudo. Esse método de memorização é chamado de Sistema de Repetição Espaçada (SRE). Desde a década de 1930, este sistema sofreu inúmeros incrementos, principalmente com as transformações tecnológicas difundidas, a partir da década de 1980. Com isto, foi possível implantar o SRE em programas que agendam e organizam uma rotina de repetição, fazendo-a no tempo adequado para sua memorização.
    [Show full text]
  • Arxiv:1712.01856V2 [Stat.ML] 10 Mar 2018 with Provable Guarantees Have Been Largely Missing Until Very Recently [19, 22]
    Optimizing Human Learning Behzad Tabibian1,2, Utkarsh Upadhyay1, Abir De1, Ali Zarezade3, Bernhard Schölkopf2, and Manuel Gomez-Rodriguez1 1MPI for Software Systems, [email protected], [email protected], [email protected] 2MPI for Intelligent Systems Systems, [email protected], [email protected] 3Sharif University, [email protected] Abstract Spaced repetition is a technique for efficient memorization which uses repeated, spaced review of content to improve long-term retention. Can we find the optimal reviewing schedule to maximize the benefits of spaced repetition? In this paper, we introduce a novel, flexible representation of spaced repetition using the framework of marked temporal point processes and then address the above question as an optimal control problem for stochastic differential equations with jumps. For two well-known human memory models, we show that the optimal reviewing schedule is given by the recall probability of the content to be learned. As a result, we can then develop a simple, scalable online algorithm, Memorize, to sample the optimal reviewing times. Experiments on both synthetic and real data gathered from Duolingo, a popular language-learning online platform, show that our algorithm may be able to help learners memorize more effectively than alternatives. 1 Introduction Our ability to remember a piece of information depends critically on the number of times we have reviewed it and the time elapsed since the last review, as first shown by a seminal study by Ebbinghaus [10]. The effect of these two factors have been extensively investigated in the experimental psychology literature [9, 16], particularly in second language acquisition research [2, 5, 7, 21].
    [Show full text]
  • Enhancing Human Learning Via Spaced Repetition Optimization
    Enhancing human learning via spaced repetition optimization Behzad Tabibiana,b,1, Utkarsh Upadhyaya, Abir Dea, Ali Zarezadea, Bernhard Scholkopf¨ b, and Manuel Gomez-Rodrigueza aNetworks Learning Group, Max Planck Institute for Software Systems, 67663 Kaiserslautern, Germany; and bEmpirical Inference Department, Max Planck Institute for Intelligent Systems, 72076 Tubingen,¨ Germany Edited by Richard M. Shiffrin, Indiana University, Bloomington, IN, and approved December 14, 2018 (received for review September 3, 2018) Spaced repetition is a technique for efficient memorization which tion algorithms. However, most of the above spaced repetition uses repeated review of content following a schedule determined algorithms are simple rule-based heuristics with a few hard- by a spaced repetition algorithm to improve long-term reten- coded parameters (8), which are unable to fulfill this promise— tion. However, current spaced repetition algorithms are simple adaptive, data-driven algorithms with provable guarantees have rule-based heuristics with a few hard-coded parameters. Here, been largely missing until very recently (14, 15). Among these we introduce a flexible representation of spaced repetition using recent notable exceptions, the work most closely related to ours the framework of marked temporal point processes and then is by Reddy et al. (15), who proposed a queueing network model address the design of spaced repetition algorithms with prov- for a particular spaced repetition method—the Leitner system able guarantees as an optimal control
    [Show full text]
  • Ekadder Dergi Şubat 2021
    Ş U B A T 2 0 2 1 1 9 . S A Y I D F F E E D D E E L L U U D E R E K A EĞİTİMCİ KADINLAR DERNEĞİ KAPAK GÖRSEL ISBN 978-975-2490-53-6 https://www.canva.com/ İÇİNDEKİLER CAN ŞEHİR 4 ERZİNCAN KATKIDA BULUNANLAR DR. FERAH GÜÇLÜ YILMAZ BİR ÖNERİMİZ VAR BİR KİTAP SAADET BERNA OCAKCIOĞLU BİR FİLM SELVİHAN YALÇINER 13 AYŞE ERDOĞAN DİJİTAL DÖNÜŞÜM ÇAĞI VE HÜRRİYET DEĞERLİ ERMİŞ UZAKTAN EĞİTİM ELİF PELİN ERİŞ DÖNEMİNDE HİBRİT VE ÖĞRENME VE WEB2.0 15 ARAÇLARI EKADDER 11 Şubat Uluslararası Bilimde 21 Kadın ve Kız Çocukları Günü İSLAM BİLİM VE TEKNOLOJİ 23 TARİHİ MÜZESİ EKADDER BÜLTEN 2 EKADDER EĞİTİMCİ KADINLAR DERNEĞİ EKADDER BÜLTEN 3 "CAN ŞEHİR " ERZİNCAN Erzincan bu gün 210 bin nüfusu ile bir Anadolu şehri olarak hayatını sürdürüyor. Şehir merkez nüfusu 90 bin 100. Gümüşhane, Bayburt, Tunceli, Erzurum, Elazığ, Malatya, Bingöl ve Sivas'a sınırı bulunan şehrin Gümüşhane, Bayburt, Erzurum, Tunceli ve Sivas'a doğrudan ana karayolu ile bağlantısı bulunmaktadır. 11.903 kilometrekare yüzölçümüne sahip Erzincan, karasal iklimin hakim olduğu bir il olmasına rağmen bölge illerine kıyasla ılıman bir havaya sahiptir. Bu sebeple bölgede narenciye hariç hemen hemen her türlü meyve yetişmekte olup, tarım gelirlerinin önemli bir kısmını meyvecilik oluşturmaktadır. Fırat nehrinin önemli bir kolu olan Karasu Nehri Tercan ve Erzincan ovalarının sulanmasında büyük katkıya sahiptir. Ayrıca Karasu'yu besleyen küçük dere ve çaylar bölgenin sulak bir yapıya sahip olmasını sağlamaktadır. Kurulan ve kurulması planlanan sulama maksatlı barajlar da buna katkı sağlamaktadır. Önemli gölleri: Aygır, Yedi Göller, Ardıçlı Göl, Otlukbeli Gölü, Tercan ve Erzincan Baraj Gölleri'dir.
    [Show full text]
  • Enhancing Human Learning Via Spaced Repetition Optimization
    Enhancing human learning via spaced repetition optimization Behzad Tabibiana,b,1, Utkarsh Upadhyaya, Abir Dea, Ali Zarezadea, Bernhard Scholkopf¨ b, and Manuel Gomez-Rodrigueza aNetworks Learning Group, Max Planck Institute for Software Systems, 67663 Kaiserslautern, Germany; and bEmpirical Inference Department, Max Planck Institute for Intelligent Systems, 72076 Tubingen,¨ Germany Edited by Richard M. Shiffrin, Indiana University, Bloomington, IN, and approved December 14, 2018 (received for review September 3, 2018) Spaced repetition is a technique for efficient memorization which tion algorithms. However, most of the above spaced repetition uses repeated review of content following a schedule determined algorithms are simple rule-based heuristics with a few hard- by a spaced repetition algorithm to improve long-term reten- coded parameters (8), which are unable to fulfill this promise— tion. However, current spaced repetition algorithms are simple adaptive, data-driven algorithms with provable guarantees have rule-based heuristics with a few hard-coded parameters. Here, been largely missing until very recently (14, 15). Among these we introduce a flexible representation of spaced repetition using recent notable exceptions, the work most closely related to ours the framework of marked temporal point processes and then is by Reddy et al. (15), who proposed a queueing network model address the design of spaced repetition algorithms with prov- for a particular spaced repetition method—the Leitner system able guarantees as an optimal control
    [Show full text]
  • Towards Reliable Storage of 56-Bit Secrets in Human Memory
    View metadata, citation and similar papers at core.ac.uk brought to you by CORE provided by CiteSeerX Towards reliable storage of 56-bit secrets in human memory Joseph Bonneau Stuart Schechter Princeton University Microsoft Research Abstract out in the 90s [4] and NIST guidelines now presume all passwords are user-chosen [32]. Most banks have even Challenging the conventional wisdom that users cannot given up on expecting customers to memorize random remember cryptographically-strong secrets, we test the four-digits PINs [22]. hypothesis that users can learn randomly-assigned 56- We hypothesized that perceived limits on humans’ bit codes (encoded as either 6 words or 12 characters) ability to remember secrets are an artifact of today’s sys- through spaced repetition. We asked remote research tems, which provide users with a single brief opportunity participants to perform a distractor task that required log- during enrolment to permanently imprint a secret pass- ging into a website 90 times, over up to two weeks, with word into long-term memory. By contrast, modern theo- a password of their choosing. After they entered their ries of the brain posit that it is important to forget random chosen password correctly we displayed a short code (4 information seen once, with no connection to past expe- letters or 2 words, 18.8 bits) that we required them to rience, so as to avoid being overwhelmed by the constant type. For subsequent logins we added an increasing de- flow of new sensory information [10]. lay prior to displaying the code, which participants could avoid by typing the code from memory.
    [Show full text]