DOCSLIB.ORG

Continual Reinforcement Learning with Memory at Multiple Timescales

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Continual Reinforcement Learning with Memory at Multiple Timescales Imperial College Department of Computing Continual Reinforcement Learning with Memory at Multiple Timescales Christos Kaplanis Submitted in part fulfilment of the requirements for the degree of Doctor of Philosophy in Computing of the University of London and the Diploma of Imperial College, April 2020 Declaration of Originality and Copyright This is to certify that this thesis was composed solely by myself. Except where it is stated otherwise by reference or acknowledgment, the work presented is entirely my own. The copyright of this thesis rests with the author. Unless otherwise indicated, its contents are licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International Licence (CC BY NC-SA). Under this licence, you may copy and redistribute the material in any medium or format. You may also create and distribute modified versions of the work. This is on the condition that; you credit the author, do not use it for commercial purposes and share any derivative works under the same licence. When reusing or sharing this work, ensure you make the licence terms clear to others by naming the licence and linking to the licence text. Where a work has been adapted, you should indicate that the work has been changed and describe those changes. Please seek permission from the copyright holder for uses of this work that are not included in this licence or permitted under UK Copyright Law. i ii Abstract In the past decade, with increased availability of computational resources and several improve- ments in training techniques, artificial neural networks (ANNs) have been rediscovered as a powerful class of machine learning methods, featuring in several groundbreaking applications of artificial intelligence. Most of these successes have been achieved in stationary, confined domains, such as a game playing [MKS+15, SSS+17] and image recognition [KSH12], but, ul- timately, we want to apply artificial intelligence to problems that require it to interact with the real world, which is both vast and nonstationary. Unfortunately, ANNs have long been known to suffer from the phenomenon of catastrophic forgetting [MC89], whereby, in a setting where the data distribution is changing over time, new learning can lead to an abrupt erasure of previously acquired knowledge. The resurgence of ANNs has led to an increased urgency to solve this problem and endow them with the capacity for continual learning [Rin94], which refers to the ability to build on their knowledge over time in environments that are constantly evolving. The most common setting for evaluating continual learning approaches to date has been in the context of training on a number of distinct tasks in sequence, and as a result many of them use the knowledge of task boundaries to consolidate knowledge during training [RE13, KPR+17, ZPG17]. In the real world, however, the changes to the distribution may occur more gradually and at times that are not known in advance. The goal of this thesis has been to develop continual learning approaches that can cope with both discrete and continuous changes to the data distribution, without any prior knowledge of the nature or timescale of the changes. I present three new methods, all of which involve learning at multiple timescales, and evaluate them in the context of deep reinforcement learning, a paradigm that combines reinforcement learning [SB98] with neural networks, which provides a natural testbed for continual learning as (i) it involves interacting with an environment, and (ii) it can feature non-stationarity at unpredictable timescales during training of a single task. The first method is inspired by the process of synaptic consolidation in the brain and involves multi- timescale memory at the level of the parameters of the network; the second extends the first by directly consolidating the agent's policy over time, rather than its individual parameters; finally, the third approach extends the experience replay database [Lin92, MKS+15], which typically maintains a buffer of the agent's most recent experiences in order to decorrelate them during training, by enabling it to store data over multiple timescales. iii iv Acknowledgements I'd like to express my immense gratitude to my supervisors, Claudia and Murray, for all their support and guidance over the past few years. Because of them and all the time they dedicated to me, I've learnt invaluable lessons that have shaped my approach to research and I've really enjoyed the process. Beyond the many stimulating intellectual discussions, they really helped me navigate the psychological ups and downs of the PhD. The whole experience would not have been the same without all the people I shared it with at Imperial, namely all those who passed through Huxley 444 during my time there, the current and former members of the Clopath lab, and the `lunch buddies'. I will miss physics, the walks down the hall for terrible (but free) coffee, weekly lab meetings and JC, perpetual discussions about consciousness and AI, Ludwig, chess puzzles, the lab xmas activities, and much more. I'm lucky to have met all of you legends. I must also thank CSG for their invaluable technical help in times of need. I want to thank my parents, for their endless support and for giving me all the opportunities in life to get to this point, my sister, who could empathise with all the PhD-related struggles, and all my other family and friends who made life easier for me when research was tough. Finally, I am utterly beholden to Georgie, who has been an absolute rock for me every single day and in every aspect of life - I can't imagine any of it without her. Oh, and to our two dogs, Gus and Hector, whose presence has the handy effect of making any stress or worry evaporate instantaneously. v vi Dedication This thesis is dedicated to my παππού Qrisτάκη, who has been there for me despite my never having met him, and to my γιαγιά NÐtsa, who was so warm and clever and made a chocolate Rice Krispie cake that would reliably serve as a reminder of why life is worth living. vii `A good idea is a good idea forever.' David Brent (The Office, Season 1, Episode 4) viii Contents Declaration of Originality and Copyright i Abstract iii Acknowledgements v 1 Introduction 1 1.1 Continual Learning . 4 1.1.1 How do we define it? . 4 1.1.2 How do we measure it? . 7 1.1.3 Continual Reinforcement Learning . 8 1.1.4 Continual Reinforcement Learning with Memory at Multiple Timescales . 10 1.2 Thesis Outline and Contributions . 10 1.3 Publications . 12 2 Background 13 2.1 Artificial Neural Networks . 13 2.1.1 Origins of ANN . 14 2.1.2 Training Deep Neural Networks . 17 ix x CONTENTS 2.2 Reinforcement Learning . 23 2.2.1 Markov Decision Processes . 24 2.2.2 Nonstationarity in Reinforcement Learning . 26 2.2.3 Value Functions . 27 2.2.4 Value Function Approximation . 31 2.2.5 Policy Gradients . 33 2.3 Catastrophic Forgetting in ANN . 36 2.3.1 Discovery and Early Approaches . 37 2.3.2 Regularisation-based Methods . 39 2.3.3 Replay-based Methods . 43 2.3.4 Architectural Approaches . 45 2.3.5 Sparse Coding / Semi-distributed Representations . 47 2.3.6 Task-free Methods . 48 3 Continual Reinforcement Learning with Complex Synapses 51 3.1 Introduction . 51 3.2 Preliminaries . 52 3.2.1 The Benna-Fusi Model . 52 3.2.2 RL Algorithms . 54 3.3 Experiments . 57 3.3.1 Continual Q-learning . 57 3.3.2 Continual Multi-task Deep RL . 61 3.3.3 Continual Learning within a Single Task . 66 CONTENTS xi 3.4 Related Work . 68 3.5 Conclusion . 69 3.6 Experimental Details . 70 3.7 Additional Experiments . 72 3.7.1 Varying Epoch Lengths . 72 3.7.2 Three-task Experiments . 72 3.7.3 Varying Size of Replay Database . 74 3.7.4 Catcher Single Task . 75 3.7.5 Varying Final Exploration Value . 76 3.8 Online Convex Optimisation Perspective . 76 4 Policy Consolidation for Continual Reinforcement Learning 82 4.1 Introduction . 82 4.2 Preliminaries . 83 4.2.1 PPO . 83 4.2.2 Multi-agent RL with Competitive Self-play . 85 4.3 From Synaptic Consolidation to Policy Consolidation . 85 4.3.1 Synaptic Consolidation . 86 4.3.2 Policy Consolidation . 88 4.4 Experiments . 89 4.4.1 Single agent Experiments . 90 4.4.2 Multi-agent Experiments . 92 4.4.3 Further Analyses . 94 xii CONTENTS 4.5 Related Work . 98 4.6 Conclusion and Future Work . 98 4.7 Experimental Details . 99 4.7.1 Single agent Experiments . 99 4.7.2 Self-play Experiments . 100 4.8 Additional Experiments . 101 4.8.1 Directionality of KL constraint . 101 5 Continual Reinforcement Learning with Multi-Timescale Replay 103 5.1 Introduction . 103 5.2 Preliminaries . 105 5.2.1 Soft Actor-Critic . 105 5.2.2 Invariant Risk Minimisation . 107 5.3 Multi-Timescale Replay . 108 5.3.1 Power Law Forgetting . 110 5.3.2 IRM version of MTR . 111 5.4 Experiments . 112 5.5 Related Work . 120 5.6 Conclusion . 121 5.7 Experimental Details . 124 5.8 Additional Experiments . 124 5.8.1 Importance Sampling . 125 5.8.2 ReF-ER . 127 5.8.3 Behaviour Regularised Actor Critic . 129 5.9 Multi-task Experiments . 131 6 Conclusion 132 6.1 Limitations and Future Work . ..
Load more

Recommended publications
  • A Queueing-Theoretic Foundation for Optimal Spaced Repetition
    A Queueing-Theoretic Foundation for Optimal Spaced Repetition Siddharth Reddy [email protected] Department of Computer Science, Cornell University, Ithaca, NY 14850 Igor Labutov [email protected] Department of Electrical and Computer Engineering, Cornell University, Ithaca, NY 14850 Siddhartha Banerjee [email protected] School of Operations Research and Information Engineering, Cornell University, Ithaca, NY 14850 Thorsten Joachims [email protected] Department of Computer Science, Cornell University, Ithaca, NY 14850 1. Extended Abstract way back to 1885 and the pioneering work of Ebbinghaus (Ebbinghaus, 1913), identify two critical variables that de- In the study of human learning, there is broad evidence that termine the probability of recalling an item: reinforcement, our ability to retain a piece of information improves with i.e., repeated exposure to the item, and delay, i.e., time repeated exposure, and that it decays with delay since the since the item was last reviewed. Accordingly, scientists last exposure. This plays a crucial role in the design of ed- have long been proponents of the spacing effect for learn- ucational software, leading to a trade-off between teaching ing: the phenomenon in which periodic, spaced review of new material and reviewing what has already been taught. content improves long-term retention. A common way to balance this trade-off is spaced repe- tition, which uses periodic review of content to improve A significant development in recent years has been a grow- long-term retention. Though spaced repetition is widely ing body of work that attempts to ‘engineer’ the process used in practice, e.g., in electronic flashcard software, there of human learning, creating tools that enhance the learning is little formal understanding of the design of these sys- process by building on the scientific understanding of hu- tems.
    [Show full text]
  • Spaced Learning Strategy in Teaching Mathematics Ace T
    International Journal of Scientific & Engineering Research, Volume 8, Issue 4, April-2017 851 ISSN 2229-5518 Spaced Learning Strategy in Teaching Mathematics Ace T. Ceremonia, Remalyn Q. Casem Abstract—Students’ low mastery of the lesson in Mathematics is one of the alarming problems confronted by Mathematics teachers. It is in this light that this study was conceptualized to determine the effectiveness of spaced learning strategy on the performance and mastery of DMMMSU Laboratory High School students (Grade 7) in Mathematics. This study used the true experimental design, specifically the pretest-posttest control group design. The main instrument used to gather data was the pretest-posttest which was subjected to validity and reliability tests. It was found out that the experimental and control groups were comparable in the pretest and posttest. Comparison on their gain scores revealed significant difference with the performance of the experimental higher than the control group. It was also found out that the effect size of using the spaced learning strategy was large. This indicates that the intervention is effective in increasing the performance and mastery of high school students in Mathematics. It is recommended the use of the Spaced Learning Strategy to improve the performance of the high school students in Mathematics. Index Terms— basic education, effect size, mathematics performance, spaced learning strategy. —————————— —————————— 1 INTRODUCTION 1.1 Situation Analysis results, such as the 2003 TIMSS (Trends in International Mathematics and Science Study), the Philippines ranked 34th out uality education is reflected on the exemplary outputs and of 38 countries in HS II Math and ranked 43rd out of 46 countries Q outstanding performances of students in academics and in HS II Science; for grade 4, the Philippines ranked 23rd out of 25 related co-curricular activities (Glossary Education Reform, participating countries in both Math and Science.
    [Show full text]
  • Universidad De Almería
    UNIVERSIDAD DE ALMERÍA MÁSTER EN PROFESORADO DE EDUCACIÓN SECUNDARIA OBLIGATORIA Y BACHILLERATO, FORMACIÓN PROFESIONAL Y ENSEÑANZA DE IDIOMAS ESPECIALIDAD EN LENGUA INGLESA Curso Académico: 2015/2016 Convocatoria: Junio Trabajo Fin de Máster: Spaced Retrieval Practice Applied to Vocabulary Learning in Secondary Education Autor: Héctor Daniel León Romero Tutora: Susana Nicolás Román ABSTRACT Spaced retrieval practice is a learning technique which has been long studied (Ebbinghaus, 1885/1913; Gates, 1917) and long forgotten at the same time in education. It is based on the spacing and the testing effects. In recent reviews, spacing and retrieving practices have been highly recommended as there is ample evidence of their long-term retention benefits, even in educational contexts (Dunlosky, Rawson, Marsh, Nathan & Willingham, 2013). An experiment in a real secondary education classroom was conducted in order to show spaced retrieval practice effects in retention and student’s motivation. Results confirm the evidence, spaced retrieval practice showed higher long-term retention (26 days since first study session) of English vocabulary words compared to massed practice. Also, student’s motivation remained high at the end of the experiment. There is enough evidence to suggest educational institutions should promote the use of spaced retrieval practice in classrooms. RESUMEN La recuperación espaciada es una técnica de aprendizaje que se lleva estudiando desde hace muchos años (Ebbinghaus, 1885/1913; Gates, 1917) y que al mismo tiempo ha permanecido como una gran olvidada en los sistemas educativos. Se basa en los efectos que producen el repaso espaciado y el uso de test. En recientes revisiones de la literatura se promueve encarecidamente el uso de estas prácticas, ya que aumentan la retención de recuerdos en la memoria a largo plazo, incluso en contextos educativos (Dunlosky, Rawson, Marsh, Nathan & Willingham, 2013).
    [Show full text]
  • Learning Efficiency Correlates of Using Supermemo with Specially Crafted Flashcards in Medical Scholarship
    Learning efficiency correlates of using SuperMemo with specially crafted Flashcards in medical scholarship. Authors: Jacopo Michettoni, Alexis Pujo, Daniel Nadolny, Raj Thimmiah. Abstract Computer-assisted learning has been growing in popularity in higher education and in the research literature. A subset of these novel approaches to learning claim that predictive algorithms called Spaced Repetition can significantly improve retention rates of studied knowledge while minimizing the time investment required for learning. SuperMemo is a brand of commercial software and the editor of the SuperMemo spaced repetition algorithm. Medical scholarship is well known for requiring students to acquire large amounts of information in a short span of time. Anatomy, in particular, relies heavily on rote memorization. Using the SuperMemo web platform1 we are creating a non-randomized trial, inviting medical students completing an anatomy course to take part. Usage of SuperMemo as well as a performance test will be measured and compared with a concurrent control group who will not be provided with the SuperMemo Software. Hypotheses A) Increased average grade for memorization-intensive examinations If spaced repetition positively affects average retrievability and stability of memory over the term of one to four months, then consistent2 users should obtain better grades than their peers on memorization-intensive examination material. B) Grades increase with consistency There is a negative relationship between variability of daily usage of SRS and grades. 1 https://www.supermemo.com/ 2 Defined in Criteria for inclusion: SuperMemo group. C) Increased stability of memory in the long-term If spaced repetition positively affects knowledge stability, consistent users should have more durable recall even after reviews of learned material have ceased.
    [Show full text]
  • Serial Position Effects and Forgetting Curves: Implications in Word
    Studies in English Language Teaching ISSN 2372-9740 (Print) ISSN 2329-311X (Online) Vol. 2, No. 3, 2014 www.scholink.org/ojs/index.php/selt Original Paper Serial Position Effects and Forgetting Curves: Implications in Word Memorization Guijun Zhang1* 1 Department of Foreign Languages, China Pharmaceutical University, Nanjing, China * Guijun Zhang, E-mail:[email protected] Abstract Word memorization is important in English learning and teaching. The theory and implications of serial position effects and forgetting curves are discussed in this paper. It is held that they help students understand the psychological mechanisms underlying word memorization. The serial position effects make them to consider the application the chunking theory in word memorization; the forgetting curve reminds them to repeat the words in long-term memory in proper time. Meanwhile the spacing effect and elaborative rehearsal effect are also discussed as they are related to the forgetting curve. Keywords serial position effects, forgetting curves, word memorization 1. Introduction English words are extraordinarily significant for English foreign language (EFL) learners because they are the essential basis of all language skills. As Wilkins said, “...while without grammar very little can be conveyed, without vocabulary nothing can be conveyed” (Wilkins, 1972). Effective word memorization plays a significant role in the process of vocabulary learning. Researchers have and are still pursuing and summarizing the effective memory methods. Schmitt, for example, classified vocabulary memory strategies into more than twenty kinds (Schmitt, 1997, p. 34). However, it is hard to improve the efficiency of the vocabulary memory in that different students remember the huge amount of words with some certain method or methods that may not suit them.
    [Show full text]
  • Vocabulary Coverage and Lexical Characterisitics in L2 Spanish Textbooks
    AAUSC 2018 Volume—Issues in Language Program Direction Understanding Vocabulary Learning and Teaching: Implications for Language Program Development Johanna Watzinger-Tharp University of Utah Kate Paesani University of Minnesota Series Editors Peter Ecke University of Arizona Susanne Rott University of Illinois at Chicago Editors Australia • Brazil • Mexico • Singapore • United Kingdom • United States 06686_fm_ptg01_i-xviii.indd 1 10/3/18 1:52 PM AAUSC 2018 Volume: © 2020, 2019 Cengage Learning, Inc. Understanding Vocabulary Learning and Teaching: Implications for Language Unless otherwise noted, all content is © Cengage Program Development ALL RIGHTS RESERVED. No part of this work covered by Peter Ecke, Susanne Rott, the copyright herein may be reproduced or distributed Johanna Watzinger-Tharp in any form or by any means, except as permitted by and Kate Paesani U.S. copyright law, without the prior written permission of the copyright owner. Sr. Product Team Manager: Heather Bradley Cole For product information and technology assistance, Product Assistant: Catherine contact us at Cengage Customer & Sales Support, Bradley 1-800-354-9706 or support.cengage.com. Marketing Manager: Sean Ketchem For permission to use material from this text or product, submit all requests online at Production Service: Lumina www.cengage.com/permissions. Datamatics, Inc. Senior Designer: Sarah Cole Library of Congress Control Number: 2018953450 Manufacturing Planner: Fola Orekoya Student Edition: ISBN: 978-0-357-10668-6 Cengage 20 Channel Center Street Boston, MA 02210 USA Cengage is a leading provider of customized learning solutions with employees residing in nearly 40 different countries and sales in more than 125 countries around the world. Find your local representative at www.cengage.com Cengage products are represented in Canada by Nelson Education, Ltd.
    [Show full text]
  • Where Companies Go Wrong with Learning and Development by Steve Glaveski
    REPRINT H055J1 PUBLISHED ON HBR.ORG OCTOBER 02, 2019 ARTICLE BUSINESS EDUCATION Where Companies Go Wrong with Learning and Development by Steve Glaveski This article is made available to you with compliments of EGN for your personal use. Further posting, copying, or distribution is not permitted. BUSINESS EDUCATION Where Companies Go Wrong with Learning and Development by Steve Glaveski OCTOBER 02, 2019 GREGOR SCHUSTER/GETTY IMAGES Organizations spent $359 billion globally on training in 2016, but was it worth it? Not when you consider the following: • 75% of 1,500 managers surveyed from across 50 organizations were dissatisfied with their company’s Learning & Development (L&D) function; COPYRIGHT © 2019 HARVARD BUSINESS SCHOOL PUBLISHING CORPORATION. ALL RIGHTS RESERVED. 2 • 70% of employees report that they don’t have mastery of the skills needed to do their jobs; • Only 12% of employees apply new skills learned in L&D programs to their jobs; and • Only 25% of respondents to a recent McKinsey survey believe that training measurably improved performance. Not only is the majority of training in today’s companies ineffective, but the purpose, timing, and content of training is flawed. Learning for the Wrong Reasons Bryan Caplan, professor of economics at George Mason University, and author of The Case Against Education, says in his book that education often isn’t so much about learning useful job skills, but about people showing off, or “signaling.” Today’s employees often signal through continuous professional education (CPE) credits so that they can make a case for a promotion. L&D staff also signal their worth by meeting flawed KPIs, such as the total CPE credits employees earn, rather than focusing on the business impact created.
    [Show full text]
  • Teaching the Science of Learning Yana Weinstein1* , Christopher R
    Weinstein et al. Cognitive Research: Principles and Implications (2018) 3:2 Cognitive Research: Principles DOI 10.1186/s41235-017-0087-y and Implications TUTORIALREVIEW Open Access Teaching the science of learning Yana Weinstein1* , Christopher R. Madan2,3 and Megan A. Sumeracki4 Abstract The science of learning has made a considerable contribution to our understanding of effective teaching and learning strategies. However, few instructors outside of the field are privy to this research. In this tutorial review, we focus on six specific cognitive strategies that have received robust support from decades of research: spaced practice, interleaving, retrieval practice, elaboration, concrete examples, and dual coding. We describe the basic research behind each strategy and relevant applied research, present examples of existing and suggested implementation, and make recommendations for further research that would broaden the reach of these strategies. Keywords: Education, Learning, Memory, Teaching Significance educators who might be interested in integrating the Education does not currently adhere to the medical strategies into their teaching practice, (b) science of learn- model of evidence-based practice (Roediger, 2013). ing researchers who are looking for open questions to help However, over the past few decades, our field has made determine future research priorities, and (c) researchers in significant advances in applying cognitive processes to other subfields who are interested in the ways that princi- education. From this work, specific recommendations ples from cognitive psychology have been applied to can be made for students to maximize their learning effi- education. ciency (Dunlosky, Rawson, Marsh, Nathan, & Willing- While the typical teacher may not be exposed to this ham, 2013; Roediger, Finn, & Weinstein, 2012).
    [Show full text]
  • Nicolas Schweighofer's Curriculum Vitae
    CURRICULUM VITAE NICOLAS SCHWEIGHOFER, PhD PERSONAL INFORMATION University Address: Division of Biokinesiology and Physical Therapy University of Southern California Los Angeles, CA, 90089-9006 Phone: (1) 323 -442 -1838 Email: [email protected] Lab website: http://pt.usc.edu/labs/cnrl EDUCATION AND PROFESSIONAL APPOINTMENTS UNIVERSITY EDUCATION 1991-1995 PhD, University of Southern California, Los Angeles: Neuroscience, (Computational Neuroscience Emphasis) (Advisor: Dr. M. Arbib) 1988-1990 Diplôme d’Ingénieur, Ecole Nationale Supérieure de Mécanique, Nantes, France (since ‘Ecole Centrale de Nantes’): Mechanics and Control System Engineering 1985-1987 Undergraduate training: Mathématiques Spéciales P’, Lycée Descartes, Tours, France: Advanced Mathematics and Physics POST-GRADUATE TRAINING 1996-1997 Postdoctoral Fellow, Human Information Science, ATR, Kyoto, Japan: Computational Neuroscience (Mentor: Dr. M. Kawato) APPOINTMENTS 2011-Present Associate Professor (with Tenure), Division of Biokinesiology & Physical Therapy, University of Southern California 2011-Present Joint Appointment (courtesy), Department of Biomedical Engineering, Viterbi School of Engineering, University of Southern California 2004-Present Joint Appointment (courtesy), Department of Computer Science, Viterbi School of Engineering, University of Southern California 2004-Present Joint Appointment (courtesy), Neuroscience Graduate Program, University of Southern California 2004-2010 Assistant Professor (Tenure track), Division of Biokinesiology & Physical Therapy, University
    [Show full text]
  • Universidad Mayor De San Andres Facultad De Humanidades Y Ciencias De La Educación Carrera De Lingüística E Idiomas
    UNIVERSIDAD MAYOR DE SAN ANDRES FACULTAD DE HUMANIDADES Y CIENCIAS DE LA EDUCACIÓN CARRERA DE LINGÜÍSTICA E IDIOMAS COMPARING THE EFFECTIVENESS BETWEEN PAPER FLASHCARDS VERSUS FLASHCARDS SOFTWARE TO IMPROVE RECEPTIVE AND PRODUCTIVE KNOWLEDGE OF ENGLISH VOCABULARY Tesis de grado presentada para la obtención del Grado de Licenciatura POR: GUIDO ALVARO LOPEZ MAMANI TUTORA: M.Sc. LEIDY IBAÑEZ RODRIGUEZ LA PAZ – BOLIVIA 2020 I UNIVERSIDAD MAYOR DE SAN ANDRÉS FACULTAD DE HUMANIDADES Y CIENCIAS DE LA EDUCACIÓN CARRERA DE LINGÜÍSTICA E IDIOMAS Tesis de grado: COMPARING THE EFFECTIVENESS BETWEEN PAPER FLASHCARDS VERSUS FLASHCARD SOFTWARE TO IMPROVE RECEPTIVE AND PRODUCTIVE KNOWLEDGE OF ENGLISH VOCABULARY Presentada por: Guido Alvaro Lopez Mamani Para optar el grado académico de Licenciado en Lingüística e Idiomas Nota numeral: ...................................................................................................................... Nota literal: .......................................................................................................................... Ha sido.................................................................................................................................. Directora a.i. de carrera: ………………………………………………. Lic. Maria Virginia Ferrufino Loza Tutora: ………………………………………………. M.Sc. Leidy Ibañez Rodriguez Tribunal: ………………………………………………. Mg.Sc. Maria Eugenia Sejas Ralde Tribunal: ………………………………………………. Mg.Sc. David Aduviri Delgado La Paz 12 de noviembre de 2020 II DEDICATORY I dedicate this work to people
    [Show full text]
  • Modelling Language Learning of Duolingo Users
    Modelling language learning of Duolingo users Floor Dikker STUDENT NUMBER: 2017217 THESIS SUBMITTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF MASTER OF SCIENCE IN DATA SCIENCE & SOCIETY DEPARTMENT OF COGNITIVE SCIENCE & ARTIFICIAL INTELLIGENCE SCHOOL OF HUMANITIES AND DIGITAL SCIENCES TILBURG UNIVERSITY Thesis committee: Dr. Hendrickson Dr. Alishahi Tilburg University School of Humanities and Digital Sciences Department of Cognitive Science & Artificial Intelligence Tilburg, The Netherlands June, 2019 Data Science and Society 2019 2 Data Science and Society 2019 Preface Dear reader, I present to you my thesis about modelling language learning of users of Duolingo. First of all, I would like to thank Dr. Hendrickson for his guidance. I have found our meetings and your suggestions very helpful. Moreover, your enthusiasm is very contagious which made writing this thesis a more fun experience. I would also like to thank my parents and brother for offering support during my entire studies and always offering a listening ear. Finally, I would like to thank my boyfriend for supporting me all the way. I hope you enjoy reading my thesis, Floor Dikker 3 Data Science and Society 2019 Table of Contents 1. Introduction ....................................................................................................... 6 2. Related Work..................................................................................................... 9 2.1 The SLAM challenge ...................................................................................
    [Show full text]
  • The Concept of Personal Learning Pathway for Intelligent Tutoring System Geome
    Computer Science Journal of Moldova, vol.27, no.3(81), 2019 The concept of personal learning pathway for Intelligent Tutoring System GeoMe Caftanatov Olesea Abstract This paper presents a study of designing personal learning pathways for our intelligent tutoring system “GeoMe”. The pur- pose of the study is to define specific requirements for our ap- plication and conceptualize the workflow for personal learning pathways. Keywords: Learning pathway, intelligent tutoring system, space repetition, forgetting curve, Leitner system. 1 Introduction Digital culture is increasingly applied in e-learning; it also contributes to improve the educational process by adapting to the student’s inter- ests, capabilities and knowledge. Nowadays there are many different kinds of educational software for students based on adaptive learning, personalized learning or even personal learning pathway (PLP). Learn- ing pathway can be described as a route, taken by a pupil through a range of e-learning activities, which allows learners to get new skills and build knowledge progressively. Clement [1] defines a learning pathway as “The sequence of intermediate steps from preconceptions to target model form what Scott (1991) and Niedderer and Goldberg (1995) have called a learning pathway. For any particular topic, such a pathway would provide both a theory of instruction and a guideline for teachers and curriculum developers.” For students personal learning paths are the best solution, because they can more effectively acquire and retain knowledge and skills that c 2019 by CSJM; Caftanatov Olesea 355 Caftanatov Olesea will help them in real world. However, what about the elementary schoolchildren? For them it is harder to decide which learning model will be the most appropriate and effective.
    [Show full text]