DOCSLIB.ORG

Implicit Sequence Learning in Recurrent Neural Networks

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Implicit Sequence Learning in Recurrent Neural Networks INSTITUT FUR¨ INFORMATIONS- UND KOMMUNIKATIONSTECHNIK (IIKT) Implicit Sequence Learning in Recurrent Neural Networks DISSERTATION zur Erlangung des akademischen Grades Doktoringenieur (Dr.-Ing.) von Dipl.-Ing. Stefan Gluge¨ geb. am 16.07.1982 in Magdeburg, Deutschland genehmigt durch die Fakult¨at f¨ur Elektrotechnik und Informationstechnik der Otto-von-Guericke-Universit¨at Magdeburg Gutachter: Prof. Dr. rer. nat. Andreas Wendemuth Prof. Dr. G¨unther Palm Jun.-Prof. PD. Dr.-Ing. habil. Ayoub Al-Hamadi Promotionskolloquium am 11.10.2013 ii Ehrenerkl¨arung Ich versichere hiermit, dass ich die vorliegende Arbeit ohne unzul¨assige Hilfe Dritter und ohne Benutzung anderer als der angegebenen Hilfsmittel angefertigt habe. Die Hilfe eines kommerziellen Promotionsberaters habe ich nicht in Anspruch genommen. Dritte haben von mir weder unmittelbar noch mittelbar geldwerte Leistungen fr Arbeiten erhalten, die im Zusammenhang mit dem Inhalt der vorgelegten Dissertation stehen. Verwendete fremde und eigene Quellen sind als solche kenntlich gemacht. Ich habe insbesondere nicht wissentlich: Ergebnisse erfunden oder widerspr¨uchliche Ergebnisse verschwiegen, • statistische Verfahren absichtlich missbraucht, um Daten in ungerechtfertigter Weise • zu interpretieren, fremde Ergebnisse oder Ver¨offentlichungen plagiiert, • fremde Forschungsergebnisse verzerrt wiedergegeben. • Mir ist bekannt, dass Verst¨oße gegen das Urheberrecht Unterlassungs- und Schadenser- satzanspr¨uche des Urhebers sowie eine strafrechtliche Ahndung durch die Strafverfol- gungsbeh¨orden begr¨unden kann. Ich erkl¨are mich damit einverstanden, dass die Dissertation ggf. mit Mitteln der elek- tronischen Datenverarbeitung auf Plagiate ¨uberpr¨uft werden kann. Die Arbeit wurde bisher weder im Inland noch im Ausland in gleicher oder ¨ahnlicher Form als Dissertation eingereicht und ist als Ganzes auch noch nicht ver¨offentlicht. Magdeburg, den 26.06.2013 Dipl.-Ing. Stefan Gl¨uge Danksagung An dieser Stelle m¨ochte ich mich bei all denjenigen bedanken, die mich in den letzten Jahren bei der Arbeit an meiner Dissertation unterst¨utzt haben. Den gr¨oßten Anteil hat sicherlich Prof. Dr. Andreas Wendemuth, der mich wissen- schaftlich betreut hat. Zus¨atzlich zu den Ideen und Anregungen bez¨uglich der wissen- schaftlichen Fragen, hat er mit seiner offenen Art f¨ur ein Klima gesorgt, in dem man gern arbeitet. Prof. Dr. G¨unther Palm danke ich vor allem f¨ur die Bereitschaft, trotz der Bergen von Arbeit auf seinem Schreibtisch, meine Dissertation zu begutachten. Dasselbe gilt f¨ur Prof. Dr. Ayoub Al-Hamadi, der sich ebenfalls bereit erkl¨art hat als Gutachter zu fungieren. Des Weiteren danke ich meinen Kollegen am Lehrstuhl f¨ur Kognitive Systeme an der Otto-von-Guericke Universit¨at. Besonders erw¨ahnen m¨ochte ich die tolle Zusammenar- beit und die vielen interessanten Diskussionen und Projekte mit Ronald B¨ock. Der Otto-von-Guericke Universit¨at und dem Land Sachsen-Anhalt danke ich f¨ur die finanzielle Unterst¨utzung w¨ahrend der Promotion. Letztendlich gilt mein liebster und innigster Dank meiner Frau Jule und meiner Fam- ilie, die immer an mich glauben. Ohne ihre Unterst¨utzung w¨are diese Arbeit nicht m¨oglich gewesen. iv Abstract This thesis investigates algorithmic models of implicit learning, and presents new meth- ods and related experiments in this field. Implicit learning is a common method of acquiring knowledge and therefore happens in childhood development and in everyday life. It can be shortly defined as incidental learning without awareness of the learned matter. As this is a highly desirable feature in machine learning, it has many applications in computational neuroscience, engineering applications and data analysis. The first part of this thesis is focused on cognitive modelling of implicit sequence learning as it was observed in behavioural experiments with human subjects. The ex- perimental setup is known as conditional associative learning scenario. Insights gained in this process are then used in the second part of this work. Here, the implicit learning of sequential information by recurrent neural networks is investigated in the context of machine learning. For cognitive modelling a Markov model is used to analyse the explicit part of the associative learning task which was given to the subjects. Thereafter, simple recurrent networks are applied to model the implicit learning of temporal dependencies that oc- curred in the experiments. Therefore, the development and storage of representations of temporal context in the networks is further investigated. Recurrent networks are a common tool in cognitive modelling, but even more an important method in the machine learning domain. Whenever it comes to sequence processing the capability of these networks is of great interest. One particular prob- lem in that area of research is the learning of long-term dependencies, which can be traced back to the problem of vanishing error gradients in gradient based learning. In my thesis I investigate the capabilities of a hierarchical recurrent network architecture, the Segmented-Memory Recurrent Neural Network, to circumvent this problem. The architecture itself is inspired by the process of memorisation of long sequences observed in humans. An extended version of a common learning algorithm adapted to this ar- chitecture is introduced and compared to an existing one concerning computational complexity and learning capability. Further, an unsupervised pre-training procedure for the recurrent networks is introduced that is motivated by the research in the field of deep learning. The learning algorithm proposed in this thesis dramatically reduces the computational complexity of the network training. This advantage is paid with a reduction of the time span between inputs and outputs that my be bridged by the network. However, this loss can be compensated by the application of a pre-training procedure. In general, recurrent networks are of interest in cognitive modelling, but in fact, this leads to their application in rather technical sequence classification and prediction tasks. This work shows, how those networks learn task-irrelevant temporal dependencies implic- itly, and presents progress which is made to make them applicable to machine learning and information engineering problems. v Kurzfassung Diese Arbeit untersucht algorithmische Modelle des impliziten Lernens und pr¨asentiert neue Methoden, sowie Experimente, auf diesem Feld. Implizites Lernen ist ein Weg des Wissenserwerbs und passiert im allt¨aglichen Leben, vor allem in der Kindheit. Es wird kurz als zuf¨alliges Lernen, ohne Bewusstsein f¨ur das Gelernte, definiert. Diese Eigenschaft ist auch f¨ur technische Systeme interessant und findet unter anderem Anwendung in der Neuroinformatik, den Ingenieurswissenschaften und im Bereich der Datenanalyse. Der erste Teil dieser Arbeit befasst sich mit der kognitiven Modellierung von im- plizitem Lernen von Sequenzen, wie es in einem Verhaltensexperiment mit Versuchsper- sonen beobachtet wurde. Derartige Experimente werden in der Kognitionsbiologie zur Untersuchung von konditionellem assoziativen Lernen genutzt. Die durch die Model- lierung gewonnen Einsichten, werden im zweiten Teil der Arbeit weiter verwendet. Dort wird, im Kontext des maschinellen Lernens, das implizite Lernen von Zeitabh¨angigkeiten durch rekurrente neuronale Netze analysiert. Um den expliziten Teil der assoziativen Lernaufgabe zu analysieren, wird ein Markov Modell vorgeschlagen. Danach werden einfache rekurrente Netze (simple recurrent net- works) genutzt, um den impliziten Teil des Lernens von Zeitabh¨angigkeiten, wie sie im Experiment auftraten, zu modellieren. Hierzu wird vor allem die Entwicklung und Speicherung von Repr¨asentationen des zeitlichen Kontextes in den Netzen untersucht. Rekurrente Netze sind ein gutes Werkzeug f¨ur die Modellierung kognitiver Prozesse, aber ein ebenso wichtiges Werkzeug im Bereich des maschinellen Lernens. Vor allem beim Verarbeiten von Informationssequenzen sind die Eigenschaften dieser Netze von großem Interesse. Ein bedeutendes Problem in dieser Dom¨ane ist das Lernen von Langzeitab- h¨angigkeiten, welches auf das Abnehmen der Fehlergradienten beim gradientenbasierten Lernen (vanishing gradient problem) zur¨uckgef¨uhrt werden kann. In meiner Arbeit unter- suche ich, in wie weit ein hierarchisches rekurrentes Netz (Segmented-Memory Recurrent Neural Network) das vanishing gradient problem umgehen kann. Die Netzarchitektur ist inspiriert durch die Art und weise, wie sich Menschen l¨angere Sequenzen merken. F¨ur die Architektur wird eine erweiterte Version eines bekannten Lernalgorithmus vorgeschlagen (BPTT), und bez¨uglich des Rechenaufwandes und der F¨ahigkeit Langzeitabh¨angigkei- ten zu lernen untersucht. Außerdem wird ein nicht ¨uberwachtes Vortraining beschrieben, welches durch die Forschung im Bereich der tiefen neuronalen Netze (deep learning) in- spiriert ist. Der in dieser Arbeit vorgeschlagene Lernalgorithmus verringert den Rechenaufwand f¨ur das Netzwerktraining erheblich. Die Zeitspanne zwischen Ein- und Ausgangssignalen des Netzes, die ¨uberbr¨uckt werden kann, ist jedoch kleiner als bei einem etablierten Algorithmus. Dieser Nachteil kann weitestgehend durch das Vortraining kompensiert werden. Im Allgemeinen haben rekurrente Netze interessante Eigenschaft f¨ur kognitive Mod- elle, aber im Speziellen werden diese oft in technischen Sequenzverarbeitungsaufgaben angewandt. Diese Arbeit zeigt, wie die Netze zeitliche Zusammenh¨ange implizit lernen und tr¨agt dazu
Load more

Recommended publications
  • A Distributed Computational Cognitive Model for Object Recognition
    SCIENCE CHINA xx xxxx Vol. xx No. x: 1{15 doi: A Distributed Computational Cognitive Model for Object Recognition Yong-Jin Liu1∗, Qiu-Fang Fu2, Ye Liu2 & Xiaolan Fu2 1Tsinghua National Laboratory for Information Science and Technology, Department of Computer Science and Technology, Tsinghua University, Beijing 100084, China, 2State Key Lab of Brain and Cognitive Science, Institute of Psychology, Chinese Academy of Sciences, Beijing 100101, China Received Month Day, Year; accepted Month Day, Year Abstract Based on cognitive functionalities in human vision processing, we propose a computational cognitive model for object recognition with detailed algorithmic descriptions. The contribution of this paper is of two folds. Firstly, we present a systematic review on psychological and neurophysiological studies, which provide collective evidence for a distributed representation of 3D objects in the human brain. Secondly, we present a computational model which simulates the distributed mechanism of object vision pathway. Experimental results show that the presented computational cognitive model outperforms five representative 3D object recognition algorithms in computer science research. Keywords distributed cognition, computational model, object recognition, human vision system 1 Introduction Object recognition is one of fundamental tasks in computer vision. Many recognition algorithms have been proposed in computer science area (e.g., [1, 2, 3, 4, 5]). While the CPU processing speed can now be reached at 109 Hz, the human brain has a limited speed of 100 Hz at which the neurons process their input. However, compared to state-of-the-art computational algorithms for object recognition, human brain has a distinct advantage in object recognition, i.e., human being can accurately recognize one from unlimited variety of objects within a fraction of a second, even if the object is partially occluded or contaminated by noises.
    [Show full text]
  • The Place of Modeling in Cognitive Science
    The place of modeling in cognitive science James L. McClelland Department of Psychology and Center for Mind, Brain, and Computation Stanford University James L. McClelland Department of Psychology Stanford University Stanford, CA 94305 650-736-4278 (v) / 650-725-5699 (f) [email protected] Running Head: Modeling in cognitive science Keywords: Modeling frameworks, computer simulation, connectionist models, Bayesian approaches, dynamical systems, symbolic models of cognition, hybrid models, cognitive architectures Abstract I consider the role of cognitive modeling in cognitive science. Modeling, and the computers that enable it, are central to the field, but the role of modeling is often misunderstood. Models are not intended to capture fully the processes they attempt to elucidate. Rather, they are explorations of ideas about the nature of cognitive processes. As explorations, simplification is essential – it is only through simplification that we can fully understand the implications of the ideas. This is not to say that simplification has no downsides; it does, and these are discussed. I then consider several contemporary frameworks for cognitive modeling, stressing the idea that each framework is useful in its own particular ways. Increases in computer power (by a factor of about 4 million) since 1958 have enabled new modeling paradigms to emerge, but these also depend on new ways of thinking. Will new paradigms emerge again with the next 1,000-fold increase? 1. Introduction With the inauguration of a new journal for cognitive science, thirty years after the first meeting of the Cognitive Science Society, it seems essential to consider the role of computational modeling in our discipline.
    [Show full text]
  • A Bayesian Computational Cognitive Model ∗
    A Bayesian Computational Cognitive Model ∗ Zhiwei Shi Hong Hu Zhongzhi Shi Key Laboratory of Intelligent Information Processing Institute of Computing Technology, CAS, China Kexueyuan Nanlu #6, Beijing, China 100080 {shizw, huhong, shizz}@ics.ict.ac.cn Abstract problem solving capability and provide meaningful insights into mechanisms of human intelligence. Computational cognitive modeling has recently Besides the two categories of models, recently, there emerged as one of the hottest issues in the AI area. emerges a novel approach, hybrid neural-symbolic system, Both symbolic approaches and connectionist ap- which concerns the use of problem-specific symbolic proaches present their merits and demerits. Al- knowledge within the neurocomputing paradigm [d'Avila though Bayesian method is suggested to incorporate Garcez et al., 2002]. This new approach shows that both advantages of the two kinds of approaches above, modal logics and temporal logics can be effectively repre- there is no feasible Bayesian computational model sented in artificial neural networks [d'Avila Garcez and concerning the entire cognitive process by now. In Lamb, 2004]. this paper, we propose a variation of traditional Different from the hybrid approach above, we adopt Bayesian network, namely Globally Connected and Bayesian network [Pearl, 1988] to integrate the merits of Locally Autonomic Bayesian Network (GCLABN), rule-based systems and neural networks, due to its virtues as to formally describe a plausible cognitive model. follows. On the one hand, by encoding concepts
    [Show full text]
  • Connectionist Models of Language Processing
    Cognitive Studies Preprint 2003, Vol. 10, No. 1, 10–28 Connectionist Models of Language Processing Douglas L. T. Rohde David C. Plaut Massachusetts Institute of Technology Carnegie Mellon University Traditional approaches to language processing have been based on explicit, discrete represen- tations which are difficult to learn from a reasonable linguistic environment—hence, it has come to be accepted that much of our linguistic representations and knowledge is innate. With its focus on learning based upon graded, malleable, distributed representations, connectionist modeling has reopened the question of what could be learned from the environment in the ab- sence of detailed innate knowledge. This paper provides an overview of connectionist models of language processing, at both the lexical and sentence levels. Although connectionist models have been applied to the full and processes of actual language users: Language is as lan- range of perceptual, cognitive, and motor domains (see Mc- guage does. In this regard, errors in performance (e.g., “slips Clelland, Rumelhart, & PDP Research Group, 1986; Quin- of the tongue”; Dell, Schwartz, Martin, Saffran, & Gagnon, lan, 1991; McLeod, Plunkett, & Rolls, 1998), it is in their 1997) are no less valid than skilled language use as a measure application to language that they have evoked the most in- of the underlying nature of language processing. The goal is terest and controversy (e.g., Pinker & Mehler, 1988). This not to abstract away from performance but to articulate com- is perhaps not surprising in light of the special role that lan- putational principles that account for it. guage plays in human cognition and culture.
    [Show full text]
  • Cognitive Psychology for Deep Neural Networks: a Shape Bias Case Study
    Cognitive Psychology for Deep Neural Networks: A Shape Bias Case Study Samuel Ritter * 1 David G.T. Barrett * 1 Adam Santoro 1 Matt M. Botvinick 1 Abstract to think of these models as black boxes, and to question whether they can be understood at all (Bornstein, 2016; Deep neural networks (DNNs) have advanced Lipton, 2016). This opacity obstructs both basic research performance on a wide range of complex tasks, seeking to improve these models, and applications of these rapidly outpacing our understanding of the na- models to real world problems (Caruana et al., 2015). ture of their solutions. While past work sought to advance our understanding of these models, Recent pushes have aimed to better understand DNNs: none has made use of the rich history of problem tailor-made loss functions and architectures produce more descriptions, theories, and experimental methods interpretable features (Higgins et al., 2016; Raposo et al., developed by cognitive psychologists to study 2017) while output-behavior analyses unveil previously the human mind. To explore the potential value opaque operations of these networks (Karpathy et al., of these tools, we chose a well-established analy- 2015). Parallel to this work, neuroscience-inspired meth- sis from developmental psychology that explains ods such as activation visualization (Li et al., 2015), abla- how children learn word labels for objects, and tion analysis (Zeiler & Fergus, 2014) and activation maxi- applied that analysis to DNNs. Using datasets mization (Yosinski et al., 2015) have also been applied. of stimuli inspired by the original cognitive psy- Altogether, this line of research developed a set of promis- chology experiments, we find that state-of-the-art ing tools for understanding DNNs, each paper producing one shot learning models trained on ImageNet a glimmer of insight.
    [Show full text]
  • Cognitive Modeling, Symbolic
    Cognitive modeling, symbolic Lewis, R.L. (1999). Cognitive modeling, symbolic. In Wilson, R. and Keil, F. (eds.), The MIT Encyclopedia of the Cognitive Sciences. Cambridge, MA: MIT Press. Symbolic cognitive models are theories of human cognition that take the form of working computer programs. A cognitive model is intended to be an explanation of how some aspect of cognition is accomplished by a set of primitive computational processes. A model performs a specific cognitive task or class of tasks and produces behavior that constitutes a set of predictions that can be compared to data from human performance. Task domains that have received considerable attention include problem solving, language comprehension, memory tasks, and human-device interaction. The scientific questions cognitive modeling seeks to answer belong to cognitive psychology, and the computational techniques are often drawn from artificial intelligence. Cognitive modeling differs from other forms of theorizing in psychology in its focus on functionality and computational completeness. Cognitive modeling produces both a theory of human behavior on a task and a computational artifact that performs the task. The theoretical foundation of cognitive modeling is the idea that cognition is a kind of COMPUTATION (see also COMPUTATIONAL THEORY OF MIND). The claim is that what the mind does, in part, is perform cognitive tasks by computing. (The claim is not that the computer is a metaphor for the mind, or that the architectures of modern digital computers can give us insights into human mental architecture.) If this is the case, then it must be possible to explain cognition as a dynamic unfolding of computational processes.
    [Show full text]
  • Connectionist Models of Cognition Michael S. C. Thomas and James L
    Connectionist models of cognition Michael S. C. Thomas and James L. McClelland 1 1. Introduction In this chapter, we review computer models of cognition that have focused on the use of neural networks. These architectures were inspired by research into how computation works in the brain and subsequent work has produced models of cognition with a distinctive flavor. Processing is characterized by patterns of activation across simple processing units connected together into complex networks. Knowledge is stored in the strength of the connections between units. It is for this reason that this approach to understanding cognition has gained the name of connectionism. 2. Background Over the last twenty years, connectionist modeling has formed an influential approach to the computational study of cognition. It is distinguished by its appeal to principles of neural computation to inspire the primitives that are included in its cognitive level models. Also known as artificial neural network (ANN) or parallel distributed processing (PDP) models, connectionism has been applied to a diverse range of cognitive abilities, including models of memory, attention, perception, action, language, concept formation, and reasoning (see, e.g., Houghton, 2005). While many of these models seek to capture adult function, connectionism places an emphasis on learning internal representations. This has led to an increasing focus on developmental phenomena and the origins of knowledge. Although, at its heart, connectionism comprises a set of computational formalisms,
    [Show full text]
  • Artificial Intelligence and Cognitive Science Have the Same Problem
    Artificial Intelligence and Cognitive Science have the Same Problem Nicholas L. Cassimatis Department of Cognitive Science, Rensselaer Polytechnic Institute 108 Carnegie, 110 8th St. Troy, NY 12180 [email protected] Abstract other features of human-level cognition. Therefore, not Cognitive scientists attempting to explain human understanding how the relatively simple and “unintelligent” intelligence share a puzzle with artificial intelligence mechanisms of atoms and molecules combine to create researchers aiming to create computers that exhibit human- intelligent behavior is a major challenge for a naturalistic level intelligence: how can a system composed of relatively world view (upon which much cognitive science is based). unintelligent parts (such as neurons or transistors) behave Perhaps it is the last major challenge. Surmounting the intelligently? I argue that although cognitive science has human-level intelligence problem also has enormous made significant progress towards many of its goals, that technological benefits which are obvious enough. solving the puzzle of intelligence requires special standards and methods in addition to those already employed in cognitive science. To promote such research, I suggest The State of the Science creating a subfield within cognitive science called For these reasons, understanding how the human brain intelligence science and propose some guidelines for embodies a solution to the human-level intelligence research addressing the intelligence puzzle. problem is an important goal of cognitive science. At least at first glance, we are nowhere near achieving this goal. There are no cognitive models that can, for example, fully The Intelligence Problem understand language or solve problems that are simple for a Cognitive scientists attempting to fully understand human young child.
    [Show full text]
  • The Cognitive Neuroscience of Music
    THE COGNITIVE NEUROSCIENCE OF MUSIC Isabelle Peretz Robert J. Zatorre Editors OXFORD UNIVERSITY PRESS Zat-fm.qxd 6/5/03 11:16 PM Page i THE COGNITIVE NEUROSCIENCE OF MUSIC This page intentionally left blank THE COGNITIVE NEUROSCIENCE OF MUSIC Edited by ISABELLE PERETZ Départment de Psychologie, Université de Montréal, C.P. 6128, Succ. Centre-Ville, Montréal, Québec, H3C 3J7, Canada and ROBERT J. ZATORRE Montreal Neurological Institute, McGill University, Montreal, Quebec, H3A 2B4, Canada 1 Zat-fm.qxd 6/5/03 11:16 PM Page iv 1 Great Clarendon Street, Oxford Oxford University Press is a department of the University of Oxford. It furthers the University’s objective of excellence in research, scholarship, and education by publishing worldwide in Oxford New York Auckland Bangkok Buenos Aires Cape Town Chennai Dar es Salaam Delhi Hong Kong Istanbul Karachi Kolkata Kuala Lumpur Madrid Melbourne Mexico City Mumbai Nairobi São Paulo Shanghai Taipei Tokyo Toronto Oxford is a registered trade mark of Oxford University Press in the UK and in certain other countries Published in the United States by Oxford University Press Inc., New York © The New York Academy of Sciences, Chapters 1–7, 9–20, and 22–8, and Oxford University Press, Chapters 8 and 21. Most of the materials in this book originally appeared in The Biological Foundations of Music, published as Volume 930 of the Annals of the New York Academy of Sciences, June 2001 (ISBN 1-57331-306-8). This book is an expanded version of the original Annals volume. The moral rights of the author have been asserted Database right Oxford University Press (maker) First published 2003 All rights reserved.
    [Show full text]
  • Models of Cognition: Neurological Possiblity Does Not Indicate Neurological Plausibility
    UC Merced Proceedings of the Annual Meeting of the Cognitive Science Society Title Models of Cognition: Neurological Possiblity Does Not Indicate Neurological Plausibility Permalink https://escholarship.org/uc/item/26t9426g Journal Proceedings of the Annual Meeting of the Cognitive Science Society, 27(27) ISSN 1069-7977 Authors Kriete, Trenton E. Noelle, David C. Publication Date 2005 Peer reviewed eScholarship.org Powered by the California Digital Library University of California Models of Cognition: Neurological possibility does not indicate neurological plausibility. Peter R. Krebs ([email protected]) Cognitive Science Program School of History & Philosophy of Science The University of New South Wales Sydney, NSW 2052, Australia Abstract level cognitive functions like the detection of syntactic and semantic features for words (Elman, 1990, 1993), Many activities in Cognitive Science involve complex learning the past tense of English verbs (Rumelhart and computer models and simulations of both theoretical and real entities. Artificial Intelligence and the study McClelland, 1996), or cognitive development (McLeod of artificial neural nets in particular, are seen as ma- et al., 1998; Schultz, 2003). SRNs have even been jor contributors in the quest for understanding the hu- suggested as a suitable platform \toward a cognitive man mind. Computational models serve as objects of neurobiology of the moral virtues" (Churchland, 1998). experimentation, and results from these virtual experi- ments are tacitly included in the framework of empiri- While some of the models go back a decade or more, cal science. Cognitive functions, like learning to speak, there is still great interest in some of these `classics', or discovering syntactical structures in language, have and similar models are still being developed, e.g.
    [Show full text]
  • Artificial Neural Nets
    Artificial Neural Nets: a critical analysis of their effectiveness as empirical technique for cognitive modelling Peter R. Krebs A thesis submitted in fulfillment of the requirements for the degree of Doctor of Philosophy. University of New South Wales August 2007 COPYRIGHT, AUTHENTICITY, and ORIGINALITY STATEMENT I hereby grant the University of New South Wales or its agents the right to archive and to make available my thesis or dissertation in whole or part in the University libraries in all forms of media, now or here after known, subject to the provisions of the Copyright Act 1968. I retain all proprietary rights, such as patent rights. I also retain the right to use in future works (such as articles or books) all or part of this thesis or dissertation. I also authorise University Microfilms to use the 350 word abstract of my thesis in Dissertation Abstract International. I have either used no substantial portions of copyright material in my thesis or I have obtained permission to use copyright material; where permission has not been granted I have applied/will apply for a partial restric- tion of the digital copy of my thesis or dissertation, and I certify that the Library deposit digital copy is a direct equivalent of the final officially approved version of my thesis. No emendation of content has occurred and if there are any minor variations in formatting, they are the result of the conversion to digital format, and I hereby declare that this submission is my own work and to the best of my knowledge it contains no materials previously published or written by another person, or substantial proportions of material which have been accepted for the award of any other degree or diploma at UNSW or any other educational institution, except where due acknowledgment is made in the thesis.
    [Show full text]
  • Why Digit Span Measures Long-Term Associative Learning
    Cognition 144 (2015) 1–13 Contents lists available at ScienceDirect Cognition journal homepage: www.elsevier.com/locate/COGNIT Questioning short-term memory and its measurement: Why digit span measures long-term associative learning ⇑ Gary Jones a, , Bill Macken b a Nottingham Trent University, UK b Cardiff University, UK article info abstract Article history: Traditional accounts of verbal short-term memory explain differences in performance for different types Received 23 December 2014 of verbal material by reference to inherent characteristics of the verbal items making up memory Revised 13 July 2015 sequences. The role of previous experience with sequences of different types is ostensibly controlled Accepted 15 July 2015 for either by deliberate exclusion or by presenting multiple trials constructed from different random per- mutations. We cast doubt on this general approach in a detailed analysis of the basis for the robust find- ing that short-term memory for digit sequences is superior to that for other sequences of verbal material. Keywords: Specifically, we show across four experiments that this advantage is not due to inherent characteristics of Digit span digits as verbal items, nor are individual digits within sequences better remembered than other types of Associative learning Short-term memory individual verbal items. Rather, the advantage for digit sequences stems from the increased frequency, Long-term memory compared to other verbal material, with which digits appear in random sequences in natural language, Sequence learning and furthermore, relatively frequent digit sequences support better short-term serial recall than less fre- Computational modelling quent ones. We also provide corpus-based computational support for the argument that performance in a short-term memory setting is a function of basic associative learning processes operating on the linguis- tic experience of the rememberer.
    [Show full text]