DOCSLIB.ORG

Connectionism, Confusion, and Cognitive Science MICHAEL R.W

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Connectionism, Confusion, and Cognitive Science MICHAEL R.W Connectionism, Confusion, and Cognitive Science MICHAEL R.W. DAWSON and KEVIN S. SHAMANSKI Biological Computation Project, University of Alberta CONTENTS Page Synopsis 216 Introduction 216 PDP Networks and the Tri-Level Hypothesis 218 Computational Descriptions of PDP Networks 221 PDP Networks Are Powerful Information Processing Systems 221 How Are Claims About PDP Competence Related to Cognitive Science? 225 Computational Connectionism and Cognitive Science 228 Algorithmic Descriptions of PDP Networks 229 PDP Networks Are Themselves Algorithms 229 Problems With PDP Algorithms 230 Algorithmic Connectionism and Cognitive Science 235 Implementational Descriptions of PDP Networks 238 PDP Networks, Biological Plausibility, and Computational Relevance 238 Problems Arise Because of Biologically Undefended Design Decisions 241 Implementational Connectionism and Cognitive Science 249 From Connectionism to Cognitive Science 250 Acknowledgements 252 References 253 215 Volume 4, Nos. 3-4, 1994 Connectionism, Confusion and Cognitive Science SYNOPSIS This paper argues that while connectionist technology may be flourishing, connectionist cognitive science is languishing. Parallel distributed processing (PDP) networks can be proven to be computationally powerful, but these proofs offer few useful constraints for developing models in cognitive science. Connectionist algorithms — that is, the PDP networks themselves — can exhibit interesting behaviours, but are difficult to interpret and are based upon an incomplete functional architecture. While PDP networks are often touted as being more "biologically plausible than classical AI models, they do not appear to have been widely endorsed by neurophysiologists because they incorporate many implausible assumptions, and they may not model appropriate physiological processes. In our view, connectionism faces such problems because the design decisions governing current connectionist theory are determined by engineering needs — generating the appropriate output — and not by cognitive or neurophysiological considerations. As a result, the true nature of connectionist theories, and their potential contribution to cognitive science, is unclear. We propose that the current confusion surrounding connectionism's role in cognitive science could be greatly alleviated by adopting a research programme in which connectionists paid much more attention to validating the PDP architecture. Key Words Parallel distributed processing, connectionism, cognitive science INTRODUCTION Connectionists appear to be making great advances in the technology of knowledge engineering, and now feel poised to answer the difficult questions about machine intelligence that seem to have passed classical artificial intelligence by. Parallel distributed processing (PDP) models have been developed for a diverse range of phenomena, as a survey of almost any journal related to cognitive science will show. For example, in recent years 216 M.R. W. Dawson and K.S. Shamanski Journal of Intelligent Systems Psychological Review has published connectionist models concerned with aspects of reading (Hinton & Shallice, 1991; Seidenberg & McClelland, 1989), classical learning theory (Kehoe, 1988), automatic processing (Cohen, Dunbar, & McClelland, 1991), sentence production (Dell, 1986), apparent Motion (Dawson, 1991), and dreaming (Antrobus, 1991). In addition, many basic connectionist ideas are being directly implemented in hardware (e.g., Jabri & Flower, 1991) under the assumption that increases in computer power and speed require radical new parallel architectures (e.g., Hillis, 1985; Müller & Reinhardt, 1990, p. 17). "The neural network revolution has happened. We are living in the aftermath" (Hanson & Olson, 1991, p. 332). While connectionist technology may indeed be flourishing, connectionist cognitive science is languishing. PDP networks may generate interesting behaviour, but it is not clear that they do so by emulating the fundamental nature of human cognitive processes. In our view, the current design decisions governing connectionist theory are determined by engineering needs — generating the appropriate output — and not by cognitive or neurophysiological considerations. The goal of this paper is to illustrate the gap between connectionist technology and connectionist cognitive science. Connectionist networks can be proven to be computationally powerful, but these proofs offer no meaningful constraints for designing cognitive models. Connectionist algorithms — that is, the PDP networks themselves — can exhibit interesting behaviours, but are difficult to interpret and are based upon an insufficient functional architecture. While PDP networks are often touted as being more "biologically plausible" than classical or symbolic artificial intelligence (AI) models, they do not appear to have been widely endorsed by neurophysiologists because they incorporate many implausible and unjustified assumptions, and they may not model appropriate physiological processes. 217 Volume 4, Nos. 3-4, 1994 Connectionism, Confusion and Cognitive Science PDP NETWORKS AND THE TRI-LEVEL HYPOTHESIS The explosion of interest in connectionist systems over the past decade has been accompanied by the development of diverse architectures (for overviews, see Cowan & Sharp, 1988; Hecht-Neilson, 1990; Müller & Reinhardt, 1990). These have ranged from simulations designed to mimic (with varying detail) specific neural circuits (e.g., Granger, Ambros- Ingerson & Lynch, 1989; Grossberg, 1991; Grossberg & Rudd, 1989, 1992; Lynch, Granger, Larson & Baudry, 1989) to new computer designs that have little to do with human cognition, but which use parallel processing to solve problems that are ill-posed or that require simultaneous satisfaction of multiple constraints (e.g., Hillis, 1985; Abu-Mostafa & Psaltis, 1987). Given this diversity, it is important at the outset to identify the branch of connectionism with which we are particularly concerned. This paper examines the characteristics of what has been called generic connectionism (e.g., Anderson & Rosenfeld, 1988, p. xv), because it appears to have had the most impact on cognitive science. A detailed description of the generic connectionist architecture is provided by Rumelhart, Hinton and McClelland (1986). PDP models are defined as networks of simple, interconnected processing units (see Figure la). Each processing unit in such a network operates in parallel, and is characterized by three components: a net input function which defines the total signal to the unit, an activation function which specifies the unit's current "numerical state", and an output function which defines the signal sent by the unit to others. Such signals are sent through connections between processing units, which serve as communication channels that transfer weighted numeric signals from one unit to another. Connection strengths can be modified by applying a learning rule, which serves to teach a network how to perform some desired task. For instance, the generalized delta rule (Rumelhart, Hinton & Williams, 1986a, 1986b) computes an error signal using the difference between the observed and desired responses of the network. This error signal is then "propagated backwards" through the network, and used to change connection weights, so that the network's performance will improve. 218 M.R. W. Dawson and K.S. Shamanski Journal of Intelligent Systems c Fig. 1: Components of the generic connectionist architecture. (A) A typical multilayer network of processing units. It is trained to generate a particular response to patterns that are presented to the input units. Hidden units can be viewed as feature detectors. (B) The three basic operations of a single processing unit. See the text for details. (C) Processing units communicate through a connection that has the numerical weight w-; learning rules are used to modify weight values. 219 Volume 4, Nos. 3-4, 1994 Connectionism, Confusion and Cognitive Science Classical artificial intelligence (AI) systems embody the notion that information processing is the serial manipulation of physical symbols that represent semantic content. Defined in this fashion, classical systems need to be described at three different levels of analysis — computational, algorithmic, and implementational — if they are to be completely understood (e.g., Marr, 1982, Chap. 1; Pylyshyn, 1984). Models created from the generic connectionist architecture are usually described (and sometimes criticized) as being radically different from classical models (e.g., Broadbent, 1985; Churchland & Sejnowski, 1989; Clark, 1989; Fodor & Pylyshyn, 1988; Hawthorne, 1989; Hecht-Nielsen, 1990; McClelland, Rumelhart & Hinton, 1986; Rumelhart, Smolensky, McClelland & Hinton, 1986; Schneider, 1987; Smolensky, 1988). One issue that arises from the view that connectionism offers a "paradigm shift" to AI researchers is whether the so-called tri-level hypothesis also applies to PDP models. For instance, some arguments indicate that PDP systems are implementational models: "In our view, people are smarter than today's computers because the brain employs a basic architecture that is more suited to deal with a central aspect of the natural information processing tasks that people are so good at" (McClelland, Rumelhart & Hinton, 1986, p. 1). However, when connectionism is described (and, in particular, criticized) as being merely implementational (e.g., Broadbent, 1985; Fodor & Pylyshyn, 1988), connectionists beg to differ. "Our primary concern is with the computations
Load more

Recommended publications
  • Arxiv:2107.04562V1 [Stat.ML] 9 Jul 2021 ¯ ¯ Θ∗ = Arg Min `(Θ) Where `(Θ) = `(Yi, Fθ(Xi)) + R(Θ)
    The Bayesian Learning Rule Mohammad Emtiyaz Khan H˚avard Rue RIKEN Center for AI Project CEMSE Division, KAUST Tokyo, Japan Thuwal, Saudi Arabia [email protected] [email protected] Abstract We show that many machine-learning algorithms are specific instances of a single algorithm called the Bayesian learning rule. The rule, derived from Bayesian principles, yields a wide-range of algorithms from fields such as optimization, deep learning, and graphical models. This includes classical algorithms such as ridge regression, Newton's method, and Kalman filter, as well as modern deep-learning algorithms such as stochastic-gradient descent, RMSprop, and Dropout. The key idea in deriving such algorithms is to approximate the posterior using candidate distributions estimated by using natural gradients. Different candidate distributions result in different algorithms and further approximations to natural gradients give rise to variants of those algorithms. Our work not only unifies, generalizes, and improves existing algorithms, but also helps us design new ones. 1 Introduction 1.1 Learning-algorithms Machine Learning (ML) methods have been extremely successful in solving many challenging problems in fields such as computer vision, natural-language processing and artificial intelligence (AI). The main idea is to formulate those problems as prediction problems, and learn a model on existing data to predict the future outcomes. For example, to design an AI agent that can recognize objects in an image, we D can collect a dataset with N images xi 2 R and object labels yi 2 f1; 2;:::;Kg, and learn a model P fθ(x) with parameters θ 2 R to predict the label for a new image.
    [Show full text]
  • Warren Mcculloch and the British Cyberneticians
    Warren McCulloch and the British cyberneticians Article (Accepted Version) Husbands, Phil and Holland, Owen (2012) Warren McCulloch and the British cyberneticians. Interdisciplinary Science Reviews, 37 (3). pp. 237-253. ISSN 0308-0188 This version is available from Sussex Research Online: http://sro.sussex.ac.uk/id/eprint/43089/ This document is made available in accordance with publisher policies and may differ from the published version or from the version of record. If you wish to cite this item you are advised to consult the publisher’s version. Please see the URL above for details on accessing the published version. Copyright and reuse: Sussex Research Online is a digital repository of the research output of the University. Copyright and all moral rights to the version of the paper presented here belong to the individual author(s) and/or other copyright owners. To the extent reasonable and practicable, the material made available in SRO has been checked for eligibility before being made available. Copies of full text items generally can be reproduced, displayed or performed and given to third parties in any format or medium for personal research or study, educational, or not-for-profit purposes without prior permission or charge, provided that the authors, title and full bibliographic details are credited, a hyperlink and/or URL is given for the original metadata page and the content is not changed in any way. http://sro.sussex.ac.uk Warren McCulloch and the British Cyberneticians1 Phil Husbands and Owen Holland Dept. Informatics, University of Sussex Abstract Warren McCulloch was a significant influence on a number of British cyberneticians, as some British pioneers in this area were on him.
    [Show full text]
  • Kybernetik in Urbana
    Kybernetik in Urbana Ein Gespräch zwischen Paul Weston, Jan Müggenburg und James Andrew Hutchinson1 Paul Ernest Weston was born on February 3rd 1935 in Garland, Maine. After stu- dying physics at Wesleyan University in Middletown (Connecticut), 1952 through 1956, Paul Weston came to the University of Illinois in summer 1956, receiving his M.S. in August 1958. The following autumn, as a graduate student of physics, he attended a seminar led by Heinz von Foerster under the promising title „Cyberne- tics“.2 In June 1959 Paul joins the newly founded Biological Computer Laboratory (BCL) as a half time research assistant employed by the Electrical Engineering Department. Through roughly 1962, Weston’s primary emphasis was upon simulation of the processes of perception, with work in neuron models and pattern recognition. Together with Murray Babcock, Weston was one of the most prominent creators of prototypes built at the BCL. Already in 1961 Weston becomes widely known as the inventor of the NumaRete, a parallel operated pattern recognition machine, which played an important role in the public perception of the BCL through out the fol- lowing years.3 After 1964 Weston continues his research at BCL on the basis of a full position as a research assistant and starts to focus on information processing, particularly inte- rested in problems associated with natural-language interaction with machine. As a member of the Committee for Cognitive Studies Weston is also increasingly interes- ted in the potentials of how the new information technologies could shape society. In June 1970 Weston receives his PhD for developing a new and more efficient data structure called CYLINDER.4 After the BCL was closed in 1975 Weston continued to work as a Research Associate at the Department of Electrical Engineering and the Coordinated Science Laboratory of the University of Illinois.
    [Show full text]
  • 1011 Neurons } 1014 Synapses the Cybernetics Group
    History 329/SI 311/RCSSCI 360 Computers and the Internet: A global history Week 6: Computing and Cybernetics in the Soviet Union Today } Review } Cybernetics: minds, brains, and machines } Soviet computing and cybernetics } Soviet economic planning } Next time Review: SAGE } Origin: Whirlwind digital computer project, MIT } SAGE = Semi-Automatic Ground Environment } Computer-controlled air defense of lower 48 states } Networked over telephone lines } Duplexed (two computers for high reliability) } Tube-based central processors made by IBM } Magnetic core memory } First truly large software development } Served as a pattern for many subsequent military projects } Major factor in US and IBM dominance of commercial computer markets by late1950s Cybernetics: minds, brains, and machines Key figures } Norbert Wiener } cybernetics } Warren McCulloch & Walter Pitts } neural nets } John von Neumann } brain modeling, cellular automata, biological metaphors } Frank Rosenblatt } perceptrons The Cybernetics Group } Norbert Wiener, MIT } WWII anti-aircraft problems } Servo/organism analogies } “Behavior, Purpose, and Teleology” (1943) } Information as measurable quantity } Feedback: circular self-corrective cycles BEHAVIOR, PURPOSE AND TELEOLOGY 21 Examples of predictions of higher order are shooting with a sling or with a bow and arrow. Predictive behavior requires the discrimination of at least two coordinates, a temporal and at least one spatial axis. Prediction will be more effective and flexible, however, if the behaving object can respond to changes in more than one spatial coordinate. The sensory receptors of an organism, or the corresponding elements of a machine, may therefore limit the predictive behavior. Thus, a bloodhound follows a trail, that is, it does not show any predictive behavior in trailing, because a chemical, olfactory input reports only spatial information: distance, as indicated by intensity.
    [Show full text]
  • Journal of Sociocybernetics
    JOURNAL OF SOCIOCYBERNETICS _________________________________________________________________ Volume 3 Number 1 Spring/Summer 2002 Official Journal of the Research Committee on Sociocybernetics (RC51) of the International Sociological Association JOURNAL OF SOCIOCYBERNETICS www.unizar.es/sociocybernetics/ Editor Richard E. Lee Newsletter Cor Van Dijkum Felix Geyer Editorial Board Mike Byron Tessaleno Devezas Jorge González Bernd R. Hornung Chaime Marcuello Vessela Misheva Philip Nikolopoulos Bernard Scott Mike Terpstra ____________________________________________________________________________ The JOURNAL OF SOCIOCYBERNETICS (ISSN 1607-8667) is an electronic journal published biannually--Spring/Summer and Fall/Winter--by the Research Committee on Sociocybernetics of the International Sociological Association. MANUSCRIPT submissions should be sent electronically (in MSWord or Rich Text File format) to each of the editors: Richard E. Lee [email protected], Felix Geyer, [email protected], and Cor van Dijkum, [email protected]. In general, please follow the Chicago Manuel of Style; citations and bibliography should follow the current journal style (APA). Normally, articles should be original texts of no more than 6000 words, although longer articles will be considered in exceptional circumstances. The Journal looks for submissions that are innovative and apply principles of General Systems Theory and Cybernetics to the social sciences, broadly conceived. COPYRIGHT remains the property of authors. Permission to reprint must be obtained from the authors and the contents of JoS cannot be copied for commercial purposes. JoS does, however, reserve the right to future reproduction of articles in hard copy, portable document format (.pdf), or HTML editions of JoS. iii SOCIOCYBERNETICS traces its intellectual roots to the rise of a panoply of new approaches to scientific inquiry beginning in the 1940's.
    [Show full text]
  • A Defense of Pure Connectionism
    City University of New York (CUNY) CUNY Academic Works All Dissertations, Theses, and Capstone Projects Dissertations, Theses, and Capstone Projects 2-2019 A Defense of Pure Connectionism Alex B. Kiefer The Graduate Center, City University of New York How does access to this work benefit ou?y Let us know! More information about this work at: https://academicworks.cuny.edu/gc_etds/3036 Discover additional works at: https://academicworks.cuny.edu This work is made publicly available by the City University of New York (CUNY). Contact: [email protected] A DEFENSE OF PURE CONNECTIONISM by ALEXANDER B. KIEFER A dissertation submitted to the Graduate Faculty in Philosophy in partial fulfillment of the requirements for the degree of Doctor of Philosophy, The City University of New York 2019 © 2019 ALEXANDER B. KIEFER All Rights Reserved ii A Defense of Pure Connectionism By Alexander B. Kiefer This manuscript has been read and accepted by the Graduate Faculty in Philosophy in satisfaction of the dissertation requirement for the degree of Doctor of Philosophy. _________________________ _________________________________________ 1/22/2019 Eric Mandelbaum Chair of Examining Committee _________________________ _________________________________________ 1/22/2019 Nickolas Pappas Executive Officer Supervisory Committee: David Rosenthal Jakob Hohwy Eric Mandelbaum THE CITY UNIVERSITY OF NEW YORK iii ABSTRACT A Defense of Pure Connectionism by Alexander B. Kiefer Advisor: David M. Rosenthal Connectionism is an approach to neural-networks-based cognitive modeling that encompasses the recent deep learning movement in artificial intelligence. It came of age in the 1980s, with its roots in cybernetics and earlier attempts to model the brain as a system of simple parallel processors.
    [Show full text]
  • Training Plastic Neural Networks with Backpropagation
    Differentiable plasticity: training plastic neural networks with backpropagation Thomas Miconi 1 Jeff Clune 1 Kenneth O. Stanley 1 Abstract examples. By contrast, biological agents exhibit a remark- How can we build agents that keep learning from able ability to learn quickly and efficiently from ongoing experience, quickly and efficiently, after their ini- experience: animals can learn to navigate and remember the tial training? Here we take inspiration from the location of (and quickest way to) food sources, discover and main mechanism of learning in biological brains: remember rewarding or aversive properties of novel objects synaptic plasticity, carefully tuned by evolution and situations, etc. – often from a single exposure. to produce efficient lifelong learning. We show Endowing artificial agents with lifelong learning abilities that plasticity, just like connection weights, can is essential to allowing them to master environments with be optimized by gradient descent in large (mil- changing or unpredictable features, or specific features that lions of parameters) recurrent networks with Heb- are unknowable at the time of training. For example, super- bian plastic connections. First, recurrent plastic vised learning in deep neural networks can allow a neural networks with more than two million parameters network to identify letters from a specific, fixed alphabet can be trained to memorize and reconstruct sets to which it was exposed during its training; however, au- of novel, high-dimensional (1,000+ pixels) nat- tonomous learning abilities would allow an agent to acquire ural images not seen during training. Crucially, knowledge of any alphabet, including alphabets that are traditional non-plastic recurrent networks fail to unknown to the human designer at the time of training.
    [Show full text]
  • Intelligence Without Representation: a Historical Perspective
    systems Commentary Intelligence without Representation: A Historical Perspective Anna Jordanous School of Computing, University of Kent, Chatham Maritime, Kent ME4 4AG, UK; [email protected] Received: 30 June 2020; Accepted: 10 September 2020; Published: 15 September 2020 Abstract: This paper reflects on a seminal work in the history of AI and representation: Rodney Brooks’ 1991 paper Intelligence without representation. Brooks advocated the removal of explicit representations and engineered environments from the domain of his robotic intelligence experimentation, in favour of an evolutionary-inspired approach using layers of reactive behaviour that operated independently of each other. Brooks criticised the current progress in AI research and believed that removing complex representation from AI would help address problematic areas in modelling the mind. His belief was that we should develop artificial intelligence by being guided by the evolutionary development of our own intelligence and that his approach mirrored how our own intelligence functions. Thus, the field of behaviour-based robotics emerged. This paper offers a historical analysis of Brooks’ behaviour-based robotics approach and its impact on artificial intelligence and cognitive theory at the time, as well as on modern-day approaches to AI. Keywords: behaviour-based robotics; representation; adaptive behaviour; evolutionary robotics 1. Introduction In 1991, Rodney Brooks published the paper Intelligence without representation, a seminal work on behaviour-based robotics [1]. This paper influenced many aspects of research into artificial intelligence and cognitive theory. In Intelligence without representation, Brooks described his behaviour-based robotics approach to artificial intelligence. He highlighted a number of points that he considers fundamental in modelling intelligence.
    [Show full text]
  • Simultaneous Unsupervised and Supervised Learning of Cognitive
    Simultaneous unsupervised and supervised learning of cognitive functions in biologically plausible spiking neural networks Trevor Bekolay ([email protected]) Carter Kolbeck ([email protected]) Chris Eliasmith ([email protected]) Center for Theoretical Neuroscience, University of Waterloo 200 University Ave., Waterloo, ON N2L 3G1 Canada Abstract overcomes these limitations. Our approach 1) remains func- tional during online learning, 2) requires only two layers con- We present a novel learning rule for learning transformations of sophisticated neural representations in a biologically plau- nected with simultaneous supervised and unsupervised learn- sible manner. We show that the rule, which uses only infor- ing, and 3) employs spiking neuron models to reproduce cen- mation available locally to a synapse in a spiking network, tral features of biological learning, such as spike-timing de- can learn to transmit and bind semantic pointers. Semantic pointers have previously been used to build Spaun, which is pendent plasticity (STDP). currently the world’s largest functional brain model (Eliasmith et al., 2012). Two operations commonly performed by Spaun Online learning with spiking neuron models faces signifi- are semantic pointer binding and transmission. It has not yet been shown how the binding and transmission operations can cant challenges due to the temporal dynamics of spiking neu- be learned. The learning rule combines a previously proposed rons. Spike rates cannot be used directly, and must be esti- supervised learning rule and a novel spiking form of the BCM mated with causal filters, producing a noisy estimate. When unsupervised learning rule. We show that spiking BCM in- creases sparsity of connection weights at the cost of increased the signal being estimated changes, there is some time lag signal transmission error.
    [Show full text]
  • THE INTELLECTUAL ORIGINS of the Mcculloch
    JHBS—WILEY RIGHT BATCH Top of ID Journal of the History of the Behavioral Sciences, Vol. 38(1), 3–25 Winter 2002 ᭧ 2002 John Wiley & Sons, Inc. (PHYSIO)LOGICAL CIRCUITS: THE INTELLECTUAL ORIGINS OF THE Base of 1st McCULLOCH–PITTS NEURAL NETWORKS line of ART TARA H. ABRAHAM This article examines the intellectual and institutional factors that contributed to the col- laboration of neuropsychiatrist Warren McCulloch and mathematician Walter Pitts on the logic of neural networks, which culminated in their 1943 publication, “A Logical Calculus of the Ideas Immanent in Nervous Activity.” Historians and scientists alike often refer to the McCulloch–Pitts paper as a landmark event in the history of cybernetics, and funda- mental to the development of cognitive science and artificial intelligence. This article seeks to bring some historical context to the McCulloch–Pitts collaboration itself, namely, their intellectual and scientific orientations and backgrounds, the key concepts that contributed to their paper, and the institutional context in which their collaboration was made. Al- though they were almost a generation apart and had dissimilar scientific backgrounds, McCulloch and Pitts had similar intellectual concerns, simultaneously motivated by issues in philosophy, neurology, and mathematics. This article demonstrates how these issues converged and found resonance in their model of neural networks. By examining the intellectual backgrounds of McCulloch and Pitts as individuals, it will be shown that besides being an important event in the history of cybernetics proper, the McCulloch– Pitts collaboration was an important result of early twentieth-century efforts to apply mathematics to neurological phenomena. ᭧ 2002 John Wiley & Sons, Inc.
    [Show full text]
  • What Is Systems Theory?
    What is Systems Theory? Systems theory is an interdisciplinary theory about the nature of complex systems in nature, society, and science, and is a framework by which one can investigate and/or describe any group of objects that work together to produce some result. This could be a single organism, any organization or society, or any electro-mechanical or informational artifact. As a technical and general academic area of study it predominantly refers to the science of systems that resulted from Bertalanffy's General System Theory (GST), among others, in initiating what became a project of systems research and practice. Systems theoretical approaches were later appropriated in other fields, such as in the structural functionalist sociology of Talcott Parsons and Niklas Luhmann . Contents - 1 Overview - 2 History - 3 Developments in system theories - 3.1 General systems research and systems inquiry - 3.2 Cybernetics - 3.3 Complex adaptive systems - 4 Applications of system theories - 4.1 Living systems theory - 4.2 Organizational theory - 4.3 Software and computing - 4.4 Sociology and Sociocybernetics - 4.5 System dynamics - 4.6 Systems engineering - 4.7 Systems psychology - 5 See also - 6 References - 7 Further reading - 8 External links - 9 Organisations // Overview 1 / 20 What is Systems Theory? Margaret Mead was an influential figure in systems theory. Contemporary ideas from systems theory have grown with diversified areas, exemplified by the work of Béla H. Bánáthy, ecological systems with Howard T. Odum, Eugene Odum and Fritj of Capra , organizational theory and management with individuals such as Peter Senge , interdisciplinary study with areas like Human Resource Development from the work of Richard A.
    [Show full text]
  • The Neocognitron As a System for Handavritten Character Recognition: Limitations and Improvements
    The Neocognitron as a System for HandAvritten Character Recognition: Limitations and Improvements David R. Lovell A thesis submitted for the degree of Doctor of Philosophy Department of Electrical and Computer Engineering University of Queensland March 14, 1994 THEUliW^^ This document was prepared using T^X and WT^^. Figures were prepared using tgif which is copyright © 1992 William Chia-Wei Cheng (william(Dcs .UCLA. edu). Graphs were produced with gnuplot which is copyright © 1991 Thomas Williams and Colin Kelley. T^ is a trademark of the American Mathematical Society. Statement of Originality The work presented in this thesis is, to the best of my knowledge and belief, original, except as acknowledged in the text, and the material has not been subnaitted, either in whole or in part, for a degree at this or any other university. David R. Lovell, March 14, 1994 Abstract This thesis is about the neocognitron, a neural network that was proposed by Fuku- shima in 1979. Inspired by Hubel and Wiesel's serial model of processing in the visual cortex, the neocognitron was initially intended as a self-organizing model of vision, however, we are concerned with the supervised version of the network, put forward by Fukushima in 1983. Through "training with a teacher", Fukushima hoped to obtain a character recognition system that was tolerant of shifts and deformations in input images. Until now though, it has not been clear whether Fukushima's ap- proach has resulted in a network that can rival the performance of other recognition systems. In the first three chapters of this thesis, the biological basis, operational principles and mathematical implementation of the supervised neocognitron are presented in detail.
    [Show full text]