DOCUMENT RESUME ED 294 889 TM 011 451 AUTHOR Hinton, Geoffrey E. TITLE Connectionist Learning Procedures. INSTITUTION Carnegie-Mellca Univ., Pittsburgh, Pa. Dept. of Computer Science. SPO AGENCY National Science Foundation, Washington, D.C.; Office of Naval Research, Arlington, Va. Personnel and Training Research Programs Office. REPORT NO TR-CMU-CS-87-115 PUB DATE Jun 87 CONTRACT N00014-86-K-00167 GRANT IST-8520359 NOTE 70p. PUB TYPE Information Analyses (070) EDRS PRICE MF01/PC03 Plus Postage. DESCRIPTORS Cognitive Processes; *Learning Processes; *Neurological Oraanization IDENTIFIERS *Connectionism ABSTRACT A major goal of research on networks of neuron-like processing units is to discover efficient learning procedures that allow these networks to construct complex internal representations of their environment. The learning procedures must be capable of modifying the connection strengths in such a way that internal units which are not part of the input or output come to represent important features of the task domain. Several interesting gradient-descent procedures have recently been discovered. Each connection computes the derivative, with respect to the connection strength, of a global measure of the error in the performance of the network. The strength is then adjusted in the direction that decreases the error. These relatively simple, gradient-descent learning procedures work well for small tasks and the new challenge is to find ways of improving the speed of learning so that they can be applied to larger, :Wore realistic tasks. (Author) *********************************************************************** * Reproductions supplied by EDRS are the best that can be made * * from the original document. * ******************************************************************-.**** UNCLASSIFIED SECURITY CLASSIFICATION OF THIS PAGE Form Approved REPORT DOCUMENTATION PAGE OMB No 0704-0188 la REPORT SECURITY CLASSIFICATION lb RESTRICTIVE MAPKINGS Unclassified 2a SECURITY CLASSIFICATION AUTHORITY 3 DISTRIBUTION /AVAILABILITY OF REPORT Approved for public release; distribution 2b DECLASSIFICATION/ DOWNGRADING SCHEDULE unlimited 4 PERFORMING ORGANIZATION REPORT NUMBER(S) 5 MONITORING ORGANIZATION REPORT NUMBER(S) CMU-CS-87-115 6a NAME OF PERFORMING ORGANIZATION 6b OFFICE SYMBOL 7a NAME OF MONITORING ORGANIZATION (If applicable) Carnegie-Mellon University Personnel and Training Research Programs Office of Naval Research 6c. ADDRESS (City, State, and ZIP Code) 7b ADDRESS (City, State, and ZIP Code) Department of Computer Science 800 North Quincy Street Pittsburgh, PA 15213 Arlington, VA 22217-5000 8a NAME OF FUNDING /SPONSORING 8b OFFICE SYMBOL 9 PROCUREMENT INSTRUMENT IDENTIFICATION NUMBER ORGANIZATION (If applicable) N00014-86-K-00167 8c ADDRESS (City, State, and ZIP Code) 10 SOURCE OF FUNDING NUMBERS PROGRAM PROJECT 'TASK WORK UNIT ELEMENT NO NO ACCESSION NO 61153N _RR04206 RR04206-08442b-467 11 TITLE (Include Security Classification) Connectionist learning procedures 12 PERSONAL AUTHOR(S) Hinton, Geoffrey E. 13a TYPE OF REPORT 13b TIME COVERED -14 DATE OF REPORT (Year, Month, Day) 15 PAGE COUNT FROM Technical TO Se.tember 4 1987 16 SUPPLEMENTARY NOTATION 17 COSATI CODES 18 SUBJECT TERMS (Continue on reverse if necessary and identify by blocknumber) FIELD GROUP SUB-GROUP Connectionism; learning procedures; neuralnetworks; gradient descent 19 ABSTRACT (Continue on reverse if necessary and identifyby block numbe) OVER 20 DISTRIBUTION/AVAILABILITY OF ABSTRACT 21 ABSTRACT SECURITY CLASSIFICATION 0 UNCLASSIFIED/UNLIMITED BE SAME AS RPT DTIC USERS Unclassified 22a NAME OF RESPONSIBLE INDIVIDUAL 22b TELEPHONE (Include Area Code)22c OFFICE SYMBOL _DrHarold Hawkins 202-696-4323 1142PT Form 1473, JUN 86 Previous editions are obsolete SECURITY CLASSIFICATION OF THIS PAGE 3 UNCLASSIFIED CONNECTIONIST LEARNING PROCEDURES' Geoffrey E. Hinton2 June 1987 Technical Report CMU-CS-87-115 Computer Science Department Carnegie-Mellon University Pittsburgh PA 15213 Abstract A major goal of research on networks of neuron-like processing units is to discover efficient learning procedures that allow these networks to construct complex internal representations of their environment. The learning procedures must be capable of modifying the connection strengths in sucha way that internal units which are not part of the input or output come to represent important features of the task domain. Several interesting gradient-descent procedures have recently been discovered. Each connection computes the derivative, with respect to the connection strength, ofa global measure of the error in the the performance of the network. The strength is then adjusted in the direction that decreaes the error. These relatively simple, gradient-descent learning procedures work well for small tasks and the new challenge is to find ways of improving the speed of learning so that they can be applied to larger, more realistic tasks. 'This research was supported by contract N00014-66-K-00167 from the Office c ''oval Research and by grant IST-8520359 from the National Science Foundation 2Auttiors current address is Computer Science Department, University of Toronto, 10 Kings College Road, Toronto, Ontano MSS 1A4, Canada. 1 Table of Contents 1. Introduction 2. Connectionist models 1 3. Connectionist research issues 1 3.1. Search 2 3.2. Representation 3 3.3. Learning 3 4 4. Associative memories without hiddenunits 4.1. Linear associators 5 5 4.2. Non-linear associative nets 6 4.3. The deficiencies of associatorswithout hidden units 5. Simple supervised learning procedures 8 9 5.1. An LMS procedure for binarythreshold units 5.2. The perceptron convergence procedure 10 5.3. The deficiencies of simple learning 11 procedures 12 6. Back-propagatIon: A multilayerLMS procedure 6.1. The shape of he error surface 13 6.2. Two exam!., of back-propagation 14 14 6.3. Back-propagation as a maximumlikelihood procedure 6.4. Iterative back-propagation 15 6.5. Self-superviseO back-propagation 16 17 6.6. A reinforcement version ofback-propagation 6.7. The deficiencies of back-propagation 18 7. Boltzmann machines 19 20 7.1. Maximizing reinforcement andentropy in a Boltzmann machine 8. Maximizing mutual information: 23 A semi-supervised learningprocedure 24 9. Unsupervised Hebblan learning 25 9.1. A recent development of unsupervisedHebblan learning 10. Competitive learning 26 10.1. The relationship between 26 competitive learning and back-propagation 28 11. Reinforcement learningprocedures 11.1. Delayed reinforcement 30 11.2. The Aq.p procedure 31 31 11.3. Achieving global optimalityby reinforcement learning 11.4. The relative payoff procedure 32 11.5. Genetic algorithms 32 34 11.6. Genetic learning and therelative payoff rule 11.7. iterated genetic hill-climbing 34 12. Discussion 35 12.1. Generalization 35 12.2. The speed of learning 35 12.3. Hardware modularity 37 12.4. Other types of modularity 38 12.5. Conclusion 39 40 1 1. Introduction Recent technological advances in VLSI and computer aided design mean that it isnow much easier to build massively parallel machines. This has contributed to a new wave of interestin models of computation that are inspired by neural nets rather than the formal manipulation of symbolicexpressions. To understand human abilities like perceptual interpretation, content-addressablememory, commonsense reasoning, and learning it may be necessary to understand how computation is organized insystems like the brain which consist of massive numbers of richly-interconnected but rather slow processingelements. This paper focusses on thequestion of how internal representationscal; be learned in "connectionist" networks. hese are a recent subclass of neuralnet models that emphasize computational power rather than biological fidelity. They grew out of work on early visual processingand associative memories (Hinton and Anderson,1981; Feldman and Ballard, 1982; Rumelhart, McClelland et. al., 1986). The paper starts by reviewing the main research issues for connectionistmodels and then describes some of the earlier work on learning procedures for associative memories andsimple pattern recognition devices. These learning procedures cannotgenerate internal representations: They are limited to forming simple associations between representations that are specifiedexternally.Recent research has led to a variety of more powenul connectior',t learning procedures thatcan discover good internal representations and most of thepaper is devoted to a survey of these procedures. 2. Conneotion!st models Connectionist models typically consist of many simple, neuron-like processingelements called "units" that interact using weighted connections.Each unit has a "state" or "activity level"that is determined by the input received from other units in the network. Thereare many possible variations within this general framework. One common, simplifying assumption is that thecombined effects of the rest of the network on the j'I' unit are mediated by a single scalar quantity,xj.This quantity, which is called the "total input" of unit j, is a linear function of the activity levels of theunits that provide input to j: xj= Iyi wji - 9j (1) where y, is the state of the ith unit, wji is the weight on the connection from the Ph to theft' unitand O. is the threshold of the jth unit. The threshold term can be eliminated by givingevery unit an extra input connection whose activity level is fixed at 1. The weight on this special conrectionis the negative of the threshold, and 3 can be learned in just the same way as the