1011 Neurons } 1014 Synapses the Cybernetics Group
Total Page：16
File Type：pdf, Size：1020Kb
History 329/SI 311/RCSSCI 360 Computers and the Internet: A global history Week 6: Computing and Cybernetics in the Soviet Union Today } Review } Cybernetics: minds, brains, and machines } Soviet computing and cybernetics } Soviet economic planning } Next time Review: SAGE } Origin: Whirlwind digital computer project, MIT } SAGE = Semi-Automatic Ground Environment } Computer-controlled air defense of lower 48 states } Networked over telephone lines } Duplexed (two computers for high reliability) } Tube-based central processors made by IBM } Magnetic core memory } First truly large software development } Served as a pattern for many subsequent military projects } Major factor in US and IBM dominance of commercial computer markets by late1950s Cybernetics: minds, brains, and machines Key figures } Norbert Wiener } cybernetics } Warren McCulloch & Walter Pitts } neural nets } John von Neumann } brain modeling, cellular automata, biological metaphors } Frank Rosenblatt } perceptrons The Cybernetics Group } Norbert Wiener, MIT } WWII anti-aircraft problems } Servo/organism analogies } “Behavior, Purpose, and Teleology” (1943) } Information as measurable quantity } Feedback: circular self-corrective cycles BEHAVIOR, PURPOSE AND TELEOLOGY 21 Examples of predictions of higher order are shooting with a sling or with a bow and arrow. Predictive behavior requires the discrimination of at least two coordinates, a temporal and at least one spatial axis. Prediction will be more effective and flexible, however, if the behaving object can respond to changes in more than one spatial coordinate. The sensory receptors of an organism, or the corresponding elements of a machine, may therefore limit the predictive behavior. Thus, a bloodhound follows a trail, that is, it does not show any predictive behavior in trailing, because a chemical, olfactory input reports only spatial information: distance, as indicated by intensity. The external changes capable of affecting auditory, or, even better, visual receptors, permit more accurate spatial locali- zation; hence the possibility of more effective predictive reactions when the input affects those receptors. In addition to the limitations imposed by the receptors upon the ability to perform extrapolative actions, limitations may also occur that are due to the internal organization of the behaving object. Thus, a machine which is to trail predictively a moving luminous object should not only be sensitive to light (e.g., by the possession of a photoelectric cell), but should also have the structure adequate for interpreting the luminous input. It is probable that limitations of internal organization, particularly of the organization of the central nervous system, determine the complexity of predictive behavior which a mammal may attain. Thus, it is likely that the nervous system of a rat or dog is such that it does not permit the integration of input and output necessary for the per- formance of a predictive reaction of the third or fourth order. Indeed, it is possible that one of the features of the discontinuity of behavior observable when Wienercomparing andhumans Bigelow,with other high mammals may lie in that the other mam- mals are limited to predictive behavior of a low order, whereas man may be “Behavior,capable potentially Purpose, of quite high orders and of prediction.Teleology” (1943) The classification of behavior suggested so far is tabulated here: First-, second-, etc. Predictive orders of (extrapo- prediction lative) Feed-back (teleo- Non-predic- logical) tive (non- Purposeful extrap- olative) Non-feed- back (non- I ~~|~ teleo- Active logical) Non-purpose- Behavior ful (random) Non active (passive) Servomechanisms The Cybernetics Group } Wiener, Cybernetics: Control and Communication in the Animal and the Machine (1948) } Information theory } Information as “negative entropy” Neurons } The human brain } 1011 neurons } 1014 synapses The Cybernetics Group } Warren McCulloch (neurologist) and Walter Pitts (logician) } Neural networks } “A Logical Calculus of the Ideas Immanent in Nervous Activity” (1943) } Neurons as on-off switches } Ideas taken from Turing, “On Computable Numbers” } Did not expect to be taken seriously Basic units of computation in the brain? Figure 1. Location of the essential nonlinearity. (Zador 2000, Nature) (a) Standard model of processing. Inputs 1−n from other neurons are multiplied by the corresponding passive synaptic weights w, summed (sum) and then passed through a nonlinearity (S). (b) An alternative model of processing in which the synapses themselves provide the essential nonlinearity. The Cybernetics Group } Claude Shannon, Bell Laboratories } “The Mathematical Theory of Communication” (1948) Shannon and Weaver, 1948 The Macy Conferences (1946-53) } “Cybernetics: Circular Causal and Feedback Mechanism in Biological and Social Systems” } Organizers: McCulloch, Wiener, von Neumann } Invitees: } Margaret Mead } Gregory Bateson } Claude Shannon } Walter Pitts } W. Ross Ashby (1948: Design for a Brain) } J. C. R. Licklider (later, sponsored ARPAnet) } Servo/organism, computer/brain analogies “Giant brains” } 1940s-50s: calculation, logic, science as paradigms of intelligence } The EDVAC report (1945) } Computer discussed as logic machine, not as electronic circuits } Logical elements = neurons } Storage = memory 1949 } von Neumann, The Computer and the Brain (1958) } Brain as switching system } 1010 neurons, each with as many as 1000 synaptic connections } Doubted whether comparison with computers would hold Brain modeling, 1940s-1950s } Dominant approach, 1940s-50s } Neurons as units which summed inputs } Assumption: brain not highly structured } Neurons as a self-organizing system } Perceptrons (F. Rosenblatt, 1958) } Turned out not to be true } Paradigm: learning A 3-layer perceptron Artificial intelligence: stages of development } General intelligence (1950s-60s) } Microworlds (1960s-70s) } Expert systems (1970s-80s) } Return to biological models (1980s-90s) } Neural nets/connectionism } Robot bugs } Genetic algorithms } Integration into other forms of computing (1990s-2000s) Soviet computing and cybernetics Gerovitch: “Soviet scientists were thus torn between two competing slogans: ‘Overtake and Surpass!’ and ‘Criticize and Destroy!’” “The Great Transformation of Nature” } Drought in 1946 led to famine } 1 million deaths } Stalin: “Nature needs to be transformed to serve mankind” 1940s and early 50s: Cybernetics as an American “pseudoscience” } 1940s: a “scientific heresy” } Linked to electronic computing, human-computer analogies } Soviet state campaigned against many Western scientific ideas at this time } Genetics, psychological theories, Einstein’s theories } Campaign against cybernetics was similar Sergei Lebedev } Studied and later taught electrical engineering at the Moscow Highest Technical School } 1946: head of the Kiev Electrotechnical Institute of the Ukraine Academy of Sciences The MESM “Small Electronic Computing Machine” } 1951: USSR’s first electronic computer } 6,000 vacuum tubes } 50 operations/second BESM “Large Electronic Computing Machine” (1953) Opening up of cybernetics in the 1950s } Lebedev’s success } moved to Moscow } cybernetics laboratory } Lab produced computing products for the Russian military and government in the 50s and 60s Lebedev Institute for Precision Mechanics and Computing Technology of the Soviet Academy of Sciences Computers for state planning } Stalin’s death -> government decentralization } Khrushchev: created “regional councils” } Management bureaucracy tripled by 1963 } 1957 Soviet Academy of Sciences report: } Computers as solution to promote efficiency “Cybernetics in the Service of Communism” } Academy Council on Cybernetics, 1961 } Entire Soviet economy as one large system of feedback control } Mathematical economics took hold by early 1960s } 500 institutions researching cybernetics by 1967, half of them studying economics Rise and decline of Cybernetics } 1960s: cybernetics became one of four divisions of Soviet science } “The Science of Sciences” } 1970s: decline of cybernetics } 1980s: cybernetics gave way to informatics Next time: The ARPANET } Reading } Lukasik, “Why the Arpanet was Built” (2011) } Computer, Chapter 11 } “Packet switching,” Wikipedia (no log on this) } Context } Ornstein, Computing in the Middle Ages, Ch. 14 (2002) } Licklider, “Memorandum For Members and Affiliates of the Intergalactic Computer Network,” 1963 } Computing (videos) } “Computer Networks: The Heralds of Resource Sharing” (1972) } Vint Cerf on the history of packet switching } Len Kleinrock: The First Two Packets on the Internet } Primary sources } ARPA Computer Network – Request For Proposals } Request For Comments (RFC) #3 (2 pp) .