Proceedings of Student-Faculty Research Day, CSIS, Pace University, May 6th, 2011

Frank Rosenblatt, Alan M. Turing, , and John M. Casarella

Seidenberg School of CSIS, Pace University, White Plains, NY 10606, USA [email protected]

Abstract: Dr. Frank Rosenblatt is commonly associated with Connectionism, an area of It was Frank Rosenblatt who in 1957 began cognitive science, which applies Artificial Neural looking at the McCulloch and Pitts [20] model of Networks in an effort to explain aspects of the neuron and started investigating neural human intelligence. Other notable networks. From his work came a new model he connectionists include Warren McCulloch, called the “”. Although Walter Pitts, and Donald Hebb, but it is Alan differed only slightly from previous neural Matheson Turing, a man of unique insight and networks, Rosenblatt made major contributions great misunderstanding, who is noticeably to the field through his experimental absent from this list. He is commonly associated investigations of the properties of perceptrons with the development of the digital computer, (using computer simulations), and through his employing his paper tape Universal Turing detailed mathematical analyses, basing the Machine. There are many who associate him perceptron model on probability theory rather with providing the foundation for defining than on symbolic logic. He was influenced by Artificial Intelligence, specifically the Hebb‟s concepts and was the first to associate development of the Turing Test as the standard the term “connectionist” with artificial neural to be met in determining if a machine exhibits networks. It was in 1958 when Rosenblatt intelligence. His contribution to AI goes beyond provided a definition to the theoretical basis of his test, laying down the foundation of connectionism in his statement, “stored Connectionism, providing insight into and information takes the form of new connections, supporting later contributions to the key models or transmission channels in the nervous system of Perceptrons, Artificial Neural Networks and (or the creation of conditions which are the Hierarchical Temporal Memory model. functionally equivalent to new connections)” [21, 22]. Introduction Dr. Rosenblatt was not primarily interested with the invention of devices for artificial intelligence, The dawn of Connectionist Theory is commonly but rather with investigating the physical traced back to McCulloch and Pitts and their structures and neurodynamic principles related to model of the Neuron. Further strength was “natural intelligence” [22]. He believed the added upon the publication of Donald O. Hebb‟s perceptron was first and foremost a brain model, influential book The Organization of Behavior not an invention for pattern recognition. (1949) [19], the source of the Hebbian approach Although, never brought to its maturity, the to neural learning studied in connectionism perceptron plays a vital role in artificial today. The contribution to connectionism by intelligence and in connectionist theory. By his Rummelhart, McClelland and the PDP Group own admission, Rosenblatt did not believe the [17, 18] cannot be minimized, yet all of them model was complete and summed up perceptrons show no awareness of Turing‟s early in this passage from his 1962 book (page 28): contribution to the field. "Perceptrons are not intended to serve as detailed copies of any actual nervous system. They're

D4.1

simplified networks, designed to permit the and the human. This is the basis of the Turing study of lawful relationships between the Test, a test integrally linked with answering the organization of a nerve net, the organization of question he poses and for determining machine its environment, and the 'psychological' intelligence. Additional details shall not be performances of which it is capable. Perceptrons presented, as it is best for one to read the original might actually correspond to parts of more article. extended networks and biological systems; in this case, the results obtained will be directly Although his famous paper was published in applicable. More likely they represent extreme 1950, Turing was harboring thoughts of machine simplifications of the central nervous system, in intelligence as early as 1941 according to Donald which some properties are exaggerated and Michie [2]. Michie remembers Turing would others suppressed. In this case, successive talk about the possibility of computing machines perturbation and refinements of the system may (1) learning from experience and (2) solving yield a closer approximation.[22]" Dr. problems by means of searching through the Rosenblatt‟s contribution to Artificial space of possible solutions [2]. He was also Intelligence, Connectionism and in providing the attempting to make a comparison between a foundation for Neural Nets and the HTM Model digital computer and the human brain. In a are significant, but he too was unaware of series of broadcast lectures, one given in May of Turing‟s early contributions. 1951, Turing provided some additional insight into what he was thinking. In reviewing In the Beginning… Turing‟s typescript, it was found he believed digital computers could be used in such a manner The mid to late 1950s is often looked upon as the they could appropriately be described as brains. beginning of artificial intelligence. This, He continues by saying, “although digital unfortunately, is incorrect. History, by way of computers might be programmed to behave like discovery and re-discovery of the writings of Dr. brains, we do not at present (1951) know how Alan M. Turing, places artificial intelligence‟s this should be done. As to whether we will or true origins to approximately 1950, and possibly will not eventually succeed in finding such a as early as 1941. To most people, it was program, I (Turing), personally am inclined to Turing‟s article entitled Computing Machinery believe that such a program will be found. Our and Intelligence [1] which affords him his fame, main problem is how to program a machine to but this is only part of Turing‟s contributions to imitate the brain, or as we might say more AI. The significance of this paper could not briefly, if less accurately, to think [3-5]”. have been anticipated at time of its publication, Turing‟s level of understanding of intelligence yet its impact on artificial intelligence cannot be and artificial intelligence was far more advanced disputed. Within this paper, Turing poses the than previously understood, specifically in how question, “Can Machines Think?” we learn. It was Turing‟s [4] understanding, in

trying to imitate an adult human mind, we should To determine the answer to this question, he consider three issues: the initial state of the provides the reader with a “game” which first mind, the education it has been subject to, and takes place between an interrogator, a man and a the other experiences it has been subject to (that woman. These three individuals are separated cannot be described as education). His final from each other. The interrogator can ask thoughts show we should try to create a questions of either of these individuals via computational model a child‟s mind and then “teletype” interface. The man will attempt to “educate” it to obtain the model of the adult convince the interrogator he is the woman and brain. It would difficult not to see the correlation the woman will be truthful. The objective is for to perceptrons, neural networks and especially to the interrogator to correctly conclude who is the the hierarchical temporal memory model. man and who is the woman. Turing now alters this game and replaces the man or the woman In the ensuing years, there remained many with a machine. It is now the objective of the unanswered questions concerning his vision of interrogator to differentiate between the machine artificial intelligence, his views on intelligent

D4.2

machinery and the continuous debate as to the into networks in a largely random manner, meaning of the “Turing Test” in defining referred to by Turing as “unorganized intelligence. There are arguments attempting to machines”. His invented neural network was show the fallacy of Turing‟s concept of machine called a “B-type unorganized machine”, which intelligence, such that a machine would need to consisted of artificial neurons and devices be conciseness (be aware of itself), but is this a capable of modify the connections between valid argument? Maybe Turing was looking them. His model was very different in that every beyond a simple definition of machine neuron in the network executes the same logical intelligence, but was unable to complete his operation of “not and” (NAND): the output is 1 work due to his untimely departure from this if either of the inputs is 0. If the both inputs are Earth. 1, then the output is 0. Turing selected NAND because every other logical or Boolean operation Some of his critics have found fault in the can be accomplished by groups of NAND behavioral approach of the Turing Test [6]; neurons [10, 11]. French [7] discusses whether passing the Turing Test is a sufficient or a necessary condition for It was Turing‟s contention machines [4, 10, 12] machine intelligence and he asks whether the test could be constructed which would simulate the can be passed at all. Perhaps it is Hayes and behavior of the human mind very closely. He Ford [8] who provided a more provoking goes further by stating these machines “will concern in a moral objection concerned with the make mistakes at times and at times they may artificial constraints the setting imposes on the make new and very interesting statements, and participants of the game and to express their on the whole the output of them will be worth inability to find a practical use for the Turing attention to the same sort of extent as the output Test. They ask why we put forth so much effort of a human mind.[4]” He looked at the creation to build a machine to imitate a human. or development of such a machine as being akin to a child, which would be able to learn by An analogy is sometimes made between artificial experience, such that by starting with intelligence and artificial flight. As long as comparatively simple machine, and by scientists and engineers tried to copy the flight subjecting it to a suitable range of „experiences‟ apparatus of birds, artificial flight remained transform in into one which was much more illusive. When they abandoned the attempt to elaborate, and was able to deal with a far greater mimic nature, but instead studied the basic range of contingencies. principles of flight in non-natural systems, successful aircraft were developed. Thus, AI The idea that an initially unorganized neural researchers should abandon the goal of imitating network can be organized by means of human intelligence and rather seek general “interference training” is undoubtedly the most principles of intelligence in non-human systems significant aspect presented in this paper. With in order to perfect artificial intelligence [9], an Turing‟s model, the training process renders avenue the memory-prediction model appears to certain neural pathways effective and others pursue. ineffective. He anticipated the modern procedure of simulating neural networks and the Individuals who consider themselves training process by means of an ordinary digital connectionists usually consider Hebb and computer [12]. This is considered the first Rosenblatt as the originators of their approach, manifesto of AI and in it Turing brilliantly but in fact both were preceded by Turing, who introduced many of the concepts associated with anticipated much of modern connectionism in his neural networks, although, in some cases after 1948 paper “Intelligent Machinery” (see the reinvention by others. One of these was the Proudfoot and Copeland, [4, 10, 12]). concept of “teaching” a network of artificial neurons to perform specific tasks. From his writings and other documents, it is conceivable Turing was the first person who Turing also claimed a proof (now lost) of the considered building computing machines out of proposition that an initially unorganized Turing simple, neurons-like elements connected together Net with sufficient neurons can be organized to

D4.3

become a universal Turing machine with a given chance of making the right identification after 5 storage capacity [12]. This proof first opened up minutes of questioning. As history has shown, the possibility, noted by Turing, that the human we continue to fail. cognitive system is a universal symbol-processor implemented in a neural network. However, The End of it All Turing's own research on neural networks was carried out shortly before the first stored- Consider how we perceive intelligence. A program electronic computers became available. problem is presented to a mathematician or It was not until 1954 (the year of Turing's death) scientist to solve. The problem was presented that Belmont Farley and Wesley Clark [13], specifically to these individuals due to their field working at MIT, succeeded in running the first of expertise. The solution may require the computer simulations of small neural networks. application of a new method or formula and it is Farley and Clark were able to train networks possible they may return an incorrect solution. It containing at most 128 neurons to recognize is also conceivable many attempts may be simple patterns. necessary before a solution is found, if a solution can be found. What is our perception of them, Many involved in the early years of AI research, are they any less intelligent in light of making an acknowledge the Turing Test was inspirational, error when attempting to obtain the solution to a but as knowledge was gained in what a machine problem? The answer is no, so why do we not is capable and not capable of doing; together apply the same standard or perception towards a with advances in neuroscience and in the machine, a computer? When given a problem, if understanding of brain function, some argue the the computer should return the incorrect Turing test should be consigned to history. As solution, should we state that it does not have put so eloquently by Hayes and Ford, “The intelligence? Intelligent humans, even highly Turing test has a historical role in getting AI regarded, intelligent humans are not infallible; started, but it is now a burden to the field, they make errors, yet we do not consider them damaging its public reputation and its won any less intelligent when they do, so why not intellectual coherence. We must explicitly reject apply the same standard or perception to the Turing Test in order to find a more mature computing machining when we attempt to description of our goals; it is time to move it determine machine intelligence. from the textbooks to the history books [8].” Some authors offer the Turing test as a definition It is unfortunate such a test has been so widely of intelligence: a computer is intelligent if and accepted and at the same time, so limiting in only if the test fails to distinguish it from a determining if a computer is capable of human being. However, Turing himself in fact intelligence. Although Turning always used the pointed out that his test couldn‟t provide a term “machine”, we have advanced such the definition of intelligence. It is possible, he said, name “computer” is more appropriate. Turing that a computer which ought to be described as never claimed in the first place that the ability to intelligent, might nevertheless fail the test pass the Turing test is a necessary condition for because it is not capable of successfully imitating intelligence. Turing indicates the point of the a human being. test is to determine whether or not a computer can “imitate a brain.” Turing made it clear that a Jeff Hawkins [23, 25, 28] basic premise, as machine might be intelligent and yet not pass his embodied by Hierarchical Temporal Memory imitation game. If „machine intelligence‟ is no Model, seems right: perceptual-memory-based longer an oxymoron, then one of Turing‟s predictions surely do play a fundamental role in important predictions has come true. intelligence and hence in brain structure and function. What does this mean? That an Turing predicated in about 50 years‟ time from intelligent agent learns from experience and in the publication of his famous article, it will be particular builds up a model of the world by possible to program computers to make them perception (not only of what is our there but of play the imitation game so well an average what actions achieve what results): and that this interrogator will not have more than 70 per cent

D4.4

experience is remembered and available virtually [8] P. Hayes, and K. Ford, "Turing Test Considered instantly for use in deciding what to expect next. Harmful," in Proceedings of the Fourteenth International Joint Conference on Artificial In the opinion of Professor David Gelernter [14], Intelligence, 1995, pp. 972-977. software today can only cope with a smattering [9] K. M. Ford, and P.J. Hayes, "On Computational of the information processing problems our Wings: Rethinking the Goals of Artificial minds handle routinely, when we recognize faces Intelligence," Scientific American Presents, vol. 9, pp. or pick elements out of a large group based on 78-83, 1998. visual cues, use common sense, understand the nuances of natural language or recognize what [10] J. Copeland and Diane Proudfoot "On Alan makes a musical instrument cadence final. Turing's Anticipation of Connectionism," Synthese, Turing seemed to believe (sometimes) that vol. 108, pp. 361 - 377, 1996. consciousness was not central to thought, simulated or otherwise. A computer can add [11] J. Copeland and Diane Proudfoot, "Alan Turing's Forgotten Ideas in Computer Science," Scientific numbers but has no idea what „add‟ means, what American, pp. 99 - 103, 1999. a „number‟ is or what „arithmetic‟ is for. Its actions are based on shapes, not meaning. Dr. [12] A. M. Turing, "Intelligent Machinery," in Gelernter agrees with Searle‟s Chinese room. Machine Intelligence 5, B. Meltzer, and D. Michie, But if we take the route Turing hinted at back in Ed. Edinburgh: Edinburgh University Press, 1948, pp. 1950, if we forget about consciousness and 3-23. concentrate on the „process of thought‟ there is every reason to believe that we can get AI back [13]B. G. Farley, and W.A. Clark, "Simulation of Self-Organizing Systems by Digital Computer," on track, and that AI can produce powerful Institute of Radio Enigneers Transactions on software and show us important things about the Information Theory, vol. 4, pp. 76 - 84, 1954. human mind [14]. [14]D. Gelernter, "Artificial Intelligence is Lost in the Woods," in Technology Review, 2007. References [15]A. M. Turing, "On computable numbers, with an [1] A. M. Turing, "Computing Machinery and application to the Entscheidungsproblem," Intelligence," Mind, vol. 59, pp. 433 - 460, October Proceedings of the London Mathematical Society, 1950. Series 2, vol. 42, pp. 230-265, 1936.

[2] B. J. Copeland, and D. Proudfoot, "What Turing [16] M. L. Minsky, and Papert, Seymour S., Did after He Invented the Universal Turing Machine," Perceptrons: An Introduction to Computational Journal of Logic, Language, and Information, vol. 9, Geometry. Cambridge, MA: MIT Press, 1969. pp. 491-509, 2000. [17] D. E. Rumelhart, and McClelland, J.L. editors, [3] B. J. Copeland, and D. Proudfoot, "The Legacy of "Parallel Distributed Processing: Explorations in the Alan Turing," Mind, vol. 108, pp. 187-195, 1999. Microstructures of Cognition." vol. 1 - Foundations Cambridge, MA: MIT Press, 1986. [4] B. J. Copeland, "The Essential Turing," Oxford, Great Britain: Oxford University Press, 2004. [18] J. L. McClelland, and Rumelhart, D.E., "Parallel Distributed Processing: Explorations in the [5] A. M. Turing, "The Turing Digital Archive," Microstructures of Cognition." vol. 2 - Psychological http://www.turingarchive.org/: University of and Biological Models Cambridge, MA: MIT Press, Southamption and King's College Cambridge, 2002. 1986.

[6] N. Block, "Psychologism and Behaviourism," [19] D. O. Hebb, The Organization of Behavior. New Philosophical Review, vol. 90, pp. 5-43, 1981. York: John Wiley & Sons, 1949.

[7] R. French, "Subcognition and the Limits of the [20] W. S. McCulloch, and Pitts, Walter H., "A Turing Test," Mind, vol. 99, 1990. Logical Calculus of the Ideas Immanent in Neural Nets," Bulletin of Mathematical Biology, vol. 52, pp. 99 - 115, 1943.

D4.5

[21] F. Rosenblatt, "The Perceptron: A Probabilistic Model for Information Storage and Organization in the Brain," Psychological Review, vol. 65, pp. 386 - 408, 1958.

[22] F. Rosenblatt, Principles of Neurodynamics. Washington, DC: Spartan Books, 1962.

[23] J. Hawkins, with Sandra Blakeslee, On Intelligence, First ed. New York: Times Books, Henry Holt and Company, 2004.

[24] D. George, and Hawkins, J., "Belief Propagation and Wiring Length Optimization as Organizing Principles for Cortical Microcircuits," Numenta, Inc., 2005.

[25] D. George, and Hawkins, J., "Invariant Pattern Recognition using Bayesian Inference on Hierarchical Sequences," Numenta, Inc., 2006.

[26] J. Hawkins, and Dileep George, "Hierarchical Temporal Memory, Concepts, Theory, and Terminology," Numenta, Inc., 2006.

[27] J. Hawkins, "Hierarchical Temporal Memory (HTM): Biological Mapping to Neocortex and Thalamus," Numenta, Inc., 2007.

[28] J. Hawkins, "An Investigation of Adaptive Behavior Towards a Theory of Neocortical Function," 1986.

[29] D. George, "How the Brain Might Work: A Hierarchical and Temporal Model for Learning and Recognition," Doctoral Dissertation; Stanford University, 2008.

D4.6