ISSN (Print) : 2319-5940 ISSN (Online) : 2278-1021

International Journal of Advanced Research in Computer and Communication Engineering Vol. 2, Issue 8, August 2013

A Review of in Neuromorphic VLSI Chips using Computational Cognitive

Mohammed Riyaz Ahmed1, B.K.Sujatha2 Assistant Professor, Department of ECE, REVA I.T.M., Bangalore, India 1 Professor, Department of TCE, MSRIT, Bangalore, India 2

Abstract: In this review paper, Cognitive models are used to implement the Reinforcement Learning in Neuromorphic VLSI Chips, to exhibit intelligence when the machines are exposed to an undefined Situation or field so as to achieve maximum rewards for taking right decision or very feasible solution or optimal path. This is done by modelling the attention and of machine just as a human being. Because the thought process is the unique nature of humans‟ intelligence can be implemented by modelling the Cognition. Comparison with other methods of attaining shows that Computational is the best and most evolved system for exhibiting intelligence to learn in real time scenarios. It is concluded that all the sensory input is not necessarily being calculated, instead attention is given to effective perception and trivial perception is ignored. This differentiation of perception is done based on BDI Model. Intelligence is broadly defined and requirements of an intelligent agent is summarized. The emerging field of electronics for implementation of reinforcement learning by imitating i.e. Neuromorphic Engineering is discussed. The significance of the accurate knowledge of intelligence in machines based on learning and decision making is discussed.

Keywords: cognitive sciences, computational intelligence, Reinforcement Learning, Intelligent systems, Neuromorphic Engineering.

I. INTRODUCTION The purpose of this paper is to review the progress being Networks, Fuzzy Logic, Cognitive Architectures, done in the field of machines that learn new tasks and Computational Sciences, Bio-inspired computing, so on and behavior from common man in everyday life. We assume so forth. But still human behavior cannot be imitated. Section that machines which can learn from everyday interaction can II makes a comparative study and identifies the missing better take advantage of the teaching and training as in [1] & ingredient for the dish of Intelligence. [2]. We understand the cognition behind learning and in this Human working memory is limited [3], machine memory is paper we have computationally modeled specific still more limited when compared with humans. Human mechanisms of human social behavior in general and learning memory addresses this by storing the information into long in particular. This paper shows that how reinforcement term memory, short term memory and working memory. learning is improved by providing proper attention to the Every kind of memory has some time duration and after human interactions. This work spans the fields of Cognitive which the information decays there by efficiently utilizing the Sciences, Psychology, and in particular memory space. But in machines no such differentiation and Reinforcement Learning, Human Computer interactions and decaying is observed. This serious limitation poses us a VLSI design. challenge of intelligently accumulation of information and A social learner [2] has to learn in real time situations on its efficiently building the Knowledge. This is where machine own, but taking aid of human partner whenever necessary performance and learning ability comes into picture. The best thereby exploring the environment and learning from it. This way to overcome this limitation is schema acquisition, which exploration can be influenced by attention direction, action means chunks of information are extracted and can be suggestions, providing positive and negative feedback. This processed as meaningful units. For our simulation the Guided Reinforcement learning has to frame its own learning language which helps us to have abstracts or chunks of problems, define goals and plan to achieve it. information is Prolog [4].When we are able to model the We have many algorithms from various domains that explain memory we have to model the attention and perception viz. to achieve some intelligence in machines, such as Neural Accumulation of only necessary information else the machine will be flooded with chunks of irrelevant Copyright to IJARCCE www.ijarcce.com 3315 ISSN (Print) : 2319-5940 ISSN (Online) : 2278-1021

International Journal of Advanced Research in Computer and Communication Engineering Vol. 2, Issue 8, August 2013 information. This modelling is done based on the BDI (belief, In contrast, “human brain works by activating thought Desire and Intention) Model. Section III describes modelling modules” (Cognitive Modules), [27]. The brain distributes of memory and attention. memory and processing functions throughout the system, Now when we are able to model the attention and undergo learning by situations and uses a complex differentiate memory, does that mean a machine is combination of reasoning, synthesis and creativity to solve intelligent? What it takes for a machine to be intelligent? And the problems it has never encountered before. what an intelligent agent should exhibit? What is the relation The brain solve the problems, using complex combination of between date, information, knowledge, intelligence and reasoning, synthesis and creativity, according to „Modha‟ wisdom? Section IV has a detailed analysis of all questions brain is hardware of which different processes such as pertaining to intelligence. sensation, perception, action, cognition emotion and A new field of Electronics is introduced i.e. Neuromorphic Engineering where the computational cognitive neuroscience interaction arises. For us cognition is of most importance. based reinforcement learning can be implemented on As motors augment human or horse power AI can use the hardware VLSI Chips. Section V identifies Neuromorphic computers to augment human thinking, [6]. Robotics and Engineering as a tool for Implementation. Neuromorphic expert systems are major branches of that. On the other hand engineering builds artificial systems utilizing both electronics computers artificial intelligence can be used to understand and computation biological nervous systems knowledge [5]. the process of thought, nothing but Cognition. Before going into implementation we need a tool for If we approach the programs not merely by what they can modelling and simulation of these intelligent ICs. Emergent accomplish but how they accomplish it, “then you are really [6] is identified as a simulation tool for guided reinforcement doing Cognitive Science; you are using AI to understand the learning [7 o riely]. To articulate significant conclusions to a human mind” [6]. wider audience, paper concludes with open research issues Cognitive Science and human mind are interconnected. and future scope. Cognitive researches study nature of intelligence from a psychological point of view, mostly building compute models which help in analysing what happens in our brains I. COMPUTATIONAL COGNITIVE NEUROSCIENCE during problem solving, remembering, perceiving and other A. Computational Sciences psychological process . Artificial Intelligence began with the imagination of B. Cognitive Sciences philosophers, where the characters of their fictions had The term Cognitive Science gained currency in last half of supernatural powers and nearly imitate the humans- the 20th century, is used to refer to the study of Cognition--- physically and emotionally. Cognitive structures and processes in mind or brain, [1]. The field of Artificial Intelligence research was founded at a Cognitive Science is an interdisciplinary field that has conference on campus of Dartmouth College in 1956. Since emerged by the intersection of existing branches of science then AI has grown exponentially. Things that seem such as Psychology, Linguistics, , impossible 2 decades back have been realized and are used Philosophy and Physiology. “The shared interest that has in successful commercial products. Now the humankind is produced this coalition understands the nature of the mind”, headed towards building a human level of intelligence, [26]. (which may be possible any time now). Origin of Cognitive Science began in mid-1950 when the The field of AI originated with game-playing and theorem- researches began to treat mind not as a machine but as a proving programs and gradually evolved with lot of research biological system which has several non-linear responses in various interdisciplinary streams. “Today, AI spans a and can be analyzed based on complex representations and wide horizon”, [25]. It deals with various kinds of computational procedures. knowledge representation schemes, different techniques of “The notion that mental states and processes intervene intelligent search, various methods for resolving uncertainty between stimuli and responses sometimes take the form of a of data and knowledge, different schemes for automated computation”, [2]. As software is to hardware, the mind is to machine learning and many others. the brain; mental states and processes are like computer Early computers had fixed programs, later came the stored programs implemented in brain. program concept. The stored-program design allows the The moral in so far discussion is the human mind can be programs modify themselves while running. Stored program investigated based on computations. Hence Computer computer also called as . But the Science in general and Artificial Intelligence in particular architecture has two fundamental drawbacks: 1. the have come to play a central role in Cognitive Science. connection between the memory and can get overloaded, there by sacrificing the speed of computer and 2. Specific programs should be written to perform specific tasks [5]. Copyright to IJARCCE www.ijarcce.com 3316 ISSN (Print) : 2319-5940 ISSN (Online) : 2278-1021

International Journal of Advanced Research in Computer and Communication Engineering Vol. 2, Issue 8, August 2013

C. Computational Cognitive Neuroscience hence background data or trivial data is limited only for the goes well beyond Artificial working memory. At any instance Effective perception can Intelligence and human-computer interaction by exploring become Trivial and vice-versa. the concepts of perception, memory, attention, language B. Psychological Background intelligence and consciousness, [7]. In psychology perception is classified as conscious and “Cognitive Computing is when computer science meets unconscious perception. The awareness of stimulus with neuroscience to explain and implement Psychology, [8]”. working memory and attention is known as conscious Both AI and Neural networks (NN) fail to replicate the perception [10]. The unconscious perception refers to stimuli phenomenon of thinking, though they have considered brain that are below the threshold for becoming conscious [11]. as the starting point to explain Cognitive phenomena. In There is no clear distinction between conscious and contrast Cognitive Computing is about learning how brain unconscious perception and many a times the lines are operates, designing algorithms, reverse engineering and crossed. testing plausible models. “Cognitive Computing is about According to philosophical view: passive observation is engineering the mind by reverse engineering the brain, [8]”. because of less interest and we don‟t pay much attention, so The principles and methodologies of Cognitive sciences are the data would be in working memory and decay after some being implemented to , so as to create humanoid time. If the interest is very high then attention is more and we robots. These robots are not just software agents but are perceive the data with utmost care and remember those things mobile robots and autonomous vehicles. for a long time i.e. stored in long term memory. As interest In , Cognitive Neuroscience is used to increases more the chances of being important and less the solve neurocognitive disorders and help in computational importance less the storage duration. modeling. There are many projects using Cognitive At this juncture we classify Perception to be of two types. computing for investigation of different languages like Active perception where sensors are waiting for the data i.e. Spanish, Russian, Japanese, Portuguese, Dutch etc, [28]. Effective Perception and Passive perception where the data is not so relevant or totally irrelevant. Table below summarizes Computational Cognitive Neuroscience is also used in the types of perception base on attention and duration for investigation of hearing and speech. Brain imaging which the perceived data is stored in memory. technique also uses Cognitive computing to see how a man speaks in pathological conditions as compared to healthy TABLE I. PERCEPTION TYPE conditions. Parameters Deciding Perception Type of II. ATTENTION AND MEMORY MODELLING Level of Percepti Duration of storage Attention/ level A. Attention on in memory of Threshold Selective Attention provides an effective mechanism to Effective High Long term memory conceptualize the perceived data both in biological systems Short term/ working and machines. The perceived data consists of important, not Trivial Low so important and irrelevant information. If we process all the memory data what we receive, then the overhead will be more and the machines will be busy in accumulating and processing, C. Modelling using BDI Model in this rush hour machine cannot dedicate time for learning The classification can be enhanced by using deliberative and imitating human behaviour. model. BDI (Belief, Desire and Intention) Model. These are This can be overcome by classifying Perceived data as the basic mental components present in rational agent Effective and Trivial. Effective perception becomes part of architecture [15]. Belief Desire and Intention reflects cognition process and on processing goes to long term motivation and learning. The BDI Model INTENSIONS are memory, whereas trivial perception data will be in the adopted plans and strategies for achieving DESIRES. working memory. It is just like viewing a Photograph, we BELIEF is the judgment along with desire and intention to observe and give great attention to the photo but not to the reason. background. We remember the photos and people in them We model the perception as effective and trivial based on but doesn‟t remember the background colour combination. the threshold set by Belief, Desire and Intention. Perceived Therefore data obtained by effective perception is going for information is the input, based on the classification the long term memory. It doesn‟t mean that background trivial data will be present only for working memory and the information is not at all necessary. If the background is effective perception data will be sent for long term memory. disturbing or if there is no usual colour then we don‟t focus on foreground i.e. the person in the image. Hence the background is very much necessary only for that instance

Copyright to IJARCCE www.ijarcce.com 3317 ISSN (Print) : 2319-5940 ISSN (Online) : 2278-1021

International Journal of Advanced Research in Computer and Communication Engineering Vol. 2, Issue 8, August 2013

Working Memory Figure 2. Combination of Circles and Triangles Trivial Perception Short Term Since there are twice number of circles, probability of new Perceived Data Memory arrival being circle is more.

Effective Long Term Perception Memory 푃푟표푏푎푏푖푙푖푡푦 표푓 푎푟푟푖푣푎푙 표푓 퐶푖푟푐푙푒 = (푁푢푚푏푒푟 표푓 푐푖푟푐푙푒) /(푇표푡푎푙 푛푢푚푏푒푟 표푓 표푏푗푒푐푡푠) (1) Perception Attention Memory 푃푟표푏푎푏푖푙푖푡푦 표푓 푎푟푟푖푣푎푙 표푓 푇푟푖푎푛푔푙푒 = (푁푢푚푏푒푟 표푓 푇푟푖푎푛푔푙푒)

Figure 1. Attention and Memory classification /(푇표푡푎푙 푛푢푚푏푒푟 표푓 표푏푗푒푐푡푠) (2) D. Mathematical Modelling The Naïve Bayes classifier is simple probabilistic classifier The Threshold is set between the probability of circle and technique based on Bayesian Theorem. It is best suited when that of Triangle. Circle probability is 16/24, whereas triangle dimensionality of inputs is high. Naïve Bayes is better than probability is 8/24. If we set the Threshold as 12/24, then sophisticated classification methods. whenever the threshold is less than 12/24, that means the Consider a mixture of triangles and circles. Let triangle be new arrival is Triangle and perception has to be effective, our interested shape i.e. Triangle will be our effective now we will increase our attention. Perception and circle would be our Trivial Perception. Our Setting the Threshold using a BDI table is as shown below in task is to identify new cases as they arrive. We have to Table II. decide the new arriving shape is of interest or not based on If the machine is unable to make out whether the perceived current available data. information then Threshold is given much Importance. For just above threshold condition the information is stored in Short term Memory.

III. INTELLIGENCE To tell a machine is Intelligent, we don‟t have any touchstone experiment. From the beginning of computing every new feature which demonstrated Autonomy was said

TABLE II. BDI MODELLING OF ATTENTION AND MEMORY

Parameters Deciding Perception Type of

Perception Level of Attention/ Duration of storage Belief Desire Intention level of Threshold in memory

Effective High Long term memory Yes Yes Yes Short term/ working Trivial Low No No No memory Effective High X X High Long term memory Effective X High X High Long term memory Effective X X High High Long term memory

Less Effective Medium Medium X Medium Short term

Medium X Medium Medium Short term Copyright to IJARCCE www.ijarcce.com 3318 ISSN (Print) : 2319-5940 ISSN (Online) : 2278-1021

International Journal of Advanced Research in Computer and Communication Engineering Vol. 2, Issue 8, August 2013

Parameters Deciding Perception Type of

Perception Level of Attention/ Duration of storage Belief Desire Intention level of Threshold in memory Less Effective

Less Effective X Medium Medium Medium Short term

Trivial Low Low low Low working memory to be intelligent. Later when machines were able to have Gathering the Data from environment computational excellence, controlling its environment, have communication between different or other computing devices Start (input perceived data) were called as intelligent machines. When there features became compulsory and every machine started to have these Check for Belief options, those systems are known as smart devices. We can Compare with prior threshold and likelihood for conclude that if any device „A‟ which exhibits autonomy, belief. senses its environment, reacts to the changes, and take best If Belief >> prior Threshold decisions is called an Intelligent machine, whenever a new Attention  High, device „B‟ with some more new intelligent feature is Else available then it is called as Intelligent machine and the Compare with prior threshold and likelihood for device A is a smart device now. Hence evolution of devices desire. bring new intelligent machines whereas previous machines If Desire >> prior Threshold are called as smarter. We can also define intelligent machine Attention  High, as one which exhibits Cognition, just like humans viz. Else extracting chunks of data, schema acquisition, building Compare prior threshold and likelihood for knowledge base and taking appropriate and best possible Intention. decisions in real time scenarios. If Intention >> prior Threshold Below is an algorithm which shows the modelling of Attention  High, Attention and Memory for Prolog. Else Algorithm for the abstract language PROLOG for the Compare with prior threshold and likelihood for modelling of Attention and Perception. Prolog is a language which helps us to make sense from chunks of information, belief and Desire and Intention. and exhibit self-learning. If BDI > prior Threshold Attention  Medium, Else Attention  Low IV. NEUROMORPHIC ENGINEERING Layout design: This stage involves the translation of the The neuromorphic engineering could be divided into circuit realized in the previous stage into silicon description neuromorphic modelling, reproducing neurophysiological through geometrical patterns aided by computer aided design phenomena to increase the understanding of the nervous (CAD) tools. This translation process follows a process rule systems and neuromorphic computation which uses the that the spacing between , wire, and wire contacts neuronal properties to build neuron like computing hardware. and so on. The layout is designed to represent the electrical Basically, the former provides the knowledge of the circuit schematics obtained from the algorithm. biological algorithm while the latter translates the algorithm Fabrication: Upon satisfactory verification of the design, the into electrical circuits. This is an iterative process, since the layout is sent to the foundry where it is fabricated. The understanding of the biological algorithm is a very complex process of chip fabrication is very complex. It involves many process. As more knowledge evolves yielding improved stages of oxidation, etching, photolithography, etc. Typically, algorithm, the electrical circuits are revised and improved. the fabrication process translates the layout into silicon or any These circuits then pass through all the stages of developing other semiconductor material that is used. (or chip), which involves the circuit layout, Testing: The final stage of the chip development is called verification, fabrication in foundry and testing and testing. Electronic equipment like oscilloscopes, probes, and subsequent deployment. A brief explanation of each of these electrical meters are used to measure some parameters of the steps is provided below:

Copyright to IJARCCE www.ijarcce.com 3319 ISSN (Print) : 2319-5940 ISSN (Online) : 2278-1021

International Journal of Advanced Research in Computer and Communication Engineering Vol. 2, Issue 8, August 2013 chip, to verify its functionalities based on the chip [6] Herbert Simon: from Doug Stewart's Interview with Herbert Simon. specifications [2]. [7] IBM Researchers Go Way Beyond AI With Cat-Like Cognitive Computing, By Pam Baker TechNewsWorld. Emergent tool is used to implement reinforcement learning in [8] Cognitive Computing 2007: May 2-3, 2007, UC Berkeley Neuromorphic Engineering. Emergent is neural simulation CITRIS and NERSC Sponsored Research. software that is used for creating complex, sophisticated [9] The Playground Experiment: Sony Computer Science lab Paris. models of the brain and cognitive processes. It can also be http://playground.csl.sony.fr/en/pag1-Research-Aim.html used for any task to which neural networks are suited. [10] Genesis Home Page: http://groups.csail.mit.edu/genesis/ [11] Department of Brain and Cognitive Sciences, MIT. http://cocosci.mit.edu/ [12] University of California, Merced, Homepage of Christopher TKello:http://www.ucmerced.edu/faculty/facultybio.asp?facultyid [13] Reverse-Engineering the Brain At MIT, neuroscience and artificial Intelligence are beginning to intersect. By Fred Hapgood Illustration by David Plunkert. [14] IBM Research – Almaden: http://www.almaden.ibm.com/ [15] IBM Moves Closer To Creating Computer Based on Insights From The Brain: http://www- 03.ibm.com/press/us/en/pressrelease/28842. [16] IBM: Computing rivaling human brain may be ready by 2019 by Daniel Terdiman on http://news.cnet.com/8301-13772_3- 10400362-52.html [17] CMU finds human brains similarly organized. By David Templeton. Pittsburgh Post-Gazette (January 4, 2008). Figure 3: Emergent Software for Neural Simulation [18] Cognitive Computing Research Grouphttp://ccrg.cs.memphis.edu/ [19] http://blogs.myspace.com/index.cfm?fuseaction=blog.view . II. CONCLUSION [20] MSc in Cognitive Computing: http://www.gold.ac.uk/pg/msc-

cognitive-computing/ In this paper we have given a critical review of different [21] Complex Cognition Laboratory, Bowling Green State streams arising because of Cognitive sciences, Universityhttp://www.bgsu.edu/departments/psych/faculty/randers/com plexco gnition/ computational Intelligence, Neuroscience and Neuromorphic [22] IBM Researchers Go Way Beyond AI With Cat-Like Cognitive Engineering. There is a new field of science which combines Computing: By Pam Baker TechNews World psychology, computer science, and http://www.technewsworld.com/story/68678.html neuroscience and Electronic Hardware Design. The [23] Published News Paper Article. Online copy available on: http://www.positivefuturist.com/archive.html cognitive neuroscience computing is at a verge to give rise [24] “10 Important differences between Brains and Computers” for many potential mainstream applications. Neuromorphic by chris chatham on http://scienceblogs.com. is one such example. [25] “The Disciplines of Artificial Intelligence” available at In cognitive sciences we have modelled Attention and http://www.learnartificialneuralnetworks.com. [26] What is Cognitive Science? At Berkley University website: Perception based on BDI modelling. Intelligence for ICs has http://ls.berkeley.edu/ugis/cogsci been defined in a generalized way. Emergent is identified is [27] How the Human Brain works? by: Jacob Palme, professor, Stockholm the best option for implementation of Reinforcement University. First version: 22 Jul 2008. Latest revision: 22 Jul Learning. 2008. [28] Speech and Language : language group in NACS, Finally, an exciting, but very difficult, prospect is that http://www.nacs.umd.edu/res/speech.html of an integrated cognitive system which evolves; a system [29] Hao Yang, Chia-Jiu Wang, Yonghong Wang, Heng-Ming Tai, Non which absorbs new knowledge, learn through experience and linear Analysis in Cognition process respond like one of us. (Being able to combine different [30] Barsalou, Perceptual symbol system [31] Neural Network for Human Cognitive State Estimation, Makoto behaviors of humans, showing creativity). Takahashi,Osamu Kubo,Masashi Kitamura and Hidekazu Yoshikawa [32] Psychology 101 Dr Christopher L Heffner REFERENCES [33] Baddeley, A. D. (1986). Working memory. Oxford: Clarendon Press. [1] Mind as Machine: A History of Cognitive Science. [34] A. S. Rao and M. P. Georgeff, Modeling Rational Agents within a BDI- Margaret Boden. Two volumes, xlviii + 1631 pp. Oxford Architecture University Press, 2006. [35] http://discover-your-mind.co.uk/ Ian Heath [2] Encyclopedia of Computer Science, 4th edition, William J. [36] Justin S Feinstein, Murray B Stein, Gabriel N Castillo, Martin P Paulus, Rapaport, John Wiley and Sons Ltd. From sensory processes to conscious perception. [3] Imagination and Choice, Anne C. Dailey, Article first published online: 19 FEB 2010. Law & Social Inquiry, Volume 35, Issue 1, pages 175–216, winter 2010. [4] By Jonah Lehrer. The Boston Globe (April 29, 2007). Available online on http://www.aaai.org/aitopics/pmwiki/pmwiki.php in section of AI Topics. [5] Non-sequential processing: history and future of bridging the semantic gap left by the von Neumann architecture. By Eric Verhulst. Available online at: www.openlicensesociety.org/docs/TP-SIPS971.pdf. Copyright to IJARCCE www.ijarcce.com 3320