Proceedings of 2012 International Conference on Modelling, Identification and Control, Wuhan, China, June 24-26, 2012

An Overview of Hierarchical Temporal Memory: A New Neocortex Algorithm

Xi Chen, Wei Wang , Wei Li

Abstract—The overview presents the development and 1988. Then T.S. Lee and D. Mumford used Bayesian application of Hierarchical Temporal Memory (HTM). HTM network in invariant pattern recognition seen in the visual is a new machine method which was proposed by Jeff cortex [6]. Finally T. Dean proposed a hierarchical Bayesian Hawkins in 2005. It is a biologically inspired cognitive method [7] based on the principle of how human works. The model based on the work by Lee and Mumford [23]ca method invites hierarchical structure and proposes a A 2 (7) (2003), pp. 1434-1448. Full Text via CrossRef to memory-prediction framework, thus making it able to predict model the invariant pattern recognition in 2006. HTM can what will happen in the near future. This overview mainly be considered a form of Bayesian network, where both the introduces the developing process of HTM, as well as its networks consist of a collection of nodes arranged in a principle, characteristics, advantages and applications in tree-shaped hierarchy and all nodes share the same vision, image processing and robots movement, some potential applications by using HTM , such as thinking process, are also computing algorithm, both networks use Bayesian-belief put forward. propagation mechanism. Input data is preprocessed before Index Terms—hierarchical Bayesian network; being fed to the bottom layer of nodes. There are several spatial-temporal; memory-prediction; temporal sequence; feed-forward and feed-back channels through the networks pattern recognition to allow a proper distribution of information throughout the networks. So they both need a set of training data to be put I. INTRODUCTION into the bottom layer of nodes multiple times [2]. N recent years a new theory on brain function has been However, unlike many Bayesian networks, HTMs are Ipresented by [1], who is a brain scientists and self-training, have a well-defined parent/child relationship founder of the redwood research institute. The between each node. What is the most important, HTM main tenets of this new theory can be modeled using emphasizes the significance of “temporal”, pointed that Bayesian network[2], but virtually there exists some every event in the world, is all composed by tiny element [8], differences. This model is called Hierarchical Temporal that’s to say, the world is a hierarchical structure and in the Memory network. It is a stimulant paradigm with a new set final analysis can be broken down into basic constitutions. of bio-inspired suppositions, which putting theories about The temporal sequences of patterns lead to memory. neocortical function into a set of algorithms. HTM theory Inherently HTM method handles time-varying data and incorporates the hierarchical organization of the affords mechanisms for covert attention, in this way, it mammalian neocortex into its topological architecture [3]. achieves prediction. HTM can be thought of a special kind of hierarchical The concept of spatial-temporal [9] was first proposed by Bayesian model. It also uses spatial-temporal theory [4] as a Torsten Hägerstrand in 1970. It was proposed based on the way to encapsulate and learn the structure and invariance of study of human migration patterns, emphasized the problems’ space. Hierarchical organization and importance of time in human activities. During decades of spatial-temporal coding are both well documented years’ research, researchers established the relationship principles of information processing in neural systems. between human’s social behavior and human intelligence Bayesian network [5] was first proposed by J. Pearl in [10]. Sun and Giles presented a useful overview [11] including the characteristics, problems, and challenges for sequence

 learning from recognition and prediction to sequential .The manuscript is submitted on March 15, 2012. This work is decision making. Temporal sequence learning [12] is one of partially supported by National Natural Science Foundation of China (Grant No. 70903026) the most critical components for human intelligence. On the Xi Chen, Associate professor in the Institute of System engineering, other hand, considering that any event exists within the Huazhong University of Science & Technology, Image Processing and space, so spatial factor is indispensable along with temporal Intelligent Control Key Laboratory of Education Ministry of China, Wuhan, factor in human intelligence. Time and space provide Hubei, China, 430074 (Phone: +86-27-87540210. Email: [email protected]) contrasting perspectives on events. A temporal perspective Wei Wang, *Corresponding author, Candidate for Master of the highlights the sequence of transitions, the dynamic changes Institute of System engineering, Huazhong University of Science & from segment to segment, reflecting things in motion. Technology, Image Processing and Intelligent Control Key Laboratory of However, a spatial perspective emphasizes the sequence of Education Ministry of China(Phone: 15549088762. Email: [email protected]) states, the static spatial configuration, reflecting things Wei Li: Associate professor in the Department of Control Science & caught still. Handling the temporal and the spatial at once Engineering, Image Processing and Intelligent Control Key Laboratory of seems out of control, but the dynamic and the static appear Education Ministry of China (Phone: +86-27-87556242. Email: to complement each other [8].Spatial and temporal relations [email protected].)

1004 Proceedings of 2012 International Conference on Modelling, Identification and Control, Wuhan, China, June 24-26, 2012 are learned in a hierarchical architecture. This method is neuroscience and AI. Most algorithms, however, covered currently successfully applied to the recognition of simple only one or few functions about human brain. Hawkins images [13]. studied these theories thoroughly and extracted their HTM, actually, performs temporal and spatial process, redeeming features, proposed the HTM after amalgamation. but not completely copy the theory of spatial-temporal. In HTM firstly learns and memories the spatial patterns, as nature, HTM inherits the spirit of spatial-temporal, which well as those patterns which usually happens at the same exhibiting the two points of thinking and recognition things time, then recognizes the temporal sequence of those spatial in the world, but how does HTM perform the process in patterns appeared one after another, at last, these stored deeds, it involves some other algorithm. In a word, HTM is patterns and sequence help produce prediction when new a integration method which combining hierarchical pattern which is similar to stored pattern comes in. Bayesian network with spatial-temporal model to gather Hawkins admits that many of the ideas in his theory aren’t their advantages, finally result in an advanced algorithm new. What is new, he says, is “putting the correct pieces which can learn and recognize in hierarchy and make together in the right way with an overall theoretical prediction based on the knowledge learned previously. framework”. The rest of this paper is organized as follows: In section 2, Then in 2005, Jeff Hawkins and his fellows Dileep the basic principle and developing process about HTM will George, Donna Dubinsky founded a company called be demonstrated through three sections: 2.1 will briefly [15]. It’s a corporation which applying itself to the introduce HTM; then the first generation of HTM algorithm research of HTM network and aiming at extending the and the contribution which Jeff Hawkins made on human impact of the HTM technology as well as its potential intelligence and will be explained in applications. Now the research carrying through mainly 2.2; in section 2.3, the second generation algorithm will be builds on a software development platform called NuPIC presented. In section 3, the application of HTM will be (Numenta Platform for Intelligent Computing). It’s enumerated and concluded. In section 4, it’s the author’s Numenta’s legacy software which contains several conclusion. generations of versions and provides tools to create, train, test and deploy HTM. So far Numenta has released two II. HTM MODEL versions of NuPIC. The first one is version 1.7.1 called Zeta, released in 2009, which allows programmers to create and A. HTM Overview test HTMs on their own problems with their own data sets. HTM is a biological-inspired method which built on the It can only support the first generation algorithm but not architecture of neocortex and trying to model the process of actively useful for second generation. The new version how human brain handles the information about vision, which is able to provide experiment platform for the second audio, behavior etc, thus leading to memory and prediction. generation algorithm is in the process of developing. An HTM system is trained rather than programming Compared with Zeta, the new version is further realized, the underlying learning algorithms used in it are well-rounded which will incorporate the new generation not specific to particular sensory domains and can be of algorithm, simultaneously permitting non-programmers applied to a broad set of problems that involve modeling to experiment with creating, training and doing inference on complex sensory data. HTM has experienced a long journey HTM prototypes with image classification and recognition. from its rudiment to current achievement. It’s believed that biological principles will drive the next All the progress owes to Jeff Hawkins’ persistence and generation of intelligent computing. And the sustaining exploration. He is a neuroscience investigator and pursuing developing HTM theory is thought of a catalyst for this new his dream of building machines with intelligence. He age with the software development platform. believes that the only way to build intelligent systems is by looking at how the brain works, especially its architecture B. First Generation of HTM Algorithm [14]. Years later, he published his book On Intelligence, As mentioned above, until now Numenta has turned up which indicated the first bridge connecting neuroscience two generations of HTM algorithms. The first generation and artificial intelligence. In this book, Hawkins mentioned which named “The HTM Learning Algorithms” [16] was the function of prediction of neocortex, pointed that proposed by Dileep George and Bobby Jaros who both are prediction is not only the main function of neocortex, but members of Numenta. The algorithm was published on also the basement of intelligence. These predictions come white paper in their company in March 1, 2007. It out based on memories which store in neocortex. New indicated the elementary theory of HTM formed formally, patterns compare with stored patterns to find similarities, following the outline of the theory described by Jeff according to these awake corresponding memories in Hawkins in his book On Intelligence. neocortex, these awaken memories then lead to predictions. In fact, before the release of first generation of HTM That is the framework of Memory-Prediction which also algorithm, much more work has been done by researchers. becomes the new framework of intelligence. Then the AI This mainly included the introduction of fundamental community is taking notice of Hawkins’ work and many concepts and terminology behind HTM. In 2006, Jeff researchers begin to seek deeper connections between published a paper named Hierarchical Temporal Memory:

1005 Proceedings of 2012 International Conference on Modelling, Identification and Control, Wuhan, China, June 24-26, 2012

Concepts, Theory and Terminology[17], It was a precursor of the first generation of HTM algorithm. In the same year, George Dileep pronounced a paper[18] to summarize the possible algorithms behind HTM and expect the potential application fields of HTM. The foremost principle of HTM is time and hierarchy factors in the vision problem. The researchers got the idea through anatomizing neocortex and fond that human and other mammal’s neocortex share the same configuration, namely hierarchy. What’s more, they both recognize objects from a single snapshot of the image without integrating information over multiple time steps. These problems Fig.2. Structure of an HTM network for learning the binary images. This confused researchers and finally the temporal factor caught network is organized in 3 levels. Input is fed in at the bottom level. Nodes the researchers’ attention. Although apparently humans are shown as squares. (This figure is quoted from the reference [16]) recognize with a snapshot, actually we learn with continuously varying data and use this temporal information to obtain important generalization characteristics. Firstly the issue of vision problem was stated, pointed that mammalian recognizes objects in a unsupervised manner, where time acts as a supervisor to tell which patterns belong together and which patterns do not[19]. In other words, although two different images are not totally the same, the fact that they take place close by in time can be used to learn that they are produced by the same cause. This identity is called invariant representation. So Learning to recognize objects involves learning invariant Fig.3 a completely learned node which both the spatial pooler and representations, for an object’s identity remains the same temporal pooler have finished their learning processes. The spatial pooler although it experiences different transformations in the now has 5 spatial groups and temporal pooler has 2 temporal groups. (This world. The concept of invariant representations firstly figure is quoted from the reference [16]) After the appearance of first generation of HTM appeared on Jeff Hawkins’ book On Intelligence. algorithm, researchers in Numenta and other institutions Then involves how HTM works and its training data as didn’t stop their steps for detailed and deeper studying on well as framework. The first generation of HTM algorithm HTM. In 2007, Sara Reese Hedberg wrote a biography [14] takes the recognition of images as an example to illustrate to narrate Jeff Hawkins’ experience on the research of HTM the principle of HTM. Training data is a sequence of binary since he was an academician and his contribution to images presented in pixels, as Fig.1 shows. neuroscience and artificial intelligence. Then one year later, Jeff Hawkins’ student Dileep George delivered his dissertation for doctor degree [20]. This paper presents the theory of how human’s neocortex performs, and then introduces the HTM’s framework and principle in detail. Afterward, George outlined a figure about intelligent machine which is able to learn, categorize and predict [21]. The ultimate purpose of researching the neocortex is to Fig.1. Input data sequence into HTM network, it is a sequence of binary produce intelligent machine that realize artificial images presented in pixels along the time axis. intelligence. HTM network is a hierarchical configuration which Besides, some other evaluation and betterment have contents numbers of nodes, these nodes share the same come forth about HTM. John Thornton and Jolon Faichney algorithm and each node has child node (except bottom have evaluated HTM’s ability to represent temporal nodes) and parent node (except top node), as Fig.2 shows. sequences of input within a hierarchically structured vector Lower nodes sensor smaller range of data and their stability quantization algorithm [22].The result revealed that the is not a patch on higher nodes. On the contrary, higher temporal pooler algorithm is a surprisingly independent nodes receive the output of their child nodes and approach that is immune from the use of preprocessing amalgamate them, thus they sensor larger range of data and techniques. Unlike HTM model which built its framework is more stable. The learning stage of HTM network covers on Bayesian network, Kiruthika Ramanathan and Luping training and inference. Each node operates two steps: Shi proposed a hierarchical temporal-spatial memory model spatial pooler and temporal pooler, as Fig.3 shows. based on neural network [23]. The spatial pooler process uses competitive neural network. It performs comparable

1006 Proceedings of 2012 International Conference on Modelling, Identification and Control, Wuhan, China, June 24-26, 2012 recognition results to HTM and show definitely improvement over MLP. However, as competitive neural network is a full-connection network, the bulky computation can easily lead to the inefficient of the model. C. Second Generation of HTM Algorithm After about three years’ researching and improvement, the second generation of HTM algorithm was published on [24] December 10, 2010 . The paper is finished by chapter Fig.5. Cells in different states. Columns with predicted cells only activate under the cooperation of the authors, but this is a draft predicted cells. Columns with no predicted cells activate all the cells in the version that some information is not available in it, the column. (This figure is quoted from the reference [24]). detailed and complete version is on the way of complement and takes image recognition as an example to illustrate and consummating. HTM’s effect on recognition, while the second generation In general, compared with the first generation of HTM pays relatively more attention to application and the new algorithm, the second generation mainly represents the algorithm is depicted in adequate detail so that a problem of recognition on a level of global perspective, programmer can easily understand and implement it if studies the architecture of neocortex from the view of desired. Besides, pseudocodes of the algorithm’s two biology anatomy. It mines the nerve connection in the processes namely spatial pooler and temporal pooler are neocortex and manages to find out the function of each type presented in the document that favor of non-experienced of nerve. In the second generation, node in each level is programmers coming to the road of implementation as soon extended to a column, not just a single node in the first as possible. The new algorithm introduces some generation, as Fig. 4 shows. Each column contains several terminologies which don’t appear in the first generation cells. Among cells there exist put-forward connection and such as region, sparse distributed representation. There is a lateral connection. The cells can be one of three states: biggest difference between the two algorithms, or calling it active, inactive and predictive. If a cell is active due to improvement, the new algorithm highlights the HTM’s put-forward input, then it is in active state, on the other function of prediction as well as key properties, and also hand, if it is active due to lateral connection, it is in breaks down the procedure a little further into three steps, predictive state. Columns only activate predicted cells. which is thought of the most influential characteristic Those with no predicted cells in them activate all the cells among existing AI algorithm. Each column which contains in the column, as Fig.5 shows. The aim of this treatment is several nodes is called variable order sequence, and each to make HTM level represent the same input in many column which contains only one node is called first order different contexts. The cell in predictive state gives a signal sequence. Variable order HTM is ideally suited for of what will happen next based on current training. recognizing time-based sequences. Compared to the first generation version, the document Thanks to the new algorithm is updated not long before, describes the new algorithm for learning and prediction in some improvements about it don’t come forth. However, detail. The second generation can be considered as the around the releasing, some other scholars put forward their continuing work about On Intelligence and an improvement ideas from different perspectives. David Rozado, Francisco B. Rodriguez, and Pablo Varona optimized the HTM about first generation. The first generation mainly [25] characterizes about HTM learning algorithm within single algorithm for multivariable time series . In allusion to the node and how nodes operate and connect to each other in a problems involving multi-variable time series where hierarchy to achieve system level results minutely, it samples unfold over time with no complete spatial primarily applies itself to vision problem representation at any point in time, HTM feels intractable and doesn’t perform well. Then this paper extends the traditional HTMs’ principles by means of a top node that stores and arranges sequences of input patterns representing the spatial-temporal structure of instances to be learned. The extended model is tested in the problem of sign language recognition which is used by deaf people and consists of an ordered sequence of hand movements. The result reveals much better performance relative to traditional HTM. HTM is on the way of continuing development and

Fig.4. A section of HTM level, HTM level are comprised of many cells. attracting more and more focus of researchers coming to The cells are organized in a two dimensional array of columns. (This figure neocortex and artificial intelligence fields. The researchers is quoted from the reference [24]). pursue optimized architecture from detailed points and at the same time many applications based on HTM lift the boom of computing intelligence.

1007 Proceedings of 2012 International Conference on Modelling, Identification and Control, Wuhan, China, June 24-26, 2012

III. APPLICATION adopted HTM to receive the exacted features from EKF and Pattern recognition has always been a hot research classify the static actions or dynamic signals which are project. In these years, researchers have presented many varying with both time and space. The experimental results theories and methods as well as improved optimizations to show that the HTM and EKF based method can perform the questions of image recognition, human action very high accuracy for the dynamic action detection. recognition and so on. For instance, Yunzhi Jiang has Another two papers involving human body movement proposed a Bayesian particle swarm optimization for image detection which used HTM are respectively about human [40] [41] segmentation [26]. Qiming Fu has put forward some body movements during daily life and fall detection , . relevance feedback techniques and genetic algorithm for In these applications, HTM perform together with one or image retrieval [27]. Fawang Liu has researched human even more other techniques. action recognition from silhouettes’ perspective using Beside these, HTM has some other applications. Firstly, manifold learning [28]. in objects categorization, in 2007, Adam B. Csapo, Peter However, HTM algorithms’ appearance stimulates the Baranyi and Domonkos Tikk used HTM and another [42] researchers interests in recognition and prediction not only method to undergo object categorization . Another [43] vision recognition but also audio and behavior. Numbers of classification application about land-use illustrates the production come out and exhibit promising application recognition and classification using the photograph of land, prospect. Nearly all the papers were released after 2007, achieving promising classification accuracy. Secondly, [44] especially recent years, with the developing of algorithm HTM is used in the telemedicine network as an learning, wider and deeper application domains have been alternative method of traditional personal diagnosis. Thirdly, taken on, come with the larger number of application results in the problem of accurately representing asymmetric [45] arose. warfare, HTM acted a strongly weapon . Especially In the domain of vision recognition, HTM has been used under the current world’s war situation of network-centric in many aspects. Tomasz Kapuscinski and Marian Wysocki warfare transformed from last century’s force-to-force used HTM to recognize signed polish words [29]. One year combat, HTM can help human understand the warfare later, Tomasz Kapuscinski published another paper [30] model and analyze how human process the complex war applying HTM to recognize hand shape when hand in state information as well as predicting the consequences of of dynamic. Not only in hand movement, but in traffic corresponding tactics. Lastly, it comes to the evaluation signal recognition, HTM also plays a perfect assistant. Wim function of HTM. Wim J.C. Melis, Shuhei Chizuwa and J.C. Melis studied the function of color channels on traffic Michitaka Kameyama used HTM method on user support light using HTM framework [31]. In hand-written digit system, and toke the cellular phone intention estimation as [46] recognition, on account of the nonstandard characteristic of an example , compared to the performance of Bayesian hand-written digits, HTM were used mainly for prediction Network, it found that HTM required little effort for and achieved approving accuracy [32], [33]. Other applications designing the application and could easily be optimized. about vision recognition involves image retrieval [34] and Subsequently, they published another paper which also face recognition [35]. about HTM’s function on intention estimation information Up to now HTM has almost exclusively been applied to appliance system, the difference was that the authors [47] image processing. However, the underlying theory can also supposed a possible VLSI architecture for HTM , be used as an approach to active perception of audio signals although the expense and computing efficiency are waiting [48], [36]. Actually, the speech recognition problem can be most for solving. Other evaluations cover automated design [49] [50] easily cast in a form similar to image recognition. In despite and automated risk assessment . of the present implementation is not perfectly suited for According to above, HTM appears to be a potential handling signals that encode information mainly in dynamic intelligence computing method that has caught large changes, the result all the same shows that the HTM numbers of investigators’ attention to research and try to approach holds promises for speech recognition. apply it in aspects which can realize intelligence. It’s a Not only about vision and audio signals, but some simple hopeful method which contains promising application behavior signals, HTM can also be used to deal with. future. Kwang-Ho Seok and Yoon Sang Kim used HTM to author robot motion [37] to produce humanoid machine with 26 IV. CONCLUSION degrees of freedom. N. Farahmand used HTM to build a HTM is a soft computing method which models the high-level self-organizing for a soccer bot [38] neocortex. It is derived from biology and is suitable for S. Zhang, and M. H Ang Jr.etc proposed a two-stage action tasks that are easy for people but difficult for computers recognition approach for detecting arm gesture related to such as recognizing objects, making predictions, and human eating or drinking [39] . The process contained two discovering patterns in complex data. It is a masterpiece steps: feature exaction and classification. The former displayed in the field of artificial intelligence and leading applied Extended Kalman filter (EKF) to exact features the computing intelligence coming to new age, also let the from arm action in a three dimensional space, the latter research of intelligent machine see the ray of hope. Now

1008 Proceedings of 2012 International Conference on Modelling, Identification and Control, Wuhan, China, June 24-26, 2012 although the algorithm is immature and application in more [3] R. K. Moore., “A comparison of the data requirements of automatic speech recognition systems and human listeners”, Proceedings of the intelligent manners exist restriction, persistent study is 8th European Conference on Speech Communication and going on all the while and the achievements about Technology, Eurospeech 2003, pp. 2582-2584. development as well as application have been popping up [4] J. A. Starzyk, and H. He, “Spatio–Temporal Memories for Machine with happy regularity. It’s believed that intelligent Learning: A Long-Term Memory Organization”, IEEE TRANSACTIONS ON NEURAL NETWORKS, vol. 20, no. 5, pp. computing and artificial intelligence are on the way of 68-780, 2009. prosperity and the principles by which HTM technology [5] J. Pearl. “Probabilistic Reasoning in Intelligent Systems”, Networks operates will lay the foundation of machine intelligence. of Plausible Inference, Morgan Kaufman, San Francisco, CA (1988). [6] T.S. Lee and D. Mumford, “Hierarchical Bayesian inference in the Up to now, the algorithm of HTM is on the way of ”, Journal of the Optical Society of America A, vol. 2, durative optimizing, and the application field is extending no. 7, pp. 1434–1448, 2003. from image recognition to more other aspects. However, [7] T. Dean., “Scalable inference in hierarchical generative models”, Ninth International Symposium on Artificial Intelligence and most applications researched now aim at basic sense organs Mathematics, 2006. in human such as vision, audio and movement, applications [8] J. M. Zacks and B. Tversky, “Event Structure in Perception and about advanced logic analysis appear rarely. For example, Conception”, Psychological Bulletin, vol. 127, no. 1, pp. 1-79, 2001. [9] T. Hägerstrand, “What about People in Regional Science?”, Regional human’s thinking process (such as recognition and reaction Science, vol. 24, no. 1, pp. 6-21, 1970. for outside information in daily life) may be a promising [10] E. Piippel, “A hierarchical model of temporal perception”, Trends in trend, as the most novel characteristic of HTM is the Cognitive Sciences, vol. 1, no. 2, pp. 56-61, 1997. [11] R. Sun and C. L. Giles, “Sequence learning: From recognition and function of prediction. In terms of dealing information, prediction to sequential decision making”, IEEE Intell. Syst., vol. 16, human can predict what will happen next when he is no. 4, pp. 67–70, 2001. confronted with a similar phenomenon which has seen [12] J. A. Starzyk and H. He., “Anticipation-Based Temporal Sequences before, thus he is able to take measures timely to react to Learning in Hierarchical Structure”, IEEE TRANSACTIONS ON NEURAL NETWORKS, vol. 18, no. 2, pp. 344-358, 2007. the affair. [13] S. E. AVONS and K.OSWALD, “Using temporal order to identify The feasibility of using HTM algorithm in cognition and spatial reference frames”, Perception & Psychophysics, vol. 70, no. 6, reaction for human is represented as follow: firstly, the main pp. 1068-1080, 2008. [14] S. R. Hedberg., “Bridging the Gap between Neuroscience and AI”, research object is the same, for both are human; secondly, IEEE Computer Society, pp. 4-7, 2007. the target remains consistent, using HTM in information [15] (Website Online Sources style) J. Hawkins, Numenta, 2005, cognition and reaction for human is to reach the aim of Available: http://www.numenta.com/about-numenta/people.php. [16] D. George and B. Jaros, “The HTM Learning Algorithms”, Numenta predicting what will happen next; thirdly, also the most Inc, 2007.5. important one, both of the inputs have temporal identity, the [17] J. Hawkins, “Hierarchical Temporal Memory: Concepts, Theory and messages which human received also possess the Terminology”, White Paper, Numenta Inc, 2006. [18] D. George, “Hierarchical temporal memory: Theory and characteristic of temporal relativity. Messages which always applications”, 28th Annual International Conference of the IEEE happen together or come one after another suggesting that Engineering in Medicine and Biology Society, pp. 6643, 2006. they have close relationship, thus when human receives one [19] D. D. Cox, P. Meier and N. Oertelt, “Breaking position invariant object recognition”, Nature Neuroscience vol. 8, pp. 1145-1147, message of them, he will predict the next message he has 2005. seen before. As for inputs, they need to be taken [20] D. George, “How The Brain Might Work:A Hierarchical and quantitative, because messages are mostly described fuzzy, Temporal Model for Learning and Recognition”, A Dissertations for in addition, the time factor should not be ignored. So a The Degree of Doctor of Philosophy, 2008. [21] D. George, “How to Make Computers that Work Like the Brain”, sequence of binary vectors labeled time flag can be taken Numenta Inc., 2009. into consideration. In the end, a test for training result can [22] J. Thornton, J. Faichney and M. Blumenstein, “Character be made to validate the correctness. A feed-back channel Recognition Using Hierarchical Vector Quantization and Temporal Pooling”, AI, vol. 5360, pp. 562–572, 2008. can be added to the network. In this way, correct prediction [23] K. Ramanathan, L. Shi and J. Li, “A Neural Network Model for a will strengthen the memory, and inaccurate prediction will Hierarchical Spatio-temporal Memory”, ICONIP 2008, Part I, vol. become an error signal to modify the training. 5506, pp.428–435, 2009. [24] J. Hawkins, S. Ahmad and D. Dubinsky, “Hierarchical Temporal Memory including HTM Cortical Learning Algorithms”, ACKNOWLEDGMENT Numenta.2010, 12. [25] D. Rozado, F. B. Rodriguez and P. Varona, “Optimizing Hierarchical The authors sincerely thank all participators for their hard Temporal Memory for Multivariable Time Series”, ICANN 2010, work and valuable suggestions that have lead to the Part II, vol. 6353, pp.506–518, 2010. improvements of this paper. At the same time, the authors [26] Y.Z. Jiang, Z.F. Hao and G.Z. Yuan, “Multilevel thresholding for image segmentation through Bayesian particle swarm optimization”, gratefully acknowledge anonymous reviewers, who help to International Journal of Modelling, Identification and Control, vol. improve the quality of this paper. 15, no.4, pp. 267 – 276, 2012. [27] Q.M. Fu, Q. Liu and X.Y. Wang, “Relevance feedback techniques and genetic algorithm for image retrieval based on multiple features”, REFERENCES International Journal of Modelling, Identification and Control, vol. [1] J. Hawkins and S. Blakeslee, “On Intelligence”, Henry Holt, New 14, no.4, pp. 279 – 285, 2011. York, 2004. [28] F.W. Liu and H.B. Deng, “Human action recognition from [2] P. Yalamanchili, S. Mohan and R. Jalasutram, “Acceleration of silhouettes using manifold learning and MDA”, International Journal hierarchical Bayesian network based cortical models on multicore of Modelling, Identification and Control, vol. 12, nos. 1/2, pp. 36-41, architectures”, Parallel Computing 2010, vol. 36, pp. 449-468. 2011.

1009 Proceedings of 2012 International Conference on Modelling, Identification and Control, Wuhan, China, June 24-26, 2012

[29] T. Kapuscinski, M. Wysocki. “Using Hierarchical Temporal Memory [50] R. J. Rodriguez, J. A. Cannady, “Automated Risk Assessment: A for Recognition of Signed Polish Words”, Computer Recognition Hierarchical Temporal Memory Approach”, 9th WSEAS System, vol. 3, no.57, pp. 355–362, 2009. International Conference on Data Networks, Communications, [30] T. Kapuscinski, “Using Hierarchical Temporal Memory for Computers, pp. 11:53-57, 2010. Vision-Based Hand Shape Recognition under Large Variations in Hand’s Rotation”, ARTIFICIAL INTELLIGENCE AND SOFT COMPUTING, Part II, vol. 6114, pp. 272-279, 2010. [31] W. J. C. Melis, M. Kameyama, “A Study of the Different Uses of Colour Channels for Traffic Sign Recognition on Hierarchical Temporal Memory”, Fourth International Conference on Innovative Computing, Information and Control, pp. 111-114, 2009. [32] B. Bobier, “Handwritten Digit Recognition using Hierarchical Temporal Memory”, 2007, 8. [33] S. Štolc, I. Bajla, “On the Optimum Architecture of the Biologically Inspired Hierarchical Temporal Memory Model Applied to the Hand-Written Digit Recognition”, MEASUREMENT SCIENCE REVIEW, vol. 10, no. 2, pp. 28-49, 2010. [34] B. A. Bobie, M. Wirthr, “Content-Based Image Retrieval Using Hierarchical Temporal Memory”, 08 Proceeding of the 16th ACM international conference on Multimedia, 2008. [35] S. Svorad, B. Ivan, “Application of the computational intelligence network based on hierarchical temporal memory to face recognition”, Proceedings of the 10th IASTED International Conference on Artificial Intelligence and Applications, AIA, pp. 185-192, 2010. [36] J. V. Doremalen, L. Boves, “Spoken Digit Recognition using a Hierarchical Temporal Memory”, ISCA, pp. 2566-2569, 2008. [37] K.H. Seok, Y. S. Kim, “A new robot motion authoring method using HTM”, International Conference on Control, Automation and Systems, pp. 2058-2061, 2008. [38] N. Farahmand, M.H. Dezfoulian and H. GhiasiRad, “A Nouri. Online Temporal Pattern Learning”, Proceedings of International Joint Conference on Neural Networks, pp. 797-802, 2009. [39] S. Zhang, M. H Ang Jr and W. Xiao, “Detection of Activities for Daily Life Surveillance: Eating and Drinking”, 2008 10th IEEE Intl. Conf. on e-Health Networking, Applications and Service, pp. 171-176, 2008. [40] F. Sassi, L. Ascari and S. Cagnoni, “Classifying Human Body Acceleration Patterns Using a Hierarchical Temporal Memory”, AI*IA 2009, EMERGENT PERSPECTIVES IN ARTIFICIAL INTELLIGENC, vol. 5883, pp. 496–505, 2009. [41] S. Cagnoni, G. Matrella and M. Mordonini, “Sensor Fusion-oriented Fall Detection for Assistive Technologies Applications”, 2009 Ninth International Conference on Intelligent Systems Design and Applications, pp. 673-678, 2009. [42] A. B. Csapo, P. Baranyi and D. Tikk, “Object Categorization Using VFA-generated Nodemaps and Hierarchical Temporal Memories”, 5th IEEE International Conference on Computational Cybernetics, pp. 257-262, 2007. [43] A. J. Pereaa, J. E. Merono and M. J. Aguilera, “Application of Numenta Hierarchical Temporal Memory for land-use classification”, South African Journal of Science, vol. 105, pp. 370-376, 2009. [44] A. R. W. Boone, “Image Processing and Hierarchical Temporal Memories for Automated Retina Analysis”, Presentation Session: Measurement Sciences and Imaging Technologies. [45] J. Sherwin, D. Mavris, “Hierarchical Temporal Memory Algorithms for Understanding Asymmetric Warfare”, IEEEAC paper, pp. 1-10, 2009. [46] W. J.C. Melis, S. Chizuwa and M. Kameyama, “Evaluation of Hierarchical Temporal Memory for a Real World Application”, Fourth International Conference on Innovative Computing, Information and Control, pp. 144-147, 2009. [47] W. J. C. Melis, S. Chizuwa and M. Kameyama, “Evaluation of the Hierarchical Temporal Memory as Soft Computing Platform and Its VLSI Architecture”, 39th International Symposium on Multiple-Valued Logic, pp. 233-238, 2009. [48] J. Hartung, J. McCormack, “Support for the Use of Hierarchical Temporal Memory Systems in Automated Design Evaluation: A First Experiment”, Proceedings of the ASME 2009 International Design Engineering Technical Conferences & Computers and Information in Engineering Conference, pp. 1-10, 2009. [49] F. Jacobus, J. McCormack and J. Hartung, “The Chair Back Experiment: Hierarchical Temporal Memory and the Evolution of Artificial Intelligence in Architecture”, International Journal of Architectural Computing, vol. ,8, no. 2, pp. 151-164, 2010.

1010