Emulating the Perceptual System of the Brain for the Purpose of Sensor Fusion
Total Page:16
File Type:pdf, Size:1020Kb
HSI 2008 Krakow, Poland, May 25-27, 2008 Emulating the Perceptual System of the Brain for the Purpose of Sensor Fusion Rosemarie Velik, Member, IEEE, Roland Lang, Member, IEEE, Dietmar Bruckner, Member, IEEE, and Tobias Deutsch, Member, IEEE Abstract — This work presents a bionic model derived II. STATE OF THE ART from research findings about the perceptual system of the There have already been made various attempts to merge human brain to build next generation intelligent sensor fusion systems. Therefore, a new information processing information coming from various sensory sources. The principle called neuro-symbolic information processing is research area generally first mentioned when discussing introduced. According to this method, sensory data are such tasks is the area of sensor fusion. In literature, there processed by so-called neuro-symbolic networks. The basic cannot be found only one but various definitions of the processing units of neuro-symbolic networks are neuro- term sensor fusion. Principally, sensor fusion is concerned symbols. Correlations between neuro-symbols of a neuro- with the combination of sensor data or data derived from symbolic network can be learned from examples. Perception sensory data in order to produce enhanced data in form of is based on sensor data as well as on interaction with an internal representation of the process environment. The cognitive processes like focus of attention, memory, and achievements of sensor fusion are robustness, extended knowledge. Additionally, a mechanism for evaluating perception by emotions is suggested. spatial and temporal coverage, increased confidence, reduced ambiguity and uncertainty, and improved Keywords — Humanlike Perception, Sensor Fusion, resolution [8]. The research field of sensor data fusion is Neuro-symbolic Networks, Learning, Knowledge-based relatively recent and dynamic. Therefore, a standard Systems, Focus of Attention. terminology has not yet evolved. There are widely used the terms “sensor fusion”, “sensor integration”, “data fusion”, “information fusion”, “multi-sensor data fusion”, I. INTRODUCTION and “multi-sensor integration” [2], [28]. Data for sensor fusion can come from one single sensor taken from HE human brain is a highly complex system, which is multiple measurements subsequently at different instants capable of performing a huge range of diverse tasks. T of time, from multiple sensors of identical types, or from One capability of the brain is to process information sensors of different types. Applications for fusion are coming from thousands and thousands of sensory various and range from measurement engineering and receptors and integrating this information into a unified production engineering over robotics and navigation to perception of the environment. Up to now, technical medicine technology and military applications [19], [25]. systems used for machine perception can by far not There have been proposed various models for sensor compete with their biological archetype. Having available fusion. However, up to now, the selection of a model a technical system, which is capable of perceiving objects, strongly depends on the special application. There does events, and situations in a similar efficient manner as the not yet exist a model for sensor fusion that is generally brain does, would be very valuable for a wide range of accepted. Many researchers even point out that it is very applications. Examples for applications are automatic unlikely that one technique or architecture will provide a surveillance systems in buildings and autonomous robots. uniformly, superior solution [9]. To perceive objects, events, and situations in an In [21], it is pointed out that it generally accepted that environment, sensors of various types are necessary. The sensor fusion in the perceptual system of the human brain challenge that has to be fenced for perceptive tasks is the is of far superior quality than sensor fusion achieved with merging and the interpretation of sensory data from existing mathematical methods. Therefore, it seems to be various sources. The aim of this paper is to introduce a particularly useful to study biological principles of sensor model for integrating and interpreting such data. As fusion. Such studies can on the one hand lead to better humans can perceive their environment very effectively, technical models for sensor fusion and on the other hand the perceptual system of the brain serves as archetype for to a better understanding of how perception is performed model development. Particularly, research findings from in the brain. Sensor fusion based on models derived from neuroscience and neuro-psychology are the guides in the biology is called biological sensor fusion. Models for development process. biological sensor fusion based on neural networks have been proposed in [5] and [15]. There have also been made Rosemarie Velik, Roland Lang, Dietmar Bruckner, and Tobias attempts to merge sensory data by transforming sensor Deutsch are with the Vienna University of Technology, Institute of data into symbols. Computer Technology, 1040 Vienna, Austria. {velik, langr, bruckner, deutsch}@ict.tuwien.ac.at Approaches to process sensor information symbolically have been described by [3], [13], [18], [22], and [23]. There are often suggested layered architectures for this nerve cells but in terms of symbols. Mental processes are purpose. Fusion of sensor data from a set of heterogeneous often considered as a process of symbol manipulation or homogeneous sensors, soft sensors, and history values [11]. of sensor data is generally called direct fusion. However, Learning and Adaptation there does also exist indirect fusion, which uses The perceptual system of the human brain is not fully information sources like prior knowledge about the developed at birth. Although certain patterns need to be environment and human input. Furthermore, it is possible predefined by the genetic code, lots of concepts and to fuse the outputs of direct and indirect fusion. In correlations concerning perception are learned during literature, different models for such hybrid systems have lifetime [20]. been described: [2], [4], [7], [10], [26]. Influence from Focus of Attention According to the hypothesis of focused attention, what we III. CHARACTERISTICS OF HUMAN PERCEPTION see is determined by what we attend to. At every moment, The aim of this paper is to introduce a new model for the environment presents far more perceptual information sensor fusion for the purpose of machine perception, than can be effectively processed. Attention can be used to which is based on research findings about the perceptual select relevant information and to ignore irrelevant or system of the human brain. To develop such a model, in a interfering information. Instead of trying to process all first step, characteristics of human perception have to be objects simultaneously, processing is limited to one object identified. Figure 1 gives an overview about important in a certain area of space at a time [17]. mechanisms and factors that form and influence human Influence from Knowledge and Memory perception. The mentioned characteristics are derived Perception is facilitated by knowledge. Prior knowledge is from research results of neuroscience and neuro- often required for interpreting ambiguous sensory signals. psychology about the perceptual system of the human Much of what we take for granted as the way the world is brain. – as we perceive it – is in fact what we have learned about the world – as we remember it. Much of what we take for perception is in fact memory. We frequently see things that are not there, simply because we expect them to be there [12]. Emotional Evaluation For perception, there are most often only considered the detection and the processing of stimuli from the external environment. However, the perception of objects, events, and situations makes little sense if we do not know what influence they have on our body. In the human brain, an evaluation of perceptual images is performed by emotions. The basic function of emotions in perception is to classify objects, events, and scenarios as good or bad. Emotions Fig. 1. Characteristics of Human Perception are necessary, to react adequately on perceived objects, events, and situations [27]. Diverse Sensory Modalities To perceive the external environment, our brain uses IV. BIONIC MODEL FOR PERCEPTION multiple sources of sensory information derived from several different modalities including vision, touch, and A. Neuro-symbolic Information Processing audition. The combination and integration of multiple To fulfill the first five characteristics mentioned in the last sources of sensory information is the key to robust section, a new concept of information processing is perception [10]. introduced, which is called neuro-symbolic information Parallel Distributed Information Processing processing. This concept is outlined in the following. As just outlined, for perception, information from various sources is processed. However, the perceptual system is Neuro-symbols as Basic Processing Units no unitary central unit that processes all information in The basic information processing units for neuro-symbolic one step. Instead, sensory information is processed in information processing are so-called neuro-symbols. The parallel [20]. inspiration for the usage of such neuro-symbols came Information Integration over Time from the following observations made from neuroscience To perceive objects, events, and