PROOF Contents List of Figures viii List of Table x List of Colour Plates xi Preface xii Acknowledgements xiv 1 PERCEPTION AS GATEWAY TO THE WORLD 1 2 THE MACHINERY: UNDERSTANDING THE NEURAL CODE 14 3 VISION 1: BRIGHTNESS 30 4 VISION 2: COLOUR 44 5 VISION 3: FROM IMAGES TO SPACE 54 6 VISION 4: TIME AND MOTION 71 7 HEARING 1: SOUND AND NOISE 85 8 HEARING 2: COMPLEX AUDITORY INFORMATION 99 9 CHEMICAL SENSES: SMELL AND TASTE 112 10 BODY SENSES: FROM THE CONTROL OF POSTURE TO TOUCH 124 11 FOCUS OF INTEREST: ATTENTIONAL MODULATION OF PERCEPTION 137 12 MANAGING INFORMATION FLOW THROUGH INTEGRATION OF SENSORY AND MOTOR SYSTEMS 148 13 MAKING SENSE: THEORETICAL APPROACHES TO PERCEPTION 161 References 171 Index 190 vii 99780230_552661_01_prexiv.indd780230_552661_01_prexiv.indd vviiii 112/18/20092/18/2009 66:31:37:31:37 PPMM PROOF PERCEPTION AS GATEWAY 121 TO THE WORLD OVERVIEW Why is perception such an important topic of study for psychologists? Throughout every single day of our life we incessantly and effortlessly solve complex tasks related to the collec- tion and interpretation of sensory input, and the planning and execution of action based on what is perceived. We (usually) experience little diffi culty in preparing a sandwich, catching a ball, riding a bicycle, or crossing a busy road – but each of these seemingly simple tasks requires a huge amount of sensory information processing! Only when we observe the hope- less efforts of robots facing much, much simpler challenges (like walking up a step), do we start to appreciate how diffi cult it is to navigate and coordinate movements in a group of independently moving individuals, for instance, on a crowded dance fl oor. To address these questions, and to understand more generally how a person collects knowledge about the world and acts in the world, the information processing paradigm is introduced, together with the computer metaphor for the brain. This approach is closely linked, through the specifi c relationship between brain and perception, and more generally the relationship between brain and mind, to the study of brain function, which embeds perception in a vari- ety of scientifi c disciplines that help us to analyse and conceptualise human behaviour. The attempt to localise mental functions in the brain is an illustration of how the information processing approach, and neuroscience, are highly relevant to gain some deep understanding of psychological phenomena. Sensory systems usually are treated as information processing channels that are tuned to particular signals (such as sounds, or odours) and used to solve particular tasks (such as communication). Studying such mechanisms from a scientifi c and/ or engineering perspective allows us to tackle questions of how their designs are optimised in the context of evolutionary adaptation and ecological constraints. In this framework, percep- tion can be described as the window between the physical world and mental states. WHAT IS THE PURPOSE OF THIS BOOK? In a time of rapidly growing competition for resources, be it the time invested by a student in reading a book (as well as the time spent by the author writing it), or the timber harvested from shrinking forests and processed into printing paper at high costs in terms of energy and water pollution, the fi rst question when picking up a book and beginning to turn the pages is ‘why should I read this book?’ The most simplistic answer for the Psychology student, ‘because it will help me to get my degree’, is certainly insuffi cient, and not a good reason to dig into hundreds of pages about phenomena and contexts that sometimes even present some challenge to understand. The reason for writing and reading this book, and for seri- ously engaging with its topic, is the fact that sensation, perception and action have a fun- damental relevance for all aspects of human behaviour and thinking. How can you prepare 1 99780230_552661_02_cha01.indd780230_552661_02_cha01.indd 1 112/18/20092/18/2009 77:06:59:06:59 PPMM PROOF 2 Sensation, Perception and Action your tea and sandwich in the morning? How do you coordinate getting dressed, picking up – and keeping hold of – your bag? How do you cross the road without getting run over, ride your bike without falling over, or steer your car through dense traffi c without getting involved in an accident? How do you fi nd your favourite cereal in the supermarket and pick it off the shelf? How do you follow, throw or catch a ball? How do you communicate with the people around you? And how do you actually read a book? In all our daily activities we need to interact with the world surrounding us, and for any interaction we need to use our senses to collect information and control our actions. And yet all of these processing steps are happening completely unaware and effortless! Without being equipped with a sensory system and perception, however, no autonomous agent would be able to act, nor survive, in its given environment. The little sketch in Figure 1.1 illustrates this crucial role of sensation and perception: the presence of an apple in the environment is picked up by the sensory system (through the eyes, or possibly by smell) and recognised as a real instance of the concept ‘apple’, which can trigger the behavioural response of grasping and eating (or avoidance, if the apple turns out to be rotten). The purpose of this book, therefore, is to provide a basic introduction to the function of sensory information processing in the context of action control and of a particular lifestyle and environment, and to give some clues about how evolution shaped these systems to make them effi cient and reliable, so that an agent (such as a person) can safely navigate in a com- plex environment and survive. It is important to note that the phrase ‘function’ has a dual meaning. On one hand this word relates to a specifi c mechanism, i.e. to the question ‘how does it work?’, which will be touched mainly in an abstract way in this book, because the actual workings of the neural machinery is the topic of more specialised neuroscience text- books. On the other hand the word ‘function’ relates to a purpose, i.e. to the question ‘how is it useful, what is it made for?’, which relates much more closely to the behavioural and ecological context. In this distinction, however, one should not forget that through evolu- tionary mechanisms the actual design of mechanisms is closely linked to its role in behaviour and environment, similar to the slogan of the Bauhaus arts and design school ‘form follows Figure 1.1 Grasp, eat ... Sensation and perception allows a person to detect real objects in the outside world and to control the appropriate action in response to objects and events. In this example an apple is picked up by the visual system of our little egghead, recognised as an instantiation of a highly relevant class of objects (the thought ‘this is an apple’), and the appropriate response will be triggered and again guided by sensory information (grasp, eat …). 99780230_552661_02_cha01.indd780230_552661_02_cha01.indd 2 112/18/20092/18/2009 77:06:59:06:59 PPMM PROOF Perception as Gateway to the World 3 function’. As a result of this conceptual framework, the current text differs from many other textbooks on the topic of sensation and perception by going beyond the mere description of perceptual phenomena, and focussing on the adaptation of particular perceptual mecha- nisms in the context of their use to solve tasks that we are facing in our everyday ‘struggle for survival’. As such this textbook, whereas mainly directed at students of Psychology and in its level of detail written for psychologists with limited previous exposure to an engineer- ing perspective, can also serve as introduction to the topic for computer scientists, engineers and biologists. REPRESENTING THE OUTSIDE WORLD So how do we perceive the outside world? What initially seems to be a straightforward question turns out to be a rather tricky philosophical problem. How do we know where we are, what is in the space surrounding us, what the effects of our actions are, how do we communicate with others and understand messages in a complex society? The answer seems to be very simple: we use our senses. We see the furniture, the walls, the trees in our environment, we hear birds, cars and airplanes, and what other people are saying to us, we smell burning toast, and sense the heat (and sometimes the pain) when trying to remove it from the toaster, we feel the knife in our hand and taste the melting butter and the fl avour of the jam. Very simple, and yet quite complicated when you think more thoroughly about it – there are myriad tricky questions lurking behind these everyday perceptions and actions, some of which will be answered in this book. How do we steer clear of furniture, know the distance away of walls and trees, how can we judge the speed of an approaching car? How do we recognise the voice of a friend, the face of our grandmother? What makes us smell the toast burning? When do we feel the heat and when does it turn into pain? How do we know how to grasp the knife at its handle and not at its blade? What makes us discriminate the taste of strawberry and raspberry jam? The com- mon theme behind these questions is the collection of information about the outside world, in order to know, to understand and to act. It could be argued that through our senses we are building a representation of the outside world in our mind.