Times Literary Supplement JUNE 23, 1972 1

Seeing as thinking: questions, with hearing the runner- up, for sight dominates by its an active theory of giving us immediate external reality. By simply looking we seem to understand what we see. Richard Gregory This close association between seeing and knowing makes the Theories of perception – of what of vision attractive not only happens to bridge the to philosophers but also to extraordinary gap between sensory experimental psychologists and stimulation and our experience of physiologists who hope to discover external objects-have a long in the brain mechanisms serving history, of astonishing variety. our experience and knowledge of Speculation goes back to the the world. By coming to beginning of recorded philosophy- understand how we see might we and scientific work on perception not at one stroke also discover how escapes the philosophical we think, remember, formulate questions and dilemmas only when hypotheses, appreciate beauty and- it narrows inquiry by over- most mysterious accept pictures blinkering specialization. How we and words as symbols conveying see remains essentially mysterious not merely present reality but other after a century of intensive realities distant in space and time? experiment, by such a variety of And if seeing involves all this, scientists that aims and surely the net of understanding communication can be lost must be cast wide. between them. An adequate Perceptual theories form a theory should include not only the spectrum -- from passive to active favoured sense of sight but also: theories. Passive theories suppose hearing, touch, hot and cold, taste, that perception is essentially smell, balance and position of the cameralike, conveying selectcd limbs, the various kinds of pain; aspects of objects quite directly, as and tickle, from its irritation to though the eyes and brain are sensuous pleasure and delirious undistorting windows. The baby, laugh – making. it is supposed, comes to see not by

using cues and hints to infer the To the philosopher and the world of objects from sensory data experimental scientist, it is how we but by selecting useful features of see that offers the most exciting Times Literary Supplement JUNE 23, 1972 2 objects available to it directly; acceptable as a scientific theory. It without effort, information fits well with – and indeed processing or inference. Active essentially is -- the familiar theories, taking a very different "stimulus/response" notion in view, suppose that are which behaviour is described as constructed, by complex brain controlled directly by prevailing processes, from fleeting conditions. This is also familiar in fragmentary scraps of data engineering : in most devices input signalled by the and drawn directly controls output ; and much from the brain's memory banks - emphasis is put on measuring input themselves constructions from and output, and relating them by snippets from the past. On this transfer functions or something view, normal everyday perceptions equivalent, to describe the system. are not selections of reality but are B. F. Skinner in his behaviourism rather imaginative constructions - claims to do much the same -- to fictions-based (as indeed is science give at least a statistical account of fiction also) more on the stored the relationship between stimulus past than on the present. On this (input) and behaviour (output) in view all perceptions are essentially animals and men. An engineer fictions: fictions based on past would go on to suggest “models”, experience selected by present of what the internal mechanisms sensory data. Here we should not might be which transform inputs equate "fiction" with "false". Even into the outputs. But, rather the most fanciful fiction as written curiously. Skinner does not is very largely true. or we would attempt to make this further step, not understand it. Fictional and apparently distrusts it. He says characters in novels generally have remarkably little about brains, and the right number of heads, noses at times denies memory and indeed and even many of the opinions of all internal processes. His people we know. Science fiction description is purely in terms of characters may have green hair and input output relations, with an exoskeleton-but is this novelty emphasis on how the probability of not a mere reshuffling of the pack certain kinds of behaviour is of our experiences? It is doubtful changed by environmental- if a new "card”, suddenly changes, especially "reinforcers". introduced could be meaningfully described or seen. Skinner himself has little interest specifically in perception, The passive paradigm may, at but passive theories of perception least initially, seem more are in many ways similar. They Times Literary Supplement JUNE 23, 1972 3 have the same initial scientific quite long gaps in sensory data, credibility, but are (I believe) and remain appropriate though essentially incorrect. They deny there is no sensory input. But how that perception is an active can "output" be controlled by combining of features stored from “input” when there is no input? the past, building and selecting The fact is that sensory inputs are hypotheses of what is indicated by not continuously required or sensory data. On the active available, and so we cannot be account we regard perceptions as dealing with a pure input-output essentially fictional. Though system. Further, when we consider generally predictive, and so any common action, such as essentially correct, cognitive placing a book on a table (a fictions may be wrong to drive us favourite example of philosophers) into error. On this active view, we cannot test from retinal images both veridical (correct-predictive) the table's solidity and general and illusory (false-predictive) book-supporting capabilities. In perceptions are equally fictions. To engineering terminology, we perceive is to read the present in cannot monitor directly the most terms of the past to predict and important characteristics of objects control the future. This account is which must be known for very different from the passive behaviour to be appropriate. This story implied by Skinner's implies that these characteristics behaviourism, and most ably are inferred, from the past. The propounded by James J. Gibson other highly suggestive-indeed and Eleanor Gibson (whose article dominating -- fact is that is on page 711). perception is predictive. In skills, there may be zero delay between Why should one want to push sensory input and behaviour. But all this stuff about " brain fictions" how could there be zero delay, (as I do) when stimuli and except by acting upon a predictive responses are so easily observed, hypothesis ? (Surely J. J. Gibson's and so like the usual stuff of description of perceptions as science? The essential reason is (I selections from the available believe) very easily demonstrated, "ambient array" will not do : it by common observation and by would have to be a selection from experiment. Current sensory data a future "ambient array" for the (or stimuli) are simply not passive account to work : but this adequate directly to control evokes a metaphysics we cannot behaviour in familiar situations. welcome. The significance of Behaviour may continue through prediction in perception has been Times Literary Supplement JUNE 23, 1972 4 for too long almost totally operations serially) that the ignored.) computer requires a vast amount of stored data of common object It is the fact that behaviour does properties with ready and rapid not need continuous, directly access. It requires, in short, what appropriate sensorv data that we have called “fictions” to forces upon us the notion of augment and make use of data inference from available sensory monitored from the world by its and brain-stored data. This camera eye, and – in machines account is very much in the dealing with real objects – its tradition of the polymath touch probes. In short: we may nineteenth – century physicist and think of perception as an physiologist, Hermann von engineering problem, but it is a Helmholtz, who described highly atypical problem even for perceptions as “unconscious advanced computer engineering,m inferences”. This notion was and it requires a special philosophy unpalatable to later generations of which is unfamiliar in science, psychologists, who were over – because only brains and to a influenced by philosophers in their limited extent computers are role – sometimes useful, but in this cognitive. case disastrous – of guardians of semantic inertia: objecting to The notion that interpreting inference without consciousness. objects from patterns is a “passive” But with further data on animal business must strike the computer perception, and computers capable programmer engaged on this of inference, this essentially problem, in Machine Intelligence, semantic inhibition has gone. as an extremely unfunny joke. His Curiously, though, the kinds of problem is to devise active inference required for perception programs adequate even for are remarkably difficult to perceptual problems solved by compute. simple creatures long before man came on the scene. The recent engineering – science of Machine Intelligence is The notion of perceptions as finding heavy weather designing predictive hypotheses going computer programs to identify beyond available data is alien and objects from television camera suspect to many physiologists. pictures. The reason seems to be Cognitive concepts appear (apart from the very large and fast unnecessary, even metaphysical – computers required to perform the to be explained away by Times Literary Supplement JUNE 23, 1972 5 physiological data. Certainly more Cognitive concepts are surely not physiological data are needed: but alien to science, when seen as the will they tell us by what brain's (relatively crude) strategies mechanisms the brain’s hypotheses for discovering the world from are mediated, or will the “brain limited data-which is very much fiction” notion drop out as the basic problem of all science. unnecessary? Prediction is Scientific observations without dangerous, but there are surely hypotheses are surely as powerless strong reasons for believing as an eye without a brain's ability cognitive concepts should be to relate data to possible realities- required for brain research, effectively blind. because the brain is unique, in nature, as an information handling The full power of human brain system. (Or at least it is on an fiction is apparent when we active theory of brain function.) consider how little current sensory With the development of information is needed, or is computers, we now have other available, in typical situations. information handling systems to Here we do not need initially to consider: it is interesting to note consider particular experiments -- that to describe computers, and indeed the intentional “software” concepts are adopted, simplifications and restrictions of similar to cognitive concepts. the laboratory environment can More basically, what are make the point less obvious - that essentially cognitive concepts are behaviour is generally appropriate very familiar in all the sciences, to features of the world which are but hidden under a different guise not continually available to the – the method of science. senses. When you trust your weight to the floor, or your mouth Generalizations and hypotheses to the spoonful of food, you have are vital to organized science, for not monitored the ground's the same reasons they are essential strength or the food's palatability : for brains handling data in terms of you have acted on trust, on the external objects. Science is itself basis of the past. You have acted not “passive” in our sense, but puts according to probabilities, based up hypotheses for testing, and acts on generalizations from past on hypotheses rather than directly events -and neither generalizations on available data. Scientific nor probabilities exist, except in observations have little or no your brain, for they are not power without related properties of the world. Now generalizations and hypotheses. suppose that you gave up acting on Times Literary Supplement JUNE 23, 1972 6 informed guesses and demanded, thus have great importance for continuously, direct selections of active theorists : they become reality. How would you get on? obsessively used tools for Would you not avoid mistakes discovering the underlying never fall through rotten assumptions and strategies of the floorboards, never be upset by bad perceptual “computer” by which food -never be misled by going we infer – not always correctly – beyond the evidence ? Yes, indeed, external objects from sensory data. if there were sufficient evidence available. But the fact is that there Looking at books written by is frequently no possibility, or passive and active theorists, we time, for testing floorboards or find an amusing difference food. They must be taken on trust- between their indexes. Passive trust based on the past as stored in books devote much space to the brain. stimulus patterns, but very little to the phenomena of perception: We have arrived at questions spontaneous reversals in depth, which may be answered by changes into other objects, experiment. We can measure distortions, perceptual paradoxes performance, in the partial or total in which the mind reels by being absence of sensory data, and apparently confronted by logically establish whether and how far impossible objects. Active perception and behaviour continue theorists fill their books with to remain appropriate. We find that examples of such phenomena, we can continue to drive or walk, interpreting them in various ways, or perform laboratory eye-hand while the passive theorist ignores tracking experiments, through gaps them, or writes them off as too in sensory data: and not merely trivial to concern him. But neither inertially, for we can make uncertainty nor ambiguity, neither decisions and change our actions distortion nor paradox, can be appropriately during datagaps. We properties of objects: so how can must then be relying on internal we perceive uncertainties, data. This requires an internal ambiguities distortions or fiction of the world-which in paradoxes if perception is but a unusual situations may be false. If passive acceptance of reality? the situation is unfamiliar, or This simple though surely changes in unpredictable ways, powerful argument is not raised or then we should expect systematic answered by passive theorists. By errors generated by false playing down the obvious predictions. Errors and illusions phenomena of perception (such as Times Literary Supplement JUNE 23, 1972 7 illusions, found as children’s There is a strong reason (apart puzzles) passive books may look from consciousness) why we wish academically safe – but at the cost to separate descriptions of aspects of leaving out what is most of brain function from physics. interesting. This is however a very tricky problem, easy to over – state and We may now return to the point to misunderstand. Granted that that, although we regard brain brain activity is physical, we wish function as physical, physical and to hold that brain states engineering concepts are not representing information and adequate for describing some problem – solving are not usefully aspects – especially perception of described in terms of physical objects. This only appears to be a restraints. Consider the black metaphysical statement if an marks (letters) on this page. They extreme reductionist view of are physical (ink absorbed by science is adopted. This matter is paper), but their arrangement, controversial: there are eminent surely, is not to be understood by scientists who hold that knowledge the principles of physics. For this of a hydrogen atom and the laws of we must call upon English spelling quantum mechanics are sufficient and grammar and upon the to describe, in principle, any structure of what I am trying to physical situation. Others hold say. In the vital respect of their that even common effects such as order, they are free of the ink and friction, heat, inertia or gravity (let paper of which they are made. If alone brain function) could not in their order were determined principle be described in these directly by their material and its elementary terms. They hold that physical properties (as in crystal with increased complexity, and structure) then they could not serve organization new properties arise as symbols. Being in this sense requiring new concepts to describe free of physical restraints, and them. It would certainly be given receptive brains (or difficult to ascribe the notion of computers) then they can serve as “cognitive fictions” to a hydrogen symbols: to represent objects in atom! (But it wold be equally other time and space; or difficult to ascribe such concepts abstractions which do not exist, in as servo-control, or even image – the sense that objects exist. This is forming – so this is not a special true for all symbols: pictures, objection to the “cognitive fiction” words, mathematical and musical notion.) notations, video and audio tapes, computer tapes. But symbols are Times Literary Supplement JUNE 23, 1972 8 powerless (or are just like any relevant to the logical problems the other objects) in the absence of brain must solve to infer objects brains or other information – from sensory patterns and stored handling systems. Evidently data. This is quite different from a symbols must affect brains in some cognitive account of perceptual more or less lawful manner: but distortions, and other phenomena for this to be possible the relevant which may be supposed to arise brain states must – like the typist’s from misapplication of strategies or compositor’s characters – be quite apart from the physiology free to adopt information storing involved. Using a slide-rule, an and representing orders. So they error may be due to physical errors must in this rather limited sense be in the rule itself, or to free of physical restraint, though misapplication of the rule for the not quite isolated from the rest of problem in hand. This is exactly the physical world for learning and the distinction involved here, perceiving to be possible. between physiological and cognitive errors. The celebrated (and I believe essentially misleading) Gestalt We should expect physiological theory of perception postulated restraints to produce the same physiological restraints to explain effects for any object situation (for many visual phenomena, such as example after – images, due to preference for, and distortion retinal fatigue, to any bright light). towards, figures of “simple” and Misplaced strategy errors should, “closed” form. Visual forms were on the other hand, be related to the supposed to be represented in the kind of perceptual inference, from brain by similarly shaped electrical sensory pattern to object, being brain fields – circles by circular carried out. So the point is that the brain traces, presumably houses by physiology should only produce house – shaped brain traces. These errors when it is exerting general brain traces were supposed to tend restraints. We should not expect to form simple and closed shapes, this except in abnormal situations, because of their physical such as when the physiological properties; much as bubbles tend “components” are driven beyond to become spheres, as this form their dynamic range. Considering has minimum potential energy. phenomena of perception, such as Now this implies that visual ambiguous, distorting or “organizations” and distortions are paradoxical figures: do these due to physical restraints and figures upset the physiology, or forces which will not in general be select inappropriate strategies, to Times Literary Supplement JUNE 23, 1972 9 generate errors? In these cases, it support a book upon is to solve a seems to be the object significance problem so difficult it challenges of the figures which is relevant. the most advanced computer So these phenomena seem quite technology, and yet to us it is so unlike after-images – here it is not simple that a passive theory of so much the physiology as the perception may seem plausible. cognitive strategies which we need This shows that passive theories to discover. This needs a different may be so misleading as to hide (but still a “scientific”) way of aspects of brain function we must thinking, and powerful see clearly to understand not only experimental techniques, to perception but all mental processes discover cognitive strategies and and how they can go wrong. how they can mislead. Recent discoveries by To separate errors due to physiologists, especially by physiological restraints from errors electrical recording from single due to misplaced strategies surely brain cells during controlled has importance beyond stimuli to the eyes, are so clearly understanding perceptual errors. important that they tend to The same distinction (between dominate much current thinking physiological and cognitive about perception. The problem of processes, and how either can go how sensory patterns are wrong) might be important for interpreted in terms of objects understanding mental illness. If tends to be ignored. The important schizophrenia is errors in the physiological discovery is that brain’s strategies for developing certain stimulus patterns (lines of hypotheses of external states of certain orientation, or movement, affairs, this should be understood etc) produce repeatable activity in not only in terms of biochemistry specific brain cells. This discovery and physiology but also in terms of came as an unpalatable shock to the strategies by which we passive theorists who tend to normally cope with things. ignore brain function. To active Perhaps this matter of strategies is theorists, it gives a clue to the hidden by the apparent ease with kinds of data accepted for building which we continually solve object-hypotheses. One might problems of the utmost difficulty think from this that passive to computer programmers: and theories would drop out, leaving which receive false answers when the field of physiologists and their programs are inappropriate. active cognitive psychologists to Seeing a table as something to work together in blissful harmony. Times Literary Supplement JUNE 23, 1972 10

Actually things are not quite like development of perception in this : the physiological advance is babies. Early changes of the so concrete, and clearly important, nervous svstem as a result of that many physiologists and experience are now being cognitive psychologists feel that discovered, which will perhaps finding more feature analysers, and help to tie up, or relate, physiology more abstract analysers, is the sole and cognition. Possibly the most path we need to follow to fundamental and rigorous ideas are understand vision. But is it? The coming not from biology but from physiological mechanisms being attempts to program computers to discovered relate to stimulus see and handle object-relations. It patterns only, and not recognition proves necessary to make the of objects its hypotheses. The computers develop hypotheses and physiological account thus remains select the most likely, given the passive, and so essentially data from its glass eye. inadequate, for the same reasons that cognitive passive accounts are There is more to this, for some inadequate. computer programs designed to give "scene analysis" (recognizing The task ahead is to relate objects from pictures by computer) physiological processes not only to assign alternative object direct input-output links, as in probabilities to selected features in reflexes, but also to the brain's the picture: and then change these logical and correlating activity probabilities, according to endowing it with the power to probabilities assigned to other predict. This will require further features of the scene. For example, physiological data, and current a given shape may be a box or a techniques are providing extremely building. If what is taken to be a important new information so this hand is above it, then the will surely be available. probability of the box hypothesis Experiments on the phenomena of will be increased and the building perception itself, in animals and in hypothesis decreased-for hands are men – essentially on how patterns generally too small and too low to are interpreted as objects -- has be above buildings, but not above confusions (or at least impeding boxes. Now this gives interactions, disagreements) in its philosophy, due to conditional probabilities, and a lack of powerful research which may generate visual effects techniques. Some of the most in computers or brains quite like interesting clues are at present the old Gestalt phenomena, but for coming from studies of an entirely different reason. The Times Literary Supplement JUNE 23, 1972 11 reason is to be understood in terms are supposed to accept the of cognitive strategies or behaviourist's writings as procedures for making effective expressing his observations, use of data for deciding what thoughts and judgments: which in objects are present in the scene. these same writings he denies In Machine Intelligence only having. We are reminded of the precisely formulated theories are poignant postcard received by adequate: any gaps or errors in the Bertrand Russell saying : " I am a theory show up as errors in the solipsist-why are there no other machine. At present machines philosophers like me ? " perform only the simplest tasks, ______and are easily confused by Professor Gregory is head of the shadows or small changes we Brain Perception Laboratory, scarcely notice. University of .

Although the difficulties in Machine Intelligence demonstrate all too well how little we know, it now seems that we are beginning to understand ourselves – the inference – mechanisms of our humanity – by inventing adequate concepts for machines to infer objects from data: to perceive our world with their metal brains and human-devised programs. Is this science fiction ? Yes-but like all fiction it may be largely true. Philosophically, this is not the end of the matter. Behaviourism, with its related passive theories of perception, is unconcerned with what goes on between the senses and behaviour: indeed denies that anything goes on. This may be a legitimate expedient for focusing attention upon certain questions in behavioural research; but as a philosophy it is a kind of nihilism with a built-in contradiction. We