Perception, Planning and

Action

Sensory perception

• Definitions.

o Sensory Perception: process of obtaining information or knowledge (about the environment and the body) from sensory stimulation and making it available for doing things

§ About gaining knowledge

§ Is a process

§ Provides information for doing things

• Extracting information, using mediums (e.g. light)

o Doing things involves many processes

§ Communication, planning, decisions, memorising

§ The most important is the initiation and guidance of goal- directed motor behaviour (movement/action)

• Organisms perceive in order to move

o Has a purpose

§ A limited amount of behaviour can be carried out using non- perceptual sensory processes

• This is reflex behaviour

o Only uses senses, no perception

o No information on the causes/source of the stimulation

§ Don’t know what caused it

§ Perception tells you about the causes of stimulation

o Sensory perception is distinguished from other kinds of perception by: § The fact that it can provide info about the environment/body based on stimulation of the sensory organs

§ The primary source of info is immediately preceding stimulation of the sensory organs

§ It can operate without conscious experience (e.g. subliminal perception)

§ Other kinds of perception are active, conscious, CNS processes that aren’t based on immediate sensory stimulation

• Meaning of stimulus

o Reflex behaviours: involuntary elicited responses to proximal stimulation

o Proximal Stimulus (stimulation): Physical energy or force (or properties thereof) that impinges on sensory receptors and evokes a change in their membrane potential.

§ Only require sensing, not perception

§ Some receptors only detect, go via passive induction

§ Others conduct APs

o Perception involves obtaining info about the environment/body

§ Info about the distal stimuli (things/events in the world)

§ Distal Stimulus: An object, substance or event in the environment that is a source or cause of proximal stimulation and/or its characteristics

• Perception is the process of obtaining info about distal stimuli from proximal stimuli

o In visual perception proximal stimulation = the patterns of light intensity that are projected onto the retina(s) by the optics of the eye(s): the retinal image(s)

• From stimulus to perception

o Objects, surfaces, visible things all reflect light from various sources (distal stimuli) --determines→ pattern of light entering the eye that form retinal images (proximal stimuli) --stimulates→ photoreceptors transduce light into neural activity -- transformation/processing→ percepts: knowledge/info about distal stimuli

• Vision contributes to behaviour

o Visual info is used for: § Detection, identification, decision making, preparation of movement, initiation

• Perceiving and moving

o Organisms perceive so they can move

§ This is the primary reason why perception exists

o To guide behaviour, you need relevant info at the right time, that isn’t contaminated by irrelevant info

o Making info available means providing relevant info in a usable form

§ To acquire it you need the right proximal stimulation

• Acquisition and processing need to be selective

• Perceptual activities

o Perception is an active process

§ Obtain/select relevant, uncontaminated info

§ Not passive reception of stimulation

§ 4 kinds of activities are involved:

• 1) Information seeking activities: seek out (or generate) stimulation that contains the information required

• 2) Information selection activities: select information needed for particular behaviour(s) from the total amount available

o Proximal stimulation can contain lots of info, selected what is needed for the behaviour you engage in

• 3) Enabling activities: establish a stable base that enables the sensory organs to operate effectively

o Lets you extract info effectively

• 4) Information extraction activities: process stimulation to extract useful information from it

o These all involve behavioural components, except for 4

§ Obtaining stimulation that contains useful info involves body movement

• Move our sensory organs around o Neural processes involved in getting the info we need out of the stimulation/our sensory organs

§ Useful info extracted from the signals from sensory organs

• Feeling: active touch

o We use relevant sensory organs in specific ways, to obtain particular types of info

o Active process, involves receptors in upper and deeper tissues

§ Touch used in everyday life is active/kinesthetic touch, or haptic perception

§ Activities that people use to obtain info have characteristic forms

§ There are 6 exploratory procedures

• Hardness, weight, texture, shape, size, temperature

• Smelling

o Breathing in, but also motor activity

§ When dogs follow a scent trail, they move side to side

• Humans can do the same, and weave as well

o They improved with practice, and two nostrils are needed

• Perceptual activities

o Movement plays a role in our sense

o We move our eyes/head/body when using retinas to obtain visual stimulation

o Movement is needed to:

§ Obtain the stimulation needed to obtain required info

§ Keep the retinal images steady

• Function of eye movements

o Both eye movement and components of superordinate systems generate head and body movement

§ Some animals don’t make eye movements/have less capacity for making them, including some invertebrates amphibians, birds • They rely on head/body movement instead and use these to stabilise their eyes

The Perceptual Process

• Perceptual activities o Think of sensory organs as measuring instruments § Must be moved in specific ways, into the right position, and held in place • Without those, measurement would be impossible § Sensory organs are instruments for measuring variables of proximal stimulation • Some with routines/procedures (e.g. exploratory) • Sensory stimulation o Motor behaviour is part of perception - to obtain info o Other parts take place in the NS (the brain) § Getting info out of signals from stimulation o Perceptual activities are involved in getting stimulation § Processing is needed to get the info out • Visual perception o Takes the form of images projected onto the retinas (retinal images) § On the back of the eye § Light reflected off objects→ back of the eye→ optics forms image on the retina o Visual processing can be seen as solving a problem, roughly as follows: § From a 2D pattern of light projected onto the retinas, extract information about the location and motion (in 3D), properties and identity of objects and surfaces that is accurate enough to support normal behaviour in the world • Just given a 2D pattern of light intensity • This is difficult to solve o The nature of the stimulation is: § No depth in images § Image size depends on viewing distance • Objects of different size can project similar sized images § Shape of the image depends on the direction it’s viewed from, so it varies and is not the same as the real shape • Visual ambiguity o The same image could have been produced from infinite possible objects of different sizes/shapes/distances § Also, a single object can project many different images o Somehow our NS can resolve this ambiguity § Determine what the objects are where regardless of viewer’s location • How: not all logical possibilities are physically/ecologically likely o The world has rules and constraints • Resolving ambiguity o Know what’s impossible/unlikely → exclude possibilities § This is the basic principle of perceptual processes • The a priori knowledge comes from genetic inheritance and from experience o We start with poor perception and it improves with practice o If the only possibility that’s compatible with prior knowledge is wrong, then the percept is wrong § If something is in one place, then another, we perceive movement • E.g. films (many still images in a row with blanks in between) • Brain creates movement • Perceptual illusions o When the percept is wrong like this, a perceptual illusion occurs § Doesn’t represent the truth o Defining perceptual illusion (attempt): a consistent and persistent discrepancy between the perceived and real physical properties of a stimulus. Illusions are consistent in the sense that they invariably occur when the particular stimulus is presented and persistent in the sense that they are strongly resistant to efforts to suppress them. § But not every discrepancy is an illusion • Sometimes we just can't perceive something (for example if it only shows in UV light) • Only distal stimuli properties that our senses can detect should be defined as illusions o Perceptual illusion: a consistent and persistent discrepancy between a sensory percept and the distal stimulus that evoked the percept (and that the percept represents) such that the observer is deceived as to the nature of the distal stimulus. The discrepancy relates only to those (distal) stimulus properties which can be detected by the sensory systems and it occurs as a result of the normal processing of proximal stimulation. § Visual illusion: due to visual processing in the nervous system (internal) § Optical illusion: due to the physical nature of light travelling between transparent media (external) o In an illusion caused by small movements of the eye, it counts as an illusion because the NS has interpreted the retinal image motion from the eye movements as due to motion in the world (the distal stimulus) rather than motion of the image on the retina (proximal stimulus) • Resolving ambiguity o Perceptual illusions are due to the way perceptual processes work § Reflect the way prior knowledge is used to extract info about the distal stimulus from proximal stimulation • Illusions can tell us something about the prior knowledge being used, can help us understand how perceptual processes work o If prior knowledge is compatible with one distal interpretation of proximal stimulus, but is discrepant with reality = illusion o Not sufficient info to reduce possible interpretations to one = ambiguous figure (e.g. necker cube) § Whether these figures are visual illusions depends on what aspect of the experience is being considered • E.g. with the necker cube, the shift from seeing one surface as the face to the other would could as an illusion o Since the figure itself does not change, to perceive this change must be illusory

Visual sensing

• The human eye o Pupil (adjusts size with light), then cornea, behind pupil is lens o Lens is adjusted to get fatter/thinner o Cornea is most important focusing lens in the eye

o o Retina is the focus § § Thick fibrous layer, then retina, on top of retina are blood vessels § Blood vessels on top of the retina supply blood § Then there are layers of neural tissue § Light→ blood vessels→ tissues→ cones • We can sometimes see the blood vessels § The photoreceptor layer • Longer finger like cells • Rods o Longer, thinner • Cones o Shorter, fatter • Photoreceptor sensitivity o In rods and cones, within the outer layers are cells that absorb light § The effect of this is a change in a receptor’s membrane potential • Hyperpolarization § Rods are very sensitive • Single photon of light is enough for a signal • This allows us to see in low light levels o Night/scotopic vision § Cones are less sensitive • Only work in high light levels o Day/photopic vision o Photoreceptors are neurons • Photoreceptor distribution o Rods and cones are not evenly distributed across the retina § One part has none at all (blind spot) § One part has no rods (foveola) o Even when there are both, relative number depends on location o o Both the blind spot and macula can be seen through an ophthalmoscope o In a cross section of the retina, small dots are rods, and larger are cones o Closer to/within the macula, there are more cones § Center of the macula has all cones, no rods § Most cones are within the macula • The macula

o § The fuzzy dark patch is the clinical macula • Defined by clinical observation § Foveola → fovea → parafovea → perifovea § Fovea=darkest area • A dip in cell layers here o Bottom of this dip is the foveola o Less cell bodies between light and the receptors o Has no ‘blue’ cones o Not overlaid by blood vessels o Roughly disk shaped when viewed through the pupil § Parafovea=region surrounding the fovea • Contains rods and cones o Cones are unevenly distributed § The 3 types of cones are distinguished by sensitivity to different wavelengths § Often referred to as red, blue, and green • This is misleading § Perifovea • The very outer ring o Contains equal numbers of rods and cones o Overlaid by blood vessels § Foveola=central part of the fovea • More cones than rods at the boundary of the fovea o The anatomical macula is slightly bigger § Distinguished by cell types • Not distinguishable by an ophthalmoscope • Cone types o Sensitivity to different wavelengths depends on the extent to which they are absorbed by the photopigments in the cone outer segments § The 3 cone types have different photopigments that absorb wavelengths differently • Different wavelengths have different colours o § The cones have different levels of sensitivity • Green > red > blue o This names are misleading, as the cones are not coloured, and not fully accurate to the colours they’re sensitive to § Preferred modern names: • Long wavelength sensitive (L) cones (red) - actually most sensitivity in yellow/orange region, about 600 • Medium wavelength sensitive (M) cones (green) • Short wavelength sensitive (S) cones (blue) • Blue: 400, green: 550, red: 600 o In the retina there are roughly 55% L, 35% M, and 10% S § There is an area of the eye with no S cones (fovea) • Retinal structure o Photoreceptors don’t transmit signals directly to the brain § They make synaptic connections with cells within the retina § Ganglion cells (axons from the optic nerves) transmit signals out of the retina • This ganglion cell body layer is thicker and used to define the anatomical macula o Where the layer is 2 or more cell bodies thick = anatomical macula • Ganglion cell axons form the optic nerves and only via these can info reaches the brain o

o § This cross section is most likely to come from the peripheral retina

Colour: function and phenomenology

• Colour o Things you see have colours, shapes, sizes, hardness, texture § Colour is different as the others can be easily measured • Can you measure colour? o Seems to be related to wavelengths of light o Different wavelengths seen as different colours § Only proximal stimulation? § Not a property of light itself o There are no colours in wavelengths, colour is in your mind, not in the world § It is a mental construct - characteristics of mental states, not properties of the world o Colours are related to wavelengths of light § The relationship is more complex than imagined • Colour effects o People believe that red objects look red because their images contain mostly red light § This isn’t always true • Colour vision o Image and wavelength=proximal stimulation § What is the distal stimulus? • Functions of colour vision o Make the world pretty, but this isn’t a good evolutionary reason § Most animals have colour vision, so it must be useful for living and surviving o You can spot fruit more quickly with colour perception § Advantage in fruit gathering, fruit wants to be eaten o Colour pop out § Items that are a different colour to other things in a scene can be located very quickly • Is said to pop out from amongst background elements o Don’t need to search for it o Recognition/identification § You can recognise the ripe and unripe fruit • Can help you identify particular items that are similar to others o When only colour differentiates them § Can help you identify different items and/or parts of items that are different from others • E.g. rotten fruits/parts of fruits o Colour can enable rapid figure-ground segregation and location of surface regions that are different to other regions § E.g. the colorblind number tests • Colour helps you segregate what is colour and what is background • Colour perception o Colour is useful for: identification of objects, figure ground segregation, differentiating surface regions with different properties and rapidly locating important items § To be useful, colour must tell us something about the objects/surfaces • Something about the distal stimuli • Must be related to some property of surfaces/substances in the world o 1) If colour is associated with proximal stimulation (wavelengths), how can it be telling us anything about surfaces in the world (distal stimuli)? o 2) If colour is a purely mental construct, how can it convey useful information about the world? • Colour categories o We can discriminate between millions of different colours o But there are only a few distinct colour categories § Considered to be: • Red, orange, yellow, green, blue, purple o Roughly matches the rainbow § Seem to correspond to particular ranges of wavelength • Missing shades of grey • Missing browns and pinks o Missed out, or something else? o Actual number of categories is undefined

Colour: dimensions, stimulation, and trichromacy

• Colour categories and experience o Different cultures have different numbers of categories § Reasons aren’t understood o Ambiguity from achromatic colours (greys, white, black) o Browns and pinks are often said not to be a distinct category § Must understand the dimensions of colour experience to know why • Colour is multidimensional, and is considered have have 3 dimensions • Dimensions of colour o Hue (shades/the actual colour) o Saturation (paleness) o Brightness (itensity) § The “colour solid” is defined by these dimensions

• o Pink is said to be desaturated red - not it’s own colour o Browns are dim oranges/yellows/orangey-reds

§ Shine light on brown, it will likely appear orange • Not all shades and hues correspond to particular wavelengths o There are non-spectral colours o No wavelength appears brown or pink or purple § No single wavelength, no monochromatic light shows this colour • Not in the spectrum o Dimness is about the amount of light present § If you take a 450 nm light, it will appear blue in hue and its brightness depends upon intensity of the light • Decrease intensity and colour will appear dimmer • You can’t do anything with a single wavelength and change saturation o Requires addition of other wavelengths § Same for many shades of hue o Can’t do anything to a wavelength of red to make it appear pink § You need to add other wavelengths to do so § Add white - has every wavelength in it to a roughly equal extent • To change saturation, you need to add equal proportions of every wavelength • Proximal stimulation o Why does light lead to perception of colour? § Often thought that different wavelengths give rise to different colour perceptions § The visible range is 400-700 nm:

• o Newton proposed that different wavelengths were responsible for different hues § Demonstrated how white light can be broken into its component colour using a prism • Led him to propose that rays of different wavelengths excited different elements in the retina o In modern terms: different hues are the result of stimulation by different wavelengths o He also thought that each type of ray stimulated a different type of excitable element in the retina • Trichromacy o Young said the eye does not possess a different type of sensor for every type of ray § He suggested 3 different types of receptor are enough • Based on the observation that artists could create many colours from just 3 primary colours o Today’s colour printing is based on mixing magenta, cyan, and yellow § More mixed = darker • Absorb more light as they all absorb differnt wavelengths differently, so darker o Subtractive mixture § Mixing all these together makes a dark brownish grey, not black, so black is included • The CMYK system o It is possible to produce any coloured light by mixing three primary coloured lights § All 3 added together=white § Blue, green, red § Additive mixture § Any colour can be matched by a suitable combo of 3 lights • 2 is not enough, 4 is more than enough • Why is one type of receptor insufficient o We have 3 types of photosensors

§ • This shows the proportion of light shining on the cone outer segments that is absorbed by the pigment molecules within o The cone response depends on the amount absorbed § More absorbed=more change in membrane potential=more response o Imagine if we had only one type; an L- type § If you shone a 670 nm light on its outer segment, you could get the same response with a 550 nm light of the same intensity

§ • The same amount of light is absorbed, so they give the same response o So you cannot tell between the 2 § Imagine you have 3 different wavelengths that are absorbed to different extents (560, 440, and 670 nm) • o The cone absorbs 6x more 470 than 440, 5.7x more 560 than 670, and 34x 560 more than 440 § The amount of light absorbed depends on the amount of light shone on it • Light absorbed by the cone is a proportion of that shone on it § Suppose you shine some 670 nm light on it • To get the same response using 440 nm light, it needs to be intense o The cone absorbs 6x more 670 nm light than 400 nm light § To get the same response, the 440 nm light will need to be 6x more intense that the 670 nm. • The cone absorbs 5.7x less light at 670 nm than 560 nm, so you’d need a source 5.7x less intense for 670 nm light to get the same response as 560 nm light § Suppose you get a response by shining a mixture of 560 nm and 440 nm light onto the outer segment of an L cone, how can you get the same intensity using 670 nm light? • Assume the intensity of 560 nm and 440 nm lights are the same - X o Cone absorbs 17% of X at 560 nm, and 0.5 % at 400 nm, so total response is proportional of 17.5% of X § Cone absorbs 3% of 670 nm light, and you want this to equal 17.5% • 17.5% = 3 x 5.83% o So intensity is 5.83 x greater o You can get any response from a cone that you want using any single wavelength that it’s sensitive to, or a mixture of wavelengths, by adjusting intensities § A single cone cannot distinguish between different wavelengths from lights of different intensities • So carries no info about the wavelengths of the stimulus o This is called the Principle of Univariance § Means colour vision is impossible with only 1 type of photoreceptor • Colour vision o Depends on getting info about the wavelengths of light present in the stimulation (image) § At least two types of photoreceptor are needed • Roughly, the more types of photoreceptor, the more wavelength info can be obtained • Most mammals have 2 cone types- dichromats • A few mammals are trichromatic o Some primates and marsupials • More than 3? o Some animals are tetra- or pentachromatic § Many fish and birds are tetra § A few birds and butterflies are pentachromatic § Some have even more- the mantis shrimp has 12 types o Just 1 photoreceptor doesn’t allow an organism to discriminate between wavelengths, but just 2 allows discrimination between at least 10,000 different stimulations, and 3 allows millions • Dichromatic wavelength discrimination o Imagine you have an M- and L-cone

o § If they are stimulated by a 660 nm light, the L-cone absorbs 25% of it’s max, and the M-cone absorbs 3% • The L-cone will be 8.3 x the M response § If intensity increases, both responses increase, the ratio between the 2 is always the same- 8.3 x

• < increase in intensity vs. decrease in intensity >

• The response of one cone relative to another is the same regardless of light intensity o This is the key to understanding why 2 types of receptors can provide info § Now simulate them with 520 nm

§ • L-cone absorbs 29% of max, M-cone 78% o L response in now 0.37 x the M § 660nm red light: L/M=8.3 520 nm green light: L/M=0.37 • You can change the intensity of these lights however you want but you will never change the patterns o So these 2 cones can distinguish between these 2 different wavelengths § The ratio of 2 responses provides a way of describing this pattern o Not all wavelengths can be distinguished § If 2 different wavelengths produce the same ratio, they cannot be distinguished • A 520 nm and 460 nm light both produce a 0.37 ratio:

o o They therefore produce the same pattern of response and can’t be distinguished § These are an example of metameric stimuli: stimuli that appear the same but are physically different • There are a lot of metamers for a 2 cone system o Mixing 520 and 670 nm lights produce the same response in the 2 receptors as any other light/mix of lights • Trichromacy o Adding another type of receptor allows greater number of wavelength discriminations § Many wavelengths that look the same to a dichromatic system look different to a trichromatic system • Looking at the 520 nm green light and 470 nm blue light again

• o With 3 receptors, the pattern of response to the 520 nm has a 29:78:0 pattern o The 470 nm light has a response pattern of 5.1:13.8:68 § These patterns are the same regardless of intensity • Clearly different o The trichromatic system can tell between wavelengths that look the same to the dichromatic system § However, metamers still exist in the system • Any light can be matched by a mixture of 3 primaries (red, green, blue) o Can match any mixture of any wavelengths § What’s been looked at at the 3 cone system amounts to the trichromatic theory of colour vision • Helmholtz knew the 3 primary colour lights could be mixed to produce any colour o Agreed there were 3 types of photoreceptors involved o He proposed that their wavelength sensitivities were broad and overlapping § o He also proposed that perceived colour is determined by the pattern of responses in the 3 receptor types § The pattern can be described in terms of the response activity in the receptors relative to one another • Total response activity in all 3 determines brightness o Trichromacy theory proposes that perceived hue is determined by the proportional activation of the three cone types in response to the wavelength composition (spectral content) of the image § Something looks blue because there is light from the ‘blue’ end of the spectrum in its retinal image and so the S (blue) cones are stimulated more than the L (red) and M (green) cones • Trichromacy theory is wrong however o Correct about the cones and sensitivity § But the rest is wrong

Trichromacy theory and its problems

• Helmholtz trichromatic theory o Perception = relative pattern of activation in the 3 cone types § These patterns form colour perception and interpretation • Very different patterns for different colours o All about NS interpretation o o There are 3 basic elements of the theory § 1. Colour vision is based on the info provided by three types of cones that are each sensitive to a range of different wavelengths, but in slightly different ways: one is most sensitive towards the long wavelength end of the visible spectrum (L-cone), one in the middle (M-cone) and one at the short end (S-cone) § 2. The pattern of responses in the three types of receptor determine the perceived hue § 3. In order for (2) to work, there must be similar numbers of the three cone types distributed evenly throughout the retina • 1) is shown to be correct o Colour vision is based on info provided by the 3 cone types that are most sensitive in the long, medium, and short wavelengths respectively • 2) and 3) are incorrect o We see things looking a colour without there being any of that colour/wavelength in their images o We know that there are not equal numbers of different cone types, and they are not evenly distributed across the retina § According to the theory, it should be impossible to see blue things (or colours that blue contributes to such as purple or white) in the central part of vision • Hering’s alternative theory o Hering didn’t know about photoreceptor distributions or colour phenomena o Based his theory on 2 observations § There are no hues that can be described as reddish-green or bluish-yellow • Note: the colour with -ish on the end means there’s less of that, so a bluish-green would be a green with a bit of blue • If you mix blue and yellow light, you get white • If you mix red and green, typically you get yellow o With the right shades, you can get white § If you mix the right shades of blue and yellow/red and green you can eliminate hue • As if red cancels out green, and yellow cancels out blue (and vice versa) • Red is the opposite of green, blue is the opposite of yellow o Off this notion, Hering developed the Opponent Colours Theory § In this, red, green, blue, and yellow form 2 opposed pairs • The blue-yellow pair, and a green-red pair o Suggested these are the 4 basic/primary colours § He proposed there are 2 colour mechanisms/processes in which the pairs are in opposition • Green-red process, and blue-yellow process • Picture as 2 voltmeters: yellow to the left, blue to the right, and same for green vs red

o § Stimulation is the mixture of 2 hues, one excited the channel, the other inhibits it § If there is more than one colour than the other in the stimulus, the perceived hue will represent that § If there is an equal amount, the pointer stays at 0 so there is no hue • M cones inhibit, L cones excitate • S cones inhibit, a combo of M and L (to make yellow) excitate § The bigger the effect (the more the pointer moves), the more saturated the colour § Stimulate a mixture of 2 wavelengths (e.g. blue and red), the NS assigns a colour to this pattern of activity in the 2 channels (e.g. purple) o Different from trichromacy as it proposes that perceived colour is determined by the outputs from the 2 colour opponent channels, not directly from the 3 cones § Better explanation of complementary colour afterimages/effects • Produced when you stare at a coloured image for a short time • Example of adaptation technique: o Selective, often prolonged, stimulation is used to desensitize (fatigue) a process § E.g. stimulation by bright light reduces the light sensitivity of the eye • Trichromacy vs Opponent colours o For years it seemed the theories were irreconcilable § Trichromacy more accepted o Then a reconciliation was proposed in which an initial trichromatic stage (the cones) was followed by an opponent stage § Trichromatic stage → opponent-process stage • Opponent stage works like already described o There are channels in the visual system like this § Green-red channel receives input from the L-cones and M-cones, the blue-yellow channel gets input from the S-cones and a combo of L- and M-cones • Many retinal ganglion cells have these characteristics

§ According to this idea, trichromacy theory was correct in saying there are 3 types of cones • But it was wrong in supposing hue is determined by relative response activity in the 3 cone types § Adding the opponent stage doesn’t solve the problems with trichromacy theory suggested before § The visual system is structured as so at a neurophysiological level: • → Trichromatic stage → opponent-process stage → § But this doesn’t account for how we see normal colours where cones are unevenly distributed/one type is absent • Must be additional processing after opponent stage o → trichromatic stage → opponent-process stage → further processing (in the cerebral cortex) →

Colour constancy and the perception of coloured surfaces

• Colour constancy o The observation that the perceived colour of a surface remains the same when the lighting conditions change § One of several perceptual constancies • Size constancy: an object is perceived as the same size regardless of distance • Shape constancy: an object is perceived as the same shape regardless of angle of view • Colour constancy: a surface is perceived as the same colour regardless of the wavelengths of light present in its retinal image • Colour perception o Constancies show the brain is extracting info about distal stimuli (objects/surfaces) rather than proximal stimulation (lights/objects) § To know the distal stimulus in the case of surface colour perception, we need to know something about how images of surfaces are formed • Light → surface → some reflected back • Surfaces made of different substances reflect wavelengths of visible light to different extents o Reflect more of some wavelengths than others o Consider an object that’s perceived to be red (a raw burger) § Shine a monochromatic light of different wavelengths on it and measure what you get reflected back as a proportion of how much you shone on it • 3 lights; 450nm (blue), 520nm (green), 680nm (red) o 10% of blue reflected, 15% of green, 50% of red o § Then cook the meat so it looks brown, repeat the measurement • 15% blue reflected, 18% green, 35% red

o • Spectral reflectance o Reflectance: the proportion of the incident light (light shone on it) of a particular wavelength that a surface reflects § Proportion of light stone on it that is reflected back § Can measure this at every wavelength in visible range § Reflectance for the meat: • o Left is short wave, middle is middle length waves, right is long length:

§ o This is called the spectral reflectance function of the surface § Amount of light reflected back as a proportion of that shone on it § Different surfaces have different functions • A surface that reflects long wavelengths more tends to look reddish or brown/orangish • Amount of light of a particular wavelength reflected = amount of light the wavelength shone on the surface X reflectance o Including all wavelengths in the visible range: § Spectral content of reflected light (proximal stimulation)= spectral content of the illuminations X surface spectral reflectance • Example: surface illuminated by perfectly white light (same amount of all wavelengths in the visible range) o Same amount of energy at all wavelengths

Spectrum of reflected light is the same shape but lower down as there’s less percentage of light (only what’s reflected).

• Illuminant spectra o No such thing as pure white light

o Spectrums: o Assume surfaces as a flat reflectance function § Reflects every wavelength in the visible spectrum to the same extent • Would be flat o § Imagine illuminating it by different illuminations (such as those above) • It will reflect back whatever percent of light it reflects (Y axis) § Forms the same spectra, just squished down as there is less light (only what’s reflected)

§ o Illuminant spectra vary from place to place and over time § Midday spectra varies a lot in different weather conditions • Relative spectral power varies o Day, weather, bulb used etc o Illuminant spectra depend on the spectrum of the light source AND the effects of other surfaces § Light reflected from one illuminates others, light filters through transparent objects • Due to the substantial variations in illuminant spectra, there will be substantial variations in the light reaching the eye from the same surface in different places/times o If hue was determined by spectral content of proximal stimulation, then the colour of the surface would be expected to change too • Colour constancy o The perceived colour of things does not change as the illumination changes because of colour constancy § Implies that perceived surface colour is determined by constant properties of the surfaces - spectral reflectances • Thus, the distal stimulus in colour vision are surface spectral reflectances o But how do we get info about the distal stim from proximal stim? § The visual system is able to factor out the contribution of the illuminant spectrum from the reflected light that stimulates the retinas § Surface spectral reflectance = spectral content of reflected light ÷ spectral content of illumination • Colour processing o What is the further processing after the opponent processing stage doing? § Colour must be telling us something about a constant characteristic of the surface • About its spectral reflectance • Extracting a description of the spectral reflectance from signals transmitted by the opponent channels • From retina to cortex o Where in the brain is further processing taking place? § The cone-opponent channels project from the retina to the LGN • No cells here show colour constancy

• Axons in the LGN make connections to the V1 o V1 is where almost all retinal signals arrive in the cortex § Extends back o V1 and V2 segregate info about different aspects of the stimulus and distribute these to areas specialized for processing them § The V4 deals with processing wavelength-related signals • The first location in the visual pathway where cells showing colour constancy are found o The processing that extracts info could occur in V4 or distributed over V1, V2, and V4. • Recovering reflectance o The process is working to extract info about the distal stim (reflectances) from the proximal stim (spectral content of the light in the image) § How can you get info about the distal stim from proximal stim? • The visual system factors out the contribution of the illuminant spectrum from the reflected light that stimulates the retinas • Colour constancy o The visual system uses light in different regions of the retinal image to determine the hue of a particular region § Hue of a surface depends on the light reaching the eye from that surface AND from other surfaces • We should be able to construct stimuli in which regions that give rise to images with the same spectral content appear to have different hues, and regions that give rise to images with different spectral content appear to have the same hue o Means not every region of the retina has to have one of each type of cone § Since colour of small surface region doesn’t depend on the response of cones where the image of that region falls • Don’t need each cone in each area o Doesn’t depend on responses in cones on just that area • We are not blue blind in our central vision where there are no s-cones o ‘Blue’ stimulation comes from everywhere, where there are blue cones § Determined by cones where the image falls, and all over the retina o By integrating spectral info from different parts of the image, the brain can discount the contribution of the illuminant and achieve constancy § Get estimate of the spectral surface reflectances • The constancy can be violated, but we’re pretty good at it § The hypothesis applied employs a “grey world assumption” to arrive at an estimation of the illuminant of the image • Supposes that if you average all the reflectances in a typical scene, the result is achromatic/grey (a flat function) o If this is what happens for all pigments in a scene, the average of the light reflected from the scene will have the spectrum of the illuminant § Everything is taken into account

o Allows NS to extract a spectrum of the illuminant, and discount it to get surface spectral reflectance o This means that the average of the light reflected from the scene has the spectrum of the illumination § Brain could compute the average of the reflected light by combining signals from the whole retina to get an average of the entire image • Colour is then computed by dividing the signals from the image of a small patch of retina by this average signal o The perceived colour of a surface would depend on light reaching the eye from it, AND upon the light reaching the eye from everything else in the scene • This is the grey world theory, and uses the grey world assumption (The world, on average, is grey) o If true, colour constancy will be possible only if the assumption is true § If not, then it should fail • Many real world environments have a prominent type of reflectance o Green woodlands, red deserts o Perceived colour should change • Predicts failures of constancy in these environments o Degree of failure depends on how different the average reflectance is from grey § Predicts that constancy will be good when the assumption is true § Successes and failures are not as predicted • It is better than predicted in non- grey average environments, and sometimes worse in those that are • The visual brain uses something more sophisticated than the grey world assumption o Don’t know what

Colour deficiencies

• Prevalence o Many people have some form of colour vision deficiency, most are male § Caucasian population: 8% of males and 0.5% of females have some form o Generally referred to as colour blind § Misleading as it implies that the person does not see in colour - seeing in black and white • Can see all colours • What is colour? o If colour is a purely mental construct, how can it convey useful info about the world? o Surface colour is the way the spectral reflectance of a surface is represented in experience § Blue is the way the reflectance of a surface made of a material that reflects more shortwave light is represented in experience • Blue→ property of mental experiences used to represent a property of the world as derived from visual stimulation • Don’t need stimulation to experience blue o Can dream/hallucinate it • Colours can be experienced with no stimulation o With no cones § Some people can experience colour as a result of other senses being stimulated • Synaesthesia (not limited to colour) • Colour blindness o Not about the properties of the mind § May be able to experience all the colours a normal person can • Can’t measure experience o Can’t label colours the same way • About discrimination between stimuli § Most colorblind people will experience all hue categories, none are missing • Colour blindness = the inability to make discriminations between stimuli based on wavelength info that a colour normal person can make o See 2 things as different or not § Don’t see as different hues o A colorblind person fails to be able to discriminate between stimuli that a normal person sees as different § See 2 stimuli as the same colour that a colour normal person sees as different

§ • Orange and green seen as same hue o Hard to discriminate between hue § Many colour blind people fail to notice their deficiency unless tests are conducted • Often subtle • Phenomenology o Textbooks may say “colour blindness means you have trouble seeing X colour” or “X-Y colour blind people have limited ability to distinguish between X colour and Y colour” § When difference is subtle • About discriminating stimuli, not telling difference between colour § These are wrong if ‘colour’ refers to qualities of conscious visual experiences § We can’t mean they are unable to distinguish between their experiences of X colour and Y colour § Correct: “the X-Y colour blind person has difficulty telling the difference between 2 surfaces a normal person sees as being X colour and Y colour” • Stimuli, not experiences o Just that experiences are used to label stimuli § Still see green/red in other situations • Ishihara test plates o Standard colour blindness tests are based on the ability to see a figure defined by spots of one colour displayed amongst spots of another colour § Usually a number, but not always § § They are printed in such a way that the figure is easily visible only in colour - not in black and white § The normal person can see the colour defined figure quickly- it will pop out § A colorblind person either fails to see it, or takes a lot longer and be uncertain • But they can see the different coloured spots • Physiological basis o The cause could arise anywhere in the visual pathway from the retina to the cortex

o o Almost all cases are due to problems with cones due to genetics; one or more types of cones missing o 7 total possibilities: § 1) all the cones are missing (only have rods) § 2) Two cone types are missing, only one present § 3) One cone type missing, so two remain • Only 5 of these have ever been reported, grouped into 2 types: o Type 1) Monochromats § i) Cone monochromats • Only S-cones and rods § ii) Rod monochromats • No cones, only rods o Type 2) Dichromats § i) Protanopia • No L-cones § ii) Deuteranopia • No M-cones § iii) Tritanopia • No S-cones o Mainly congenital, but can be caused by damage from disease or chemical poisoning

o Cone distributions in the central region of a normal retina (left) and protanopic retina (right)

o • Monochromatics are extremely rare, and tritanopes are very rare • Protanopia and deuteranopia each occur in about 1- 1.5% of the male population and 0.05% in females • The other 5-6% comes from a 3rd type: o Type 3) Anomalous trichromats § i) Anomalous protanopia: L-cones similar to M-cones § ii) Anomalous deuteranopia: M-cones similar to L-cones (most common; 5% males, 0.35% females) § iii) Anomalous tritanopia: S-cones have unusual sensitivity

o Anomalous tritanopia is basically unheard of and isn’t well understood- may have diverse causes • In all types, all 3 cones work fine in these deficiencies, the M/L cones simply have slightly different absorbance spectra than normal o The photopigment is defective in the sense that it is not sufficiently different from the normal pigment for normal colour vision, but works fine § Can’t make discriminations • Only monochromats are likely to be unable to see colour o They are photophobic in normal daylight illumination o Can only see at light levels where rods are active (scotopic and mesopic conditions) § Normal people do not see in colour in scotopic conditions o Cone monochromats will have their blue cones active in mesopic (low light) conditions, some report seeing shades of blue and yellow o Also known as congenital achromatopsia § All these types are due to anomalies in cones • There are a few instances of colour blindness due to problems further along the visual pathway o From damage or disease and can result in acquired achromatopsia: § 1) Cerebral achromatopsia: result of damage to V4 § 2) Thalamic achromatopsia: which results from damage to regions of the LGN (conveys colour related info to the cortex) • True achromatopsia from damage to the thalamus and cortex are very rare • Cases of partial colour vision loss due to cortical/thalamic damage in which colour vision in impaired, but some colour still perceived o Dyschromatopsia § Colours are still seen but are desaturated and have colour matching deficiencies different to those from colour blindness involving cones

Colour illusions and some others

• Colour illusions o Recall the definition of illusion: § Perceptual illusion: a consistent and persistent discrepancy between a sensory percept and the distal stimulus that evoked by the percept such that the observer is deceived as to the nature of the distal stimulus. The discrepancy relates only to those stimulus properties which can be detected by the sensory systems; it occurs as a result of the normal processing of proximal stimulation. § Are there any phenomena in colour vision to which this definitions applies • Not obvious as colour is not a property of the world but a quality of experience § Surface colour is a representation of the surface spectral reflectance • The way in which spectral reflectance info in presented in experience • It is not spectral reflectance o How the NS represents it in experience o An illusion would occur if the wrong label (colour) were attached to a spectral reflectance § E.g. blue is the label that should be attached by the NS to spectral reflectances that are greatest in the short wavelength region of the visible spectrum • How the NS presents in the mind that the reflectance function in which the greatest reflectance is at the short wave end • If a surface with a different spectral reflectance appears blue, it would be illusory • The dress o o Survey showed that 60% see it as blue and black/dark grey, 30% see it as white and yellow/gold, and 10% were intermediate (usually blue and yellow/brownish) o This demonstrates individual differences in colour perception § Has nothing to do with colour blindness or short-term adaptation effects from prior inspection of different coloured images • 2 normal people who view the same image can still see the image differently o Is it an illusion- do some people misperceive the dress? § The image is actually of a blue dress, so those who see it as white are misperceiving it o Why do some experience the illusion and others not? § The image is washed-out blue-grey § Similar to this image, where any redness of strawberries is illusory:

• o The redness can be explained as the result of estimating that they are illuminated by a bluish light, and discounting it § Similar with the dress • Differences in the percept depend upon the extent to which people attribute a bluish wash to the illumination and discount it o Discounting the bluish illuminant estimate will yield a white/grey and yellow/brownish dress § Such differences could be a result of long- term experience with different lighting • Lightness/brightness illusions o The brain is trying to interpret proximal stimulation in terms of the reflectance of surfaces, and has the wrong answer § Deceived to the true nature of the distal stim as a result of processing carried out by the brain of proximal stim

• o A and B are in fact the same colour § Grey is to be understood as the way in which the NS presents in the mind the fact that the reflectance function is flat • Over 80% reflected = white, less than 5% = black

• § Squares A and B have the same reflectance, but the NS presents them as different • Because of the other things in the image o The image is seen as part of a scene where the cylinder casts a shadow on the board- in this context they appear different § But there is no cylinder, no squares, no shadow, and no board- all just an image • Everything you see could be described as an illusion as it is not real o We know it’s a picture of a scene and not real, so don’t feel deceived o But the squares seem different due to the unfamiliarity they cause § If this was real there would be no illusion • Would see A and B as different- which is a correct perception • Retinal images are formed from the light reflected from the paper the image is on, then we misperceive the reflectance properties of the paper § The NS is designed to interpret the real world o The ability to perceive reflectance despite changes in the illumination intensity is called lightness constancy (or achromatic colour constancy) § A black object in bright light may reflect more light than a white object in a dimly lit room • But we see them as black and white regardless of the amount of light that reaches our eye § The chequerboard illusion shows that to get this constance, the VS takes scene geometry and shadows into account • Brightness illusions rely on this o Brightness illusions are telling us something about how visual perception works § The VS usually gives the correct distal interpretation of the proximal stim when the image is in real life § It gives us the wrong one if the image is a picture • The VS is wired to extract info about a 3D world when presented with proximal stimuli (retinal images) o This makes perception possible • Illusions o There are a huge variety of known visual illusions, categorised by description of the type of distortion § 1. Misperceptions of lightness & colour § 2. Distortions of tilt, shape and curvature § 3. Distortions of size and distance § 4. Misperceptions of motion § 5. ‘Invented’ contours and edges § 6. Real but impossible objects • Twisted cords o § See this as a spiral even though they are really many consecutive circles § From a family of illusions created by ‘twisted cord’ patterns o Illusions composed of sequences of elements with a basic form like this:

§ • Laying out the elements in straight lines makes them seem tilted at an angle o Adding triangles at the end of stripes accentuates the illusory effect

o • Illusions of size and distance o Many famous ones are of this § To do with distance; same size, closer = smaller, further = bigger § o Shepherds tables

§ • The table tops are the same size, just tilted o Illusion of size o Only occurs with a picture • Motion illusions o Motion is perceived when no motion is present in stimulation § This is how we see motion in movies § Called apparent motion • Actually many still images one after the other, with a pause between frames • Subjective contours o Illusory ‘edges’ induced by certain stimulus patterns o Illusory shapes are produced when the subjective contours enclose a region § • Impossible figures

o o Are these illusions? § They are pictures of objects that cannot exist • But they are no more illusory than any picture of a possible object o What is being misperceived? § Can occur in real life • Must be viewed at the right angle, using gaps and angles o NS interpretation

Perceiving depth, distance and size

• Depth and distance o We see the world as 3D § Experience solid, 3D objects at different distances from us and each other o Proximal stimulation of retinal images is 2D - no 3rd dimension § Need to obtain info about the 3rd dimension from these images o Visual perception of the 3rd dimension is called depth perception § Need accurate info to stop us making mistakes • Cues to depth o Sources of stimulus info about a property of the environment/object are called cues § Sources of info about the distance away from you are called depth cues • There are a number of different depth cues o Binocular cues (both eyes) o Monocular cues (one eye) § Pictorial (in pictures) § Non-pictorial (movement, distance enhances movement) • Pictorial cues o Occlusion: when one object obscures an object that is behind it from an observer, it is occluding it § When one object occludes another the occluded object must be further away • Only information about depth order, not about distances o Ordinal cue: gives info about depth order but not distance o Metric cue: gives info about absolute/relative distance • Size and position cues o Size provides the basis for depth cues § Recall: • 1) Image size alone provides no info about distance • 2) Image size depends upon the object's distance from the eye o Larger when object is closer o As image size depends on distance, if you know how large an object is, then the size of its image can be used to estimate its distance § Use “similar triangles” to work this out:

• • Means that an object’s size can be used to determine distance from image size means that image size can serve as a depth cue for objects that are familiar o The use of prior knowledge about an object’s size is known as familiar size cue to depth § Provides metric info about depth o Use of familiar size involves making the assumption that the object being viewed as the same size as those previously seen o Size can provide depth info even if the actual size is unknown, provided that there are a number of objects of the same type at different distances § Are assumed to be the same size, so smaller ones are further away • • Called the relative size cue to depth o Provides only relative info about depth (e.g. one rabbit is 2X as far away as another) § Metric info • Relative height o Relative height cue: closer objects on the ground plane are lower in the visual field § Further up=further towards horizon o Multiple similar elements over the image that are larger lower down in the image, and get progressively smaller and higher produces a pattern that gives a strong impression of depth § Texture gradient

• § The impression is of a ground plane receding in the distance - don’t need discrete elements for the effect § If elements are laid out orderly, then there is a texture gradient cue and a perspective cue • § Foreshortening also increases with distance • More stretched out/oval-shapes § Foreshortening and size cues can be used to create impressions of depth and shape of the ground

• • Aerial perspective o Aerial perspective (haziness): cue based on the implicit understanding that light is scattered by the atmosphere § Further = looks bluer, fuzzier o Can give impression of depth when added to an otherwise flat scene § Provides depth order info, but not info about relative/absolute distance (ordinal info) • • Shadows o Can also provide ordinal depth info § Some metric info if it allows comparison

• • Linear perspective o Based on the idea that lines that are parallel in the 3D world will appear to converge § Point of convergence is the vanishing point o If you draw your objects at different depths aligned correctly with radial lines, a depth impression can be created

§ o Other cues are often present in addition to perspective cues § But don’t need much to get a depth impression • Pictorial depth information o Monocular cues §

Pictorial cues Info provided Useful range

Occlusion Ordinal Near to >>30m

Familiar size Metric (absolute) Near to >30m

Relative size Metric (relative) Near to >30m

Relative height Metric (relative) 2 to >>30m

Texture gradient Combo of relative size and height

Aerial perspective Ordinal >>30m

Shadows Unsure, likely ordinal <10m

Linear perspective Metric (relative) 2 to >>30m • Image size and actual size o Size of an object’s image is determined by its distance away o In normal conditions we don’t see objects as having sizes that depend on their distances § When we approach an object it doesn’t appear to get bigger as the retinal image gets bigger, stays constant even when image size differs • Image size = proximal stimulus, but we perceive the distal stimulus (the image’s actual size) - actual property of the world o The perception of the distal stimulus is called size constancy § Only approximate and works well in some situations and not others • Work fairly well in natural environments o In which we evolved in and are adapted to § Can subvert size constancy o Image size can serve as a depth cue because the size of an object's image is determined by distance § Familiar size can be used as a distance cue and texture gradient as depth cue § If you don’t know how big something is but know how far away it is, you can determine actual size from its image size and distance • The idea that the size something appears to be depends upon one’s perception of how far away it is is sometimes called the size-distance invariance hypothesis o Perceived size = perceived distance x image size § See it as further, appear larger, than if same size image and perceived as closer § Accurate if: • 1) image size is very small o Retinal images are small • 2) size can be accurately measured by the visual system o Size of the image • 3) perception of its distance is accurate • These 3 are true, perception of size should be accurate o Leads to size constancy if distance perception is accurate (perceived size won’t change because estimate of size remains the same), if this holds you should be size constant § If size constancy fails/is imperfect, is it because SDIH is incorrect, or because distance perception is inaccurate, or because image size can’t be measured accurately • Can be any of these, can’t tell when size constancy starts to break down, they go together o Sometimes one, sometimes another • Emmert’s law o Supports the SDIH (perceived size depends on perceived distance), describes the perceived size of afterimages viewed in different circumstances § “the apparent size of an afterimage is proportional to the distance of the ground against which the afterimage is seen” • After image → look at surface → far away, after image appears large. Close, after image looks small o Proportional relationship o The afterimage depends on the distance of the surface being viewed § Further away = appears larger • The afterimage itself is due to bleaching of the cones stimulated by the original bright thing o Can be considered having a fixed retinal size as the part of the retina the image comes from is the same size • Size of the retina the afterimage is on is always the same o Only distance changes • Size illusions o As perceived size depends upon perceived distance, this can be used to construct size illusions § Cues for distance and depth - image sizes are different • IMAGES are identical in size

• § In most cases, the main depth cue is linear perspective • Linear perspective makes it seem further, SDIH makes it seem larger o You don’t need much to get the effect § Ponzo illusion works horizontally and vertically

• o These illusions are failures in perception if you think that vision is for telling you about images § Not failures if you think vision is about seeing the 3D world, not sizes/shapes of 2D images • You see a 3D interpretation of the image derived from the cues to depth and distance that are present in it • Advantages of binocularity o 1) Built in spare eye o 2) larger field of view § Prey animals often have almost complete coverage of the visual field § Seen by both eyes = field of binocular overlap/binocular visual field • This region is larger in animals with forward facing eyes (nose gets in the way) o 3) Binocular summation § Two eyes see better than 1 in the overlapping region • Add the image together § Threshold intensity for seeing a faint stimulus is lower in binocular than monocular § Probably contributed to evolution of forward facing eyes in nocturnal animals • Detect prey, climb through trees o 4) Stereopsis § Differences between images of the same features in the 2 eyes provides a depth due • Using this cue to see depth = binocular stereopsis • See as 3D § Can see same thing in both eyes, but slightly different • Eyes in slightly different places o This difference lets you get impression of a solid 3D object § The images of the same 3D scene that are formed by an imaging device in 2 locations will be different from one another • The difference in the positions of the same element in the 2 images is called disparity o Binocular disparities = the differences between the image seen by the 2 eyes § Purely binocular o A depth cue o 5) Convergence of the 2 eyes § Isn’t a visual cue § To fixate, closer images your eyes have to turn in, creating a larger angle of convergence • Depends on distance § The angle of convergence provides a cue to the distance of the fixated object, can be used to scale relative cues in the image § No retinal image information about its distance or direction • Use convergence o Get roughly right • Visual space perception o Kinaesthetic info about the position of the eyes in the head and about the position of the head on the shoulders is used § Visual space perception not based exclusively on retinal stimulation, involves proprioceptive afference/efference • Receptors and muscles that are relevant o Eyes, neck • Distance info of the spot is provided by proprioceptive info about the angle of convergence of the eyes • Extra-retinal info o The larger the angle of convergence, the closer the fixated object § Objects >6m away the eyes are parallel o Potentially useful distance cue for nearby fixated objects § Not for further away, as eyes are parallel • Less than 5-6m § Do we use it? • Yes, when in isolation this is sometimes the only cue they can use o Artificial § Is it used when we have other depth cues? • Vergence angle info o The angle of convergence is related in a 1-to-1 fashion to the distance of the fixated object o Lets you know the distance to the fixation point § If it serves as a cue to distance, then the perceived distance of the fixated object should change if vergence angle changes if it is the only available distance cue • Left eye needs to move to fixate, right eye does not

o • Can be changed with prism over eye, thick at the base on temporal side (side of temples), when there are no other depth cues • Rays of light are refracted so the apparent direction of the spot is changed o Eye must move to fixate on the image, so vergence angle increases

§ Move to apparent location • Change convergence needed to fixate o If use convergence to perceive distance, the light should appear closer § If vergence is used as a cue, then the spot should be perceived as its apparent distance, not actual • Imagine that different prisms are placed to create a range of apparent positions o People asked to fixate on the target and reach to its location o § If vergence is sufficient for accurate distance perception, then people should accurately point to the target’s apparent distance • The distance as specified by vergence o Results show vergence angle changes perceived distance

• Red line: using vergence as depth cue, and accurate in its use o Reality: when vergence angle is large, see as closer, when small they see it as further § Not accurate - makes errors (largest at the extremes) • Consistent relationship between vergence and perceived distance o Yes people use vergence angle in consistent way, but not used very accurately under these conditions o Similar results reported when other cues are used in isolation § When only 1 cue is available this pattern occurs • Only accurate when you have multiple cues o The cues work in the same way § Image or extraocular muscles § Regardless of relative or absolute • When more than 1 cue is present, impressions of depth are more compelling, distance estimates more accurate o More cues = more accurate § Multiple cues are combined to yield accurate perceptions of distance and depth in the normal world § Not all from the same stimulus modality (not all visual, e.g. Convergence is about receptors of extraocular muscles) § Most retinal, vergence isn’t • Role of vergence in distance perception o Multimodal percept § The final percept of distance involves multiple modalities • Not just based off what we see/light reaching the retina o Use whatever source of info the NS can access § Use cues they have access to § Closing one eye doesn’t seriously impair distance or depth perception • Does vergence make a small contribution in normal conditions? o Same amount as any other cue § When there are few cues, removal of one will have a larger effect § Remove one of any cue, not much effect • Only have vergence → close one eye → no idea • Cue combination o How is vergence combined/integrated with the retinal cues? o One way to combine cues that provide info about the same thing is to average them (in a weighted fashion), create better than any of the individual § Vergence provides info about the absolute distance to a (binocularly) fixated object or location - not relative • There are only a few other cues that provide such info and these may not always be present o Familiar size is the only absolute distance monocular cue § Most other depth cues provide relative distance info, and not usually about the distance to the fixated location but to/between other objects/surfaces • This is 2x as far, but not exactly where it is • Often only ordinal (e.g. occlusion) • Relative provides absolute o Relative distance info means you know the distance from yourself in a proportional manner (texture gradient, linear perspective) § You don’t know the scale, only how distances between objects compare (if you don’t have absolute distance cue) • Know things are multiples of X distance from you (e.g. one object is 2 times as far), but don’t know what X is o You don’t know the scale § Relative, not absolute • Can see depth but don’t know absolute distance o Add familiar size cue, and get good impression of how large something is/its distance § Add familiar size cue → can know size/scale factor • Scale factors o If you know the absolute distance to one object (e.g. fixated object), you can know the distances to all other objects, you know X, and know multiples § Got vergence/distance to an object, know the rest § Vergence provides info about absolute distance to the fixation point (X), so gives a scale factor • Shows its importance • Role of vergence in giving a scale factor is supported by telestereoscope o a system of mirrors that increases the vergence angle needed to fixate an object § as if you a looking at the world with the eyes much further apart than normal • Further apart, more convergence needed to fixate o So things appear nearer • Things far enough to not need to be converged on with normal eyes need to be when using this, as eyes are further apart § Perceptual effect: everything appears as nearer and smaller than it is - like a scale model • Need to converge more/larger vergence angle → must be nearer o But retinal image size is the same, so must be smaller • Explained in terms of the scale factors/ absolute depth cue provided by vergence angle o That’s what’s being changed § Gives you absolute distance • Almost like an illusion o Subverge normal relationship between convergence and distance by using the mirrors • Scale factor changes • Increases vergence angle needed to fixate on the object o Larger angles = closer fixation distance o Closer, and image size unchanged = appears smaller § Emmert’s law: if the object appears closer and image size is unchanged, then it will appear smaller § If vergence provides the scale factor for relative depth cues, it would be expected that the world would appear like a small scale model, and this is what happens • Depth perception o Perception of depth and distance is based on numerous depth cues § Most of these are visual cues (retinal image) • Convergence isn't - it’s kinesthetic o Called an extra-ocular cue to depth (outside of the eye)

o § Perceived depth results from integrating cues • Familiar size and vergence give absolute distance to fixated object, which provide a scale factor for other (not fixated) objects where we don’t have absolute cues o Vision isn’t completely ‘visual’ § Stimulation in muscles and receptive tissues 2nd half- essay Qs Summary of disorders and case studies for this half

Deficits with Perception • Cognitive neuropsychology o Studies cognitive impairments o By looking at what’s gone wrong, we can see how processes work normally § Use patterns of deficits to develop models of normal cognitive functions and processes o Look at case studies of patients with brain lesions § Use this to understand normal cognitive ability o Use a variety of neuropsychological tests to understand the patient’s impairment o Look for dissociations and double dissociations of different abilities § Dissociation: a patient who is impaired at 1 task but normal in another • E.g. Broca’s aphasia; producing vs understanding speech § Double dissociation: 2 or more patients with opposing deficits • E.g. Wernicke’s vs Broca’s aphasia § From these we can conclude that 2 functions involve separable processes o There are individual differences in case studies § Observations from single case studies or groups of cases can provide valid data to test and develop cognitive theories • Ways of acquiring brain damage o Neurosurgery § In absence of pharmacological treatments patients sometimes have neurosurgery • E.g. corpus callosotomy in cases of severe epilepsy o Strokes § Disruption of the blood supply in the brain • Ischemic: due to blood clot • Hemorrhagic: due to ruptured blood vessel § Leads to death of neurons and loss of brain tissue o Traumatic head injuries § The most common form of brain injury for those <40 • Common in young males from road accidents o Tumors § The brain is the 2nd most common site for tumors • Formed when new cells are produced in a poorly regulated manner • This puts pressure on the neurons o Disrupts function, can lead to cell death o Viral infections § Viruses target cells in the brain • E.g. herpes simplex encephalitis (Clive Wearing), HIV o Neurodegenerative disorders § Increase in brain-affecting degenerative illnesses as we live longer • E.g. Alzheimer’s, dementia, Parkinson’s • Visual agnosia o Means “without knowledge” o Impairment in ability to recognise visual objects § Not a visual problem • Can make brightness judgments, detect movement, no acuity impairment o Just can’t say what objects are o From video case study: § Can use small components (e.g. colour) and memory to infer what an object is • Adapt using other senses § Affects reading § Can recognise faces • Can’t see the whole object, but can see the whole face o Different brain center for objects/symbols than faces • Double dissociation from prosopagnosia o Case study: Dr P § Neurological testing found he had agnosia § Asked what a pair of gloves was • “A continuous surface infolded in on itself. It appears to have five outpouchings, if this is the word. [It could be] a container of some sort? It would contain its contents! There are many possibilities. It could be a change purse… for coins of five sizes” o Sees details, describes parts, abstract o Cannot connect description to object o Case study: HJA § Shown a picture of a carrot • “I have not the glimmerings of an idea. The bottom point seems solid and the other bits are feathery. It does not seem logical unless it is some sort of brush” o Only looks at aspects • Saw onion as necklace • Types of agnosia o Lissauer said there were 2 basic forms § Apperceptive agnosia: result of impaired perceptual input (peripheral) • Problem with percept o Can’t form stable representation of an object o Fail to copy images § But see their copy as accurate • Can occur after strokes, anoxia, CO poisoning § Associative agnosia: result of impaired semantic knowledge (central) • Have normal percept o No ability to attribute identity o Unable to map percept onto stored knowledge • Can copy an image accurately, but still can’t recognise it o Approach copying as if they’ve never seen the object before • Modern account of agnosia: hierarchical model (Humphreys and Riddoch) o § Object: basic details § Edge grouping by collinearity: shape coding § Feature integration/figure-ground segmentation: integrated bits and separate them from the back ground § View normalisation: recognise it at different angles, mental rotation, object constancy § Stored structural descriptions: map representation onto these, items have X structure even if they visually differ § Semantic knowledge: SSD onto stored knowledge its use, its name o Apperceptive agnosia § Edge grouping by collinearity tested by: • Copying pictures o Unable to copy pictures § Need to do further to rule out hand skills • Matching basic shapes o Cannot match basic shapes/letters o Can’t make basic shape rep • Efron test o Shown 2 items and asked if they’re the same or different § Correct = intact shape coding, wrong = impaired shape coding • Debate around pseudoagnosia o Any sensory problems in identifying an object § Brightness, colour, acuity, orientation etc o Is shape sensory? Is failure to perceive shape agnosia? § People disagree • Case study: DF (Goodale et al) o CO poisoning caused posterior cerebral lesions o Failed at Efron test, but could use shape to guide action § Shown a slot • Unable to describe its position or rotate her hand to match it • But could put her hand/a card into the slot o “Knew” about orientation, can calibrate hand movements § Can’t discriminate between blocks of different sizes • But could pick them up accurately o Impaired at perceiving shape, but good at making actions towards shape o Dorsal and ventral routes § Dorsal: vision in action → parietal areas § Ventral: vision in perception → inferotemporal area • DF had a lesion in the occipital lobe leading to ventral route o Issue in perception but not action § Feature integration/figure-ground segmentation • Tested using overlapping figures o Overlapping figures § If patients can see what features are part of which object and if they can separate from the background • Patient Mrs FRG had cortical degeneration o Could discriminate between shapes, but could not make figure-ground distinction § 0/10 correct § Bad at picking out individual shapes § View normalisation • Tested using the unusual views test o Asked patients with right posterior lesions to name an object § Shown at unusual view = cannot name § Shown at usual view = fine at recognising • Dissociation • See them as different objects § Warrington and Taylor: treat same object at different view as different objects o Problems with the perceptual system § Perceptual system → derives representation of an object from sensory input • Matches input to our knowledge about objects o not be able to say that the two objects are the same because they haven’t got access to view independent representations of the object o Biederman: objects are made of Geons (basic building blocks like bricks, cone, cylinders) § Combine these to make different objects § In unusual views these geons are distorted • Need to be transformed by the perceptual system to recognise them o Patients are impaired at this o Associative agnosia § Stored structural descriptions • Tested by two exemplar matching task and object decision task o FRA: suffered left hemisphere stroke that affected his occipital lobe § Could see percept of object- no apperceptive agnosia- but couldn’t name them o Matching function task: two dissimilar pictures of the same object, then one of a similar looking different object § FRA performed badly- matched the wrong objects § When shown images, failed to be able to put them in order of their real life size (put them in order of image size) - need mental reps to do this • When verbally told to put them in order, he did fine o Still has access to semantic knowledge o Object decision tasks: shown a real object or made up one, asked which is real § Impaired structural knowledge • HJA was impaired at this o Was better when they were silhouettes § Semantic knowledge • Tested by semantic knowledge task o FRA was unable to access objects visually but still have semantic knowledge if given auditory info § Semantic knowledge component is accessed by different modalities (hearing, vision, touch) o Patient HO had herpes simplex encephalitis that causes a lesion of the temporal lobe § Was good at object-decision task (stored structural descriptions intact) § When asked semantic knowledge about an object she was poor • Semantic access agnosia o Proposed that if semantic knowledge is damaged, then you will not be able to access semantic info regardless of modality o Don’t recognise the object § Missing meaning associated with the object

•

Apperceptive agnosia Associative agnosia (central) (peripheral)

Description Result of impaired Result of impaired semantic perceptual input knowledge

Object perception Cannot form stable Can copy accurately, but fails to representation of an identify the object as unable to object - fails to copy map percept onto stored knowledge

Edge grouping by Unable to copy pictures, Can copy pictures, can match collinearity unable to match basic basic shapes, passes Efron test shapes, fails Efron Test, but good at making actions towards shapes

Feature Fails at these, cannot pick Can pick out individual shapes integration/figure- out individual shapes ground segmentation

View normalisation Cannot identify object in Recognises the object as the unusual views test same, but cannot name it

Stored structural Would fail at matching to Can percept object but not descriptions function as cannot name, fails at matching to perceive the object, fails function (goes by shape rather at object-decision tasks as than object use), cannot put cannot recognise objects animals in order of size from at all images but can if told verbally, fails object-decision tasks

Semantic knowledge Cannot perceive object so Poor semantic knowledge cannot give semantic knowledge Can give knowledge when asked via auditory question

Deficits in perception II: prosopagnosia and blindsight

• What is prosopagnosia? o The inability to recognise familiar faces § Family, friends, celebs, own face o Hecaen and Angelergues § Patient could recognise difference between man and woman • Could not differentiate between his wife’s and his mother’s faces • Uses other cues to recognise o Wife’s silhouette • Prosopagnosia o Usually occurs after bilateral damage to ventral occipital cortex o Can occur after damage to right hemisphere o Damage is common, disorder is rare o May be able to: § Match unfamiliar faces, perceive facial expressions, identify people based on other cues o Introduced by Bodamer: case studies § S: could tell certain objects were faces, not who they belonged to • Couldn’t recognise own face • Difficulty distinguishing between human and animal face • Could imagine faces (dissociation) § A: developed strategies for recognising people • E.g. Hitler by moustache and side parting • Very specific details others may only know implicitly, they use explicitly o Slower o Frustration • Functional model of face recognition (Bruce and Young) - how we normally perceive a face o o Structural encoding § Produces structural descriptions of faces • Configuration, features (where they are) § Provides descriptions independent of viewpoint or expression • Produced regardless o Configuration of features § Not what they express • Mental rep of how the face is built o Different info is extracted in parallel - for unfamiliar faces to remember them o Expression analysis § Configuration of features make up expression • Link (feedback loop) to cognitive system: feedback from LTM • Look at features to work out their configuration, send to CS, feedback decision about what expression is being shown o LTM for facial expressions, feedback info to expression analysis o Speech analysis § Mouth and tongue movements, lip reading • Analyse facial speech § Feedback loop from CS - interpret, decision making • Feed info about what the movements are, CS provides interpretation and feedback o What info is being conveyed o Directed visual processing § Selective attention to visual form of the face • Attention to individual features of the face • Esp used when searching for someone or trying to remember a new face o Remember facial configuration o Look for specific features, e.g pointy nose § When trying to find them • Selective attention to feature of the face § Looking at specific face feature o Unfamiliar § Try and encoded features to remember § Link to CS: attention, memory processes • Feed forward and back o The next stages are for recognition, also link to CS o Face recognition units § Stored structural codes about faces, contains info about all faces of a person • Expressions, angles • Meet → store structural information about what their face looks like § Sends signal to cognitive system depending on match of FRU to structural encoding • Info to and from CS to see if match § Primed directly if seen a person recently § Primed indirectly by PINs if you expect to see a person • Expect to see person, prime FRU so quicker to recognise due to prime o Person identity nodes § Access all the info we have about about the person • Occupation, relationship o Activate → access all the info we have about a person § Can be accessed by other means • Voice, clothing • Some prosopagnosics are impaired at face recognition but can access PINs via other methods o Name generation § Got structural description, compared to stored FRU, generated info we know about the person, then generate name § Back and forth between CS/LTM for info on who the person is § When we know someone but can’t generate a name, this is failure at this stage § Model predicts that we don’t know someone’s name if we don’t know their identity • Can’t get name without FRU or PIN o Won’t know name from face without knowing anything else about them o Hierarchical model: need to do previous steps to reach the next one o Cognitive system § Vague • Responsible for decision processes? Generate memory of person? Stores other info? • Unclear what this cognitive system does o But involved in recognition • Evidence for the FMoFR o Look at their impairments and work out where they fit in the model § Where the deficit in the process occurred o Benton and Van Allen § Prosopagnosic patient performed okay with unfamiliar faces • Say if they’ve been shown a face before o Show unfamiliar faces, than asked which they’ve seen before • SE okay, FRU, PIN and NG fail o Causes issue with familiar faces o Can generate structural descriptions with faces § Use memory based on that to say if they’ve seen it before o Benton § Patients who are not prosopagnosic but have problems matching unfamiliar faces • EA, SA and DVP fail o Can use to determine if they’ve seen an unfamiliar face or not • SE, FRU, PIN, NG are fine o Double dissociation § Impaired at unfamiliar unpaired at familiar o Tranel et al, Shuttleworth et al § Patient recognises expressions but not familiar faces • EA fine, FRU, PIN, NG fail o Kurucz and Feldmar § Recognised faces but could not identify facial expression • EA fail • SE onwards fine • Double dissociation o Can recognize faces, no expression o Others can’t recognise faces but can’t recognise faces • Covert recognition of faces o Prosopagnosics can’t recognise faces explicitly, but is there evidence of implicit recognition? § Some show implicit facial recognition § Unconscious recognition o Bruyer et al: Mr W § Paired a face with a name § When name is correct, they find face pairing easier § Mr W not aware of having recognised face • Covert awareness of name and face matching § o Greve and Bauer § Presented patient with unfamiliar faces § Re-presented pairs of faces, one old and one new • When asked which they’d seen before they just guessed • When asked which he preferred, he chose the previously seen face o Implicit awareness of seeing face before o DeHaan et al: PH § Couldn’t recognise faces overtly § Asked to classify a name as politicians or not - given face with printed name • Slower to respond if face is from different category o Something about incongruent matching between face and name that slows process § Implicit awareness of matching § That name is politician but face isn’t → slower § That name isn’t politician but face is → slower • Interference from face

• o Bauer § Patient looks at photos, and skin conductance responses was measured • Response increased when correct name was given with correct face § Suggested 2 neural systems of facial recognition • Conscious→ ventral route • Unconscious→ dorsal

§ • Is prosopagnosia face specific? o Faces contain important social info, but also most faces share similar characteristics § Is prosopagnosia a result of breakdown of within-category recognition, or is it specific to faces? o McNeil and Warrington: WJ § Brain damaged § Learned to recognise sheep but unable to recognise faces o Riddoch et al: FB § Fine with naming other categories of complex novel objects • Recognition deficit confined to faces o Recognition deficits in other categories; not specific § LH: poor at recognising faces and 4-legged animals § PH: poor at recognising cars and flowers § Bornstein et al: prosopagnosic birdwatcher unable to recognise birds, prosopagnosic farmer unable to recognise cows • Theories of prosopagnosia (from seminar) o Visual expertise at within category discrimination o Faces are a distinct, separate category o Task difficulty o Part based vs holistic based perceptual processing • Blindsight o Have a scotoma (blindness in part of the visual field) o Clinically blind (from problems with striate cortex) § But able to identify a visual stimulus despite no conscious awareness of stimulus - detection but unaware o Nothing wrong with eyes § Only brain damage; cortically blind o Poppel et al § Patients suffered lesions to the striate cortex (V1), blind in the area of their scotoma • But could make saccades towards visual target when asked to guess where it was o Weiskrantz: DB § Surgery removed striate cortex, caused left visual field defect (hemianopia) § Could “see” things that appeared in his blind field § When asked to guess DB could: • Point to markers on the wall • Say if a stick is horizontal or vertical o But unaware of visual stimulus or that he was correct • Detect if an object was present/absent • Detect presence of moving vs. still target • Discriminate between some stimuli § Said “no sensation, just guessing”, but 93% accurate • Paradoxical • Neurobiology of vision o Info passes from retina → V1 via primary geniculo-striate pathway § 90% of visual info transmitted via this o There are other pathways from the retina to the brain

§ § If the striate cortex is damaged, you can use other visual pathways • Non striate vision • Other pathways: o Retino-tectal o Geniculo-extrastriate § Both terminate in extrastriate cortex • Types of blindsight o Weiskrantz § Type I: no awareness, but can detect stimulus § Type II: awareness of something but not visual percept (“feeling”) o Danckert and Rossetti § Proposed a new taxonomy • Action blindsight: can point/saccade/grasp target - make actions • Attention-blindsight: sense stimulus, motion detection, discrimination (similar to type II) • Agnosopsia: no awareness but can guess perceptual characteristics (similar to type I) § Different pathways for each type • Action: visual projections may terminate in dorsal extrastriate o Important for visually guided actions • Agnosopsia: visual projections may terminate in ventral extrastriate cortex o Responsible for form and colour perception • What does blindsight tell us about the visual system? o Kolb and Braun § Gave ppts contrasting textures to each eye, and using polarising goggles these were combined to form one image, simulating blindsight • Given a target differing in texture to spot o Glasses makes it seem like there’s no variation

• § Ppts report seeing a uniform texture with no explicit awareness of the target • Accuracy of target detection and confidence ratings were recorded o Images presented briefly or one eye gains dominance and they see the target • Compare target detection to trials where target is explicitly visible o No awareness, but correctly guess location of target 75.2% of the time § Similar to accuracy where target is visible o Not aware they were correct, thought they were guessing § Less confident when they can’t “see” the target

Deficits in planning

• Deficits in planning o To investigate deficits in planning we look at patients with frontal lobe damage § The frontal lobes may be involved in planning, programming, and regulating behaviour • Motor functions o Move limbs in response to goals and intentions • Higher order functions o Planning/impulse control o Reasoning/problem solving o Memory § Has no discrete function • Critical role in how we use info from other areas of the brain § Damage doesn’t mean they lose knowledge - retain intelligence (dissociation) • Lose ability to apply knowledge and problem solve § Subtle problems, so need to devise tests to find them • Tend to just jump into the test, don’t think and plan § Most people have a purposeful plan, make judgments, assess and choose ideas • This is from the frontal lobe o Damaged → respond without reflection § Can’t plan o Regulates behaviour to control the environment • Phineas Gage o Accident caused a metal rod to pierce his orbito-frontal areas § Survived, but behaviour changed o Pre-accident: shrewd, smart businessman, persistent in executing plans, efficient and capable, well liked o Post-accident: vulgar, intolerable, impatient when desires are conflicted, abandons plans for ones that appear more feasible, confabulated § Problems inhibiting impulses o Suggests frontal lobes involve planning and maintenance of behaviours o Issues with social cues (e.g. facial expressions) • Saver and Damasio; patient EVR o Operation removed parts of frontal lobes o High IQ o Post operation: § Developed socially abnormal behaviour § Had problems planning and organising life § Relationships suffered § Bad financial decisions → bankruptcy § Deliberates a lot over simple choices • E.g. what to wear • Demazio and Demazio o Man suffered a stroke § Always emotionally “neutral” § No skin conductance response to disturbing images § Doesn’t emotionally connect o Lost part of social brain o Reason and planning damaged § Test for reasoning, planning and intuition (gambling test) • Normal person; shifts to safe option • Patient: doesn’t switch, becomes bankrupt § Patients don’t show anticipatory response • Can’t learn by experience o Social cognition damaged § Cues, what’s appropriate, theory of mind • Karpov et al o Asked patient to scan an image to answer a specific question § Strategies used were measured by eye movement § Normal: scan image then look at individual parts § Patients had difficulty modulating eye movements • Focus on one prominent detail right away, hard to look away § Failure to adopt/generate an appropriate strategy • Executive function of frontal lobes o Luria: frontal lobes are responsible for: § Programming, planning, regulating behaviour § Verifying if a behaviour is appropriate for a situation o Frontal lobe patients have problems with these • Supervisory Attentional System o Norman and Shallice: § Developed the idea of the executive • Responses can be controlled by different ways: o 1) Automatic/partially automatic control § Triggered by environmental factors • E.g. walking, driving § Something happens → can deal with o 2) Willed actions § Where the SAS comes into play § More complex actions when routine control is insufficient • Novel situation, create a strategy o SAS may help with: § Planning/decision making § Error correction/troubleshooting § Responses that are not well learnt § Novel actions § Responses that are difficult or dangerous § Overcoming strong habitual response/resisting temptation o If there are problems with the SAS: § Automatic actions will persist until inhibited by SAS • Damage then old actions persist and can’t be modified o In frontal lobe patients: rigid, inflexible behaviors § The SAS will inhibit unneeded/inappropriate responses • Damaged → inappropriate response in some situations • Neuropsychological evidence for SAS o Wisconsin Card Sorting Task (WCST) § Patients asked to sort cards by a rule, after sorting by the rule a few times, the rule changes (without being told, learn it’s wrong via error) • Need to sort by new rule • Frontal lobe patients sort by old rule o Nelson: Even when told it’s wrong, or sometimes even when told the rule has changed § Behavioural rigidity o Perseveration § Unable to inhibit actions § Perseverate at WCST § Movement perseveration • Luria: patients continue to draw circles until the pen is taken away, patient asked to write to dictation then asked to draw; the 2 tasks blend together o Fluency test § How many words can you think of beginning with a specific letter in 1 min? § Patients: have problems generating a list of words • Often perseverate with the same word/derivatives of it § Goodglass and Kaplan, L.E. • Said 3 words in 50 secs o Alternate uses test § Name as many different uses for X in 1 min • Think of normal use, then atypical uses • Patients have problems doing this as they have issues inhibiting the automatic response (normal use) o Stroop task; distraction § Asked to name colour of the ink, word may not match colour • Have to ignore word to respond correctly § Patients perform poorly • Possibly because of increased distraction o Utilization behaviour § Patients will reach out and use objects within reach • Even if it’s not theirs and they were not asked to do so • Objects afford a response § Perceptual input leads to activation of action schema • Action associated with the object • No SAS, then action carried out o No inhibition of action schemas triggered by perceptual input o Tower of London § Ppts have to move discs to match a goal position, in a specific number of moves • Requires forward planning § Owens et al • Patients fine at making first move, but took more time then controls on subsequent moves o Hayling sentence completion § 1) Complete sentence with a word that makes sense § 2) Complete sentence with a word that doesn’t make sense • Burgess and Shallice o Patients have a delay in response in task 1, and find it difficult to complete task 2 - fixate on word from task 1 Task name Description How frontal lobe patient would perform

Wisconsin Card Sort cards by a rule, then the rule Continue to sort by old rule Sorting Task will change (without person being (even when told it’s wrong or (WCST) told) and they need to sort by a told the rule has changed) new rule

Fluency test Name as many words beginning Have problems generating a with a certain letter as you can in list of words, often one minute perseverate with the same word/derivatives of it

Alternate uses Name as many different uses for Problems inhibiting automatic test an object as you can in one minute response, find it hard to generate atypical uses

Stroop task Name the colour of the ink of a Perform poorly, possibly (distraction) work, sometimes word is because of increased incongruent to colour the ink so distraction have to ignore word to respond correctly

Utilization Put an object in front of them and Will reach out and use it, even behavior don’t tell them to use it if it doesn’t belong to them

Tower of Move discs to match a goal Fine at making first move, London position in a certain number of take more time on moves subsequent moves

Hayling Task 1: complete a sentence with a Task 1: delay in response Sentence word that makes sense Task 2: difficult to complete Completion Task 2: complete the sentence with with word that doesn’t make a word that doesn’t make sense sense, want to say word used in task 1

Six Element Complete 6 separate tasks in 15 Perform below average, spent Test minutes (e.g. dictating route, too long time and issues with arithmetic problems) decisions Multiple Errands Go to a busy shopping mall and Find difficult, perform worse, Test perform various simple tasks, and are inefficient with time, break have to be at a certain place 15 and fail to interpret rules, fail mins after the task begins, give tasks specific rules

• Deficits in strategy application o Some patients show fine performance on neuropsychological tests § But impaired at everyday activities where they need to plan behaviour over longer times o In neuropsychology tests: § 1 problem at a time § Trials are short § Clear goals - know what has to be achieved o Real life: § Multiple goals § Tasks generally longer § Many ways to do a task (not a clear goal) § Have to overcome problems • More sensitive measures of frontal lobe problems o Shallice and Burgess § Constructed tests to investigate more open-ended multiple goal situations § AP: bifrontal damage • Severe organisational issues in life, unable to shop for himself • Fine IQ and performance on WCST, stroop, fluency, and ToL task § Six Element Test • Investigate ability to complete 6 tasks in 15 mins o Have to plan • Patients performed below normal range on these tasks o E.g. AP had problems dictating route § Spent too long making notes, hadn’t decided § Multiple Errands Task • Go to busy shopping market and perform various simple tasks o E.g. send a postcard with pre-specified info on it o Had certain rules • Be at a certain place 15 mins after the task started • Very complex o Planning, awareness, tracking time • Patients: o Found task difficult, performed worse, were inefficient in time, broke rules, failed tasks, showed interpretation failure of rules o More ecologically valid o These tests need maintenance of goals and intentions in planning § Plan in relation to environment • Need the SAS - can’t use routine actions to carry them out § Highlights need for SAS in: • Goal articulation, provisional plan formation, evaluation of process, plan modification o Goldstein et al § Patient underwent left frontal lobectomy • Had difficulty in making decisions • No changes in IQ or memory • Fine at WCST, sentence completion task, six element test • Less efficient in Multiple Errands Test o Broke rules • Deficit in strategy application, most likely due to impaired SAS o Multiple Errands Task is a more sensitive measure of frontal lobe impairment

Deficits in action

• Apraxia o Inability to carry out skilled actions § In command or mimicking o Not an elemental motor disorder § Not a weakness of 1 side of the body, not due to tremors o Likely a problem in cognitive motor planning o Causes: § Usually occurs after brain damage to left hemisphere § Stroke, Parkinson’s, Alzheimer’s § Prevalence rate of 33% of LH stroke damage o Steinhal § First coined “apraxia” → deficit in voluntary actions • Did not differentiate between agnosia and actions with objects o Patients with apraxia do recognise objects o More complex actions are harder o Will repeat previous action o Okay with some actions, some not o Thinks things through • Apraxia case studies o Liepmann: MT § Unable to imitate simple hand positions with right hand • E.g. okay sign § Unable to perform pantomimes (pretending to use an object) § Fine with his left hand, can comprehend the task • Unable to perform skilled motor actions with right hand o Pick § Patient aphasic but could understand simple order § When asked to use objects he made errors • Especially when the task had a sequence of actions o Wrong order, forget steps, add extra steps § Asked to light a candle: • 1) Held match with both hands and no more • 2) Inverted match and ground it into candle • 3) After match was lit for him he lit the candle and put it into his mouth (extra step) o Osiurak et al: MJC § Left parietal skull fracture § Difficulties in pantomiming and imitating § Asked “show me how to use a screwdriver” • More problems when a screw isn’t present (object recipient cues action and makes it easier) • Types of apraxia o Debated how many types there are § Old research suggested 3 o Wheaton and Hallett suggested 6 main types § 1) Limb-kinetic apraxia • Problems in basic motor coordination • Difficulties in hand/finger dexterity o E.g. picking up small object o Cannot be explained by more elemental motor problems involving cerebellum (e.g. tremors) § 2) Conceptual apraxia • Loss of knowledge of how to use an action/object o Not agnosia § Still recognize object, just lost of knowledge of how to use it o No loss in motor function • Inappropriate use of tools and objects o E.g, use screwdriver as hammer o Don’t link object to knowledge § 3) Ideational apraxia • Difficulty in executing correct sequence of steps of a complex action o Has knowledge of how to perform a complex task but struggles to carry it out and fail § Miss out steps, wrong order etc • E.g. making sandwich o Can tell you the general steps on how to do it § But struggle to do it o Action disorganisation syndrome § Failure at everyday tasks § 4) Verbal-motor dissociation apraxia • Patients fail to respond to verbal commands to make movements o Can instruct another way and they likely will be able to • May be disorder in speech processing rather than motor performance o Controversial § Unsure if form of apraxia § May be to do with speech processing, not motor performances § 5) Tactile apraxia • Unable to use hand as sense organ o Won’t ‘explore’ objects the same way others do • Hand skills that are not related to object exploration or manipulation are unaffected • Explore objects with hand differently to normal § 6) Ideomotor apraxia • Inability to: o Pantomime actions § Pretend to use a hammer o Imitate actions § Copy others o (Sometimes) use tools § May use it incorrectly, but not always • Object can be enough of a cue • Movements are spatially incorrect and slow • Can sometimes do actions automatically o Not under voluntary control o Other apraxias § Constructional apraxia • Difficulty in spatial tasks, to do with drawing o Drawing, making patterns with blocks § Dressing apraxia • Not dressing properly, usually leaving one side unclad o Could be result of visual neglect § Put on in wrong order, put on upside down § Facial apraxia • Unable to move facial parts on command o E.g. Unable to protrude tongue on command • Oral apraxia o Cannot pretend to blow out a match • Talking not affected, can imitate, repeats previous attempts • Ideomotor apraxia (IMA) o Motor problem not due to other motor/cognitive disorders § Weakness or numbness, tremors o IMA patients know what they are told to do § Not aphasic - no communication problems, understand command • Have the cognitive capacity § Not visual deficit, rule out other causes/disorders • No visual agnosia or visual neglect o Can recognise/see object o Have problems doing correct actions only, no other factor § Deficits in IMA § 1) Orientation errors • E.g. holding object upside down and attempt to use in that orientation § 2) Spatial and temporal errors • Wrong motion, timing wrong (too fast/slow) o E.g. up-down movements instead of forward- backward when pretending to carve a turkey, going too fast § 3) Making extra or unnecessary movements • Sometimes move wrong joints o E.g. over large movement § 4) May use a body part instead of a tool • E.g. use their arm as a knife to cut bread even when holding a knife • E.g. pretend to pick up cup but use hand as cup • Diagnosing IMA o Little consensus, but in general: § Rule out elemental movement disorders § Rule out disorders of simple movements • IMA affects complex actions o TOLA- test of oral and limb apraxia § 20 gestures classified along 2 dimensions • Tool use, limb involvement o Involve tool: transitive task o Don’t involve tool: intransitive tasks o Involve upper limb: proximal o Only involve hand: distal § Patients asked to perform each gesture, 1st to command and then to imitate § Includes 4 types of gestures, with 5 gestures of each type: • Proximal intransitive: upper limb + no tool o Wave goodbye • Proximal transitive: upper limb + tool o Brushing teeth • Distal intransitive: hand + no tool o Okay sign • Distal transitive: hand + tool o Turn screwdriver § TOLA is incomplete; does not assess how patient uses tools • Might be able to do when they have actual tools • But may help to assess patients’ daily living

• Theories of (ideomotor) apraxia o Disconnection hypothesis § Damage to white matter tracts connecting parietal areas (higher level area that formulates movements/ideas of task) to lower level motor areas (execute the task) § Left parietal cortex stores motor representations- engrams • If damaged affects motor performance o Can do the action automatically - engram is intact o But can’t access it for voluntary action? § Network involved in movement:

• § Damage to abstract knowledge of actions (parietal cortex) → conceptual/ideational apraxia § Damage to knowledge of action in sensorimotor form (link between parietal cortex and premotor cortex) → IMA § Damage to motor mechanisms (motor cortex) → Limb kinetic apraxia • Alien hand syndrome (AHS) o Hand acts of ‘own free will’ § Usually non-dominant hand § Interferes with other hand o Can occur after lesions to medial frontal lobe and corpus callosum § Large bundle of nerve fibres that connect the hemispheres • Sometimes surgically divided in those with severe epilepsy • AHS case studies o Parkin and Barry: MP § Aneurysm ruptured and damaged corpus callosum • “Tug of war” with own hands o Open draw with one hand, other closes it § Disrupt daily activities • E.g. when making an omelette left hand was ‘unhelpful’ (threw in uncracked eggs) o Bogen § One hand does up buttons, other alien hand undoes § Diagnostic apraxia: one hand acts in opposition of the other o Banks § Patient woke up to find hand choking them o Kumral § Unable to stop alien hand from grabbing nearby objects o Biran et al § Patient found eating difficult, alien hand opposed the normal one o Syndrome is varied - may be umbrella term for many different syndromes § Some depersonalise the hand • Call it “it”, give own personality • Some shout at it • Some show utilization behaviour o Archetti and Della Salla § Suggest term “alien hand” can cover many different clinical conditions • Propose term “anarchic hand syndrome” for patients with knowledge that limb is theirs o Biran et al: JC § Stroke damaged frontal areas and corpus callosum § Developed AHS, symptoms: • 1) Unresponsiveness o Alien hand wouldn’t carry out actions, even when given object for it • 2) Uncontrolled actions o Simple repetitive movements o Perseverate o Complex continuous actions § Movement initiated intentionally but unable to stop • E.g. keep pouring water o Intermanual conflict: alien hand interferes with the other hand o Can be unaware of hand’s actions § Esp with utilization behaviours • 3) Subjective reactions to hand o Attributes actions to right hand o Refers to hand as “it” o Biran et al: 3 components needed to produce AHS § 1) limb must be disinhibited • JC: alien hand faster to respond initially than normal hand o Does things on it’s own, can’t stop it • Can be disinhibited without AHS: motor tics § 2) movements must be purposeful • Not random - has a goal • Can be purposeful without being AHS § 3) patient must be aware of alien hand’s behaviour • Can give verbal reports of the hand’s action • Can detect when it made an error • Theories of AHS o Supplementary Motor Area plans complex movements § Left and right SMA both active when one hand is carrying out a task • Any planned movement must be suppressed in the ipsilateral (same side) hemisphere to active hand

• § In AHS the suppression fails • Alien hand will try to become involved in the task o Simple task that’s been completed by normal hand → Alien hand does “next best” action § Often opposite action to intended action • Open drawer/close drawer § Doesn’t explain utilization behaviour • Objects afford response → alien hand carried out o Other factors o SAS may fail to inhibit the action

Attention

• We need attention to “see” things o Notice what’s going on in our environment § Selective - or we’d be overwhelmed § Select what’s important o Perceptual info is entering our senses all the time § We need attention to filter out irrelevant info and to select important info for further processing o Change blindness: when there’s a mask between changes, we don’t notice it § Go through each detail slowly to see § Can occur with multiple and gradual changes • First studies of selective attention o Cherry - Cocktail party problem § How can you follow one convo among many others o Dichotic listening task: 2 different messages played through headphones to left and right ear § Told to attend to 1 (the attended message) • Better at reporting this message then unattended § Shadowing: repeat back the attended message § Difficult - regulate input and output § Poor info processing in unattended message • Typically unable to report o

Not detected in unattended Detected in unattended

Semantic meaning Pitch of voice (man/woman)

Language Bleep/tone played

Forwards or backwards ^ physical aspects helpful at separating the 2 playing messages

o Bottle neck in attention § First stimulus has to be attended to before you can process second • Can’t do both at once • Only so much info you can process at once o Limit in attention § Only so much we can process at once § Inattentional bias: focused on something else • Shows how limited our attention is o Focus on task → not enough left over to notice other • Broadbent’s early selection theory o When do we select stimuli for further processing? o Split span technique: given 3 number pairs, 1 number in each ear § Pairs: 9-4, 2-8, 6-1 • Left: 9 2 8 • Right: 4 8 1 § Reported back by ear • 928 and 481 • One ear is processed first o Process by physical aspect § Location

o § Can’t flow backwards, only forwards o 1) All stimuli enter sensory buffer in parallel o 2) Only 1 input passes through the filter - the other remains in the buffer for later § Decays quickly, don’t process = no memory o 3) Info passing through LCP becomes conscious

o § Early selection model • Prevents cognitive system/LCP becoming overloaded o Supporting evidence: Moray § Repeated set of words in unattended ear 35 times • Ppts could not recognise or remember these words o Attended → awareness/memory/semantics o Unattended→ held back in sensory buffer and decays o • Problems for Broadbent’s theory o Moray § Ppts recognised their own name from unattended message • Had undergone some processing

§ § Somehow got to LCP • Should only be held in buffer, or get to LCP through filter o Corteen and Wood § Conditioning phase • City name paired with electric shock § Second phase • Dichotic listening task and shadowing • City name elicits a GSR (galvanic skin resistance) 38% of the time o Claim they didn’t notice § Was some processing o Go against model § Related words also produced a GSR • Semantic analysis o Von Wright, Anderson, and Stenman § Found similar results as Corteen and Wood • GSR detected only on a fraction of trials o Processing of unattended message only occurs some of the time o Gray and Wedderburn § Broadbent claims we separate words by physical aspect • By ear § Left ear: mice 4 cheese § Right ear: 2 eat 6 § Report back: mice eat cheese • Grouped messages by meaning rather than ear o Part of unattended message reaches LCP

§ o Underwood § If given more practice ppts pick up more info from unattended ear • Detect digits o Naive: 8%, Practiced: 67% § Broadbent’s theory says the buffer/filter is rigid • Should not be affected by practice effects • Late selection theory: Deutsch and Deutsch o All stimuli analysed fully o Selection of a message is only determined at the end of processing § What to attend to

§ o Competing stimuli are weighted for importance o Most important stimulus is selected for response after full processing of all messages § Meaning, physical o Problems: § Rigid theory § Seems effortful to attend to all messages • Brain wants to save energy and not be wasteful • If every message was processed fully you’d assume more awareness of unattended message o Why couldn’t they report the repeated words? No memory of them (Moray) o The majority can’t process unattended info, only some gets through • Attenuation theory: Treisman o Filter is flexible o Info in unattended channel was attenuated/reduced § Only some gets through

§ o Evidence: § Pts shadow the left ear, at some point the message are switched • Ppts shadow a few words from message 1 after the switch o Content in unattended ear affects shadowing performance § Must be processed

§ § More dissimilar messages, better at shadowing • Voice, content, semantics, language o Unattended stimuli are analysed in a hierarchical fashion: § Physical cues, syllables, specific words → other words, grammar, meaning • If insufficient processing capacity, then don’t reach the top of the processing hierarchy • Why we often process physical but not latter o Physical analysed first § Meaning only processed if we have the capacity to do so • Depends on difficulty of task • Johnston and Heinz’s theory o More flexible model o Focus on task at hand and that impacts on how much of unattended message you can process o Selection can occur at several different stages of processing § Depends on demands of the task

§ • Take up lots of processing capacity, info in unattended doesn’t make it out of sensory buffer, no capacity to process further

§ • Unattended message may make it to filter

§ • Easy → LCP so you can process both and report both o Report on meaning o 1) More stages of processing unattended message can move through, the more effort it takes to do that task o 2) Selection of info for processing occurs as early as possible to save energy § No more processing than that needed for task at hand § Be efficient, save mental energy o About the filter § Can switch between early and late modes • Perceptual load theory: Lavie o Not about choosing early or late § Not about the filter, about attentional resources o Perceptual load = difficulty of the task § How much input there is, how difficult it makes it o Selection is neither early nor late § Depends on perceptual loads of task at hand, rather than when we are selecting information o Manipulated set size (number of items on screen) § Identify item as fast as possible (x or y) • Low perceptual load: only 1 item and a distractor item that’s congruent (the same, e.g. 2 Xs, easy to decide what target item is as same) or incongruent (different, e.g. x and y, makes it harder to identify target item as it differs and distracts)

o o Distractor had affect- made RTs slower § Distractor interference effect makes it harder to decide what target is • High perceptual load: item is in a list of others (much more effort, have to search for it), and distractor item that’s congruent or incongruent

o o Distractor had little/no effect o No capacity left over to process extra item § No extra attentional resources

o • Low load:

o § Can notice distractor, potential to be paid attention to and affect performance • High load:

o § Ignore distractor, don’t affect performance o Low load → easy → less attention needed § Attention available to pick up distractor § Longer RTs in incongruent trials due to distractor effect o High load → difficult → more attention needed § No attention available to pick up distractor § No effect of incongruent trials o Perceptual load theory: § Difficulty of task affects selection • Easy task → late selection o Go to LCP • Difficult task → early selection o Stop at buffer o About saving mental energy