Multisensory Integration - Examples

Home , Multisensory integration

I. Posture – maintain center of gravity above the feet

A) Vestibular

B) Visual

C) Proprioception – ankle sway

McCollum et al. J Theor Biol 180: 257-270, 1996 D) Other Proprioceptive Cues – arm posture

Rabin, E. et al. J Neurophysiol 82: 3541-3549 1999

1) Fingertip contact attenuates sway II. Ilusions

A) Proprioception – vibration produces illusion of muscle stretching

1) Example – vibrate biceps, feel arm extend

2) Pinocchio effect

3) Multimodal effects

a) Overrides vision

B) Weight and Pressure Cues

Lackner and DiZio, Trends Cogn Sci 4:279-288, 2000 C) But vision can also override proprioception

Lackner and DiZio, Trends Cogn Sci 4:279-288, 2000 III. Sensing Heading – Neural Correlates in MST

A) Optic Flow

1) The focus of expansion (FOE) defines heading and it depends on gaze direction

2) Neurons in MST are tuned to optic flow and the locus of the FOE

Page and Duffy, J Neurophysiol 81: 596-610, 1999 B) Influence of motion cues on MST neuron activity 1A) Monkey rides a cart (CW or CCW) and views lights on a wall 1B & 1C) – a heading selective neuron 1D) But not all neurons are heading selective – some encode place and some responses depend on the path taken

Froehler and Duffy, Science 295: 2462-2465, 2002 “Heading” Cell

“Place” Cell

Froehler and Duffy, Science 295: 2462-2465, 2002 C) Vestibular contributions to MST activity 1) The task – discriminate heading 2) Behavior Intact

Vestibulectomy

Motion cues

Gu et al. Nature Neurosci 19: 1038- Optic flow 1047, 2007 3) MST neurons respond to vestibular only cues (without optic flow)

Gu et al. Nature Neurosci 19: 1038-1047, 2007

a) A sample neuron b) Tuning to optic flow (•), vestibular aligned on best direction (•) or on visual (•) c) Same, after vestibular lesion

Some MST neurons have same tuning to optic flow and vestibular, others have tuning 180o out of phase D) MST neurons encode optic flow in space rather than the motion on the retina:

pursuit stimulus (see eye velocity trace) to right or left

optic flow stimulus for 200 ms

but MT neurons encode retinal image velocity

E) Summary – MST neurons respond to:

optic flow vestibular signals extraretinal signals (efference copy)

Inaba et al. J Neurophysiol 97: 3463-3483, 2007 IV The weightings given to different sensory modalities depends on how reliable they are

As vision becomes progressively less reliable, it is weighted less and tactile cues are weighted more

Ernst and Banks, Nature 415: 429-433, 2002