Multisensory Integration - Examples I. Posture – maintain center of gravity above the feet A) Vestibular B) Visual C) Proprioception – ankle sway McCollum et al. J Theor Biol 180: 257-270, 1996 D) Other Proprioceptive Cues – arm posture Rabin, E. et al. J Neurophysiol 82: 3541-3549 1999 1) Fingertip contact attenuates sway II. Ilusions A) Proprioception – vibration produces illusion of muscle stretching 1) Example – vibrate biceps, feel arm extend 2) Pinocchio effect 3) Multimodal effects a) Overrides vision B) Weight and Pressure Cues Lackner and DiZio, Trends Cogn Sci 4:279-288, 2000 C) But vision can also override proprioception Lackner and DiZio, Trends Cogn Sci 4:279-288, 2000 III. Sensing Heading – Neural Correlates in MST A) Optic Flow 1) The focus of expansion (FOE) defines heading and it depends on gaze direction 2) Neurons in MST are tuned to optic flow and the locus of the FOE Page and Duffy, J Neurophysiol 81: 596-610, 1999 B) Influence of motion cues on MST neuron activity 1A) Monkey rides a cart (CW or CCW) and views lights on a wall 1B & 1C) – a heading selective neuron 1D) But not all neurons are heading selective – some encode place and some responses depend on the path taken Froehler and Duffy, Science 295: 2462-2465, 2002 “Heading” Cell “Place” Cell Froehler and Duffy, Science 295: 2462-2465, 2002 C) Vestibular contributions to MST activity 1) The task – discriminate heading 2) Behavior Intact Vestibulectomy Motion cues Gu et al. Nature Neurosci 19: 1038- Optic flow 1047, 2007 3) MST neurons respond to vestibular only cues (without optic flow) Gu et al. Nature Neurosci 19: 1038-1047, 2007 a) A sample neuron b) Tuning to optic flow (•), vestibular aligned on best direction (•) or on visual (•) c) Same, after vestibular lesion Some MST neurons have same tuning to optic flow and vestibular, others have tuning 180o out of phase D) MST neurons encode optic flow in space rather than the motion on the retina: pursuit stimulus (see eye velocity trace) to right or left optic flow stimulus for 200 ms but MT neurons encode retinal image velocity E) Summary – MST neurons respond to: optic flow vestibular signals extraretinal signals (efference copy) Inaba et al. J Neurophysiol 97: 3463-3483, 2007 IV The weightings given to different sensory modalities depends on how reliable they are As vision becomes progressively less reliable, it is weighted less and tactile cues are weighted more Ernst and Banks, Nature 415: 429-433, 2002.