Doctoral School in Cognitive and Education Science XXII cicle Assessing audiotactile interactions: Spatiotemporal factors and role of visual experience Thesis submitted for the degree of Doctor of Philosophy by Valeria Occelli Supervisor: Prof. Massimiliano Zampini Academic Year 2009/2010 Audiotactile interactions: Spatiotemporal factors and role of visual experience Table of Contents Table of Contents.............................................................................................................................................................II List of Figures .................................................................................................................................................................. V List of Tables................................................................................................................................................................. VII Thesis Overview................................................................................................................................................................1 PART 1: Theoretical background ................................................................................................................................... 3 Chapter 1. ......................................................................................................................................................................3 Multisensory integration and audiotactile interactions: an overview ...........................................................................3 1.1. Introduction........................................................................................................................................................3 1.2. Early studies on crossmodal interactions between hearing and touch................................................................5 Chapter 2. ......................................................................................................................................................................9 Audiotactile interactions in low-level behavioural tasks ...............................................................................................9 2.1. Introduction........................................................................................................................................................9 2.2. Detection/discrimination performance ...............................................................................................................9 2.3. Intensity............................................................................................................................................................ 12 2.4. Texture ............................................................................................................................................................. 14 2.5. Sensory illusions between hearing and touch................................................................................................... 17 2.6. Synaesthetic and pseudosynaesthetic correspondences.................................................................................... 20 Chapter 3. .................................................................................................................................................................... 23 Audiotactile interactions in the temporal domain........................................................................................................23 3.1. Introduction...................................................................................................................................................... 23 3.2. Temporal resolution, synchrony and temporal order between hearing and touch............................................ 23 3.3. Numerosity....................................................................................................................................................... 28 3.4. Interactions based on frequency similarity....................................................................................................... 31 Chapter 4. .................................................................................................................................................................... 39 Audiotactile interactions in the spatial domain ........................................................................................................... 39 4.1. Introduction...................................................................................................................................................... 39 4.2. Audiotactile crossmodal effects on stimulus localization ................................................................................ 39 4.3. The specificity of the space stimulated: Behavioural evidence........................................................................ 44 4.4. Audiotactile integration in the peripersonal space: Neuroimaging studies ...................................................... 49 4.5. Audiotactile spatial attention: Behavioural studies .......................................................................................... 52 4.6. Audiotactile spatial attention: Neuroimaging studies....................................................................................... 60 4.7. Crossmodal interactions in the perception of motion....................................................................................... 65 4.8. Audiotactile spatial interactions in neurological populations........................................................................... 69 Chapter 5. .................................................................................................................................................................... 73 Neural substrates of audiotactile interactions ............................................................................................................. 73 5.1. Introduction...................................................................................................................................................... 73 5.2. General features of multisensory integration at neuronal level ........................................................................ 74 5.3. Multisensory areas ........................................................................................................................................... 76 5.4. Areas subserving audiotactile integration in monkeys ..................................................................................... 78 5.5. Areas subserving audiotactile integration in humans ....................................................................................... 83 Chapter 6. .................................................................................................................................................................... 93 Audiotactile interactions and visual experience .......................................................................................................... 93 6.1. Introduction...................................................................................................................................................... 93 6.2. Spatial aspects .................................................................................................................................................. 95 6.3. Attention........................................................................................................................................................... 99 6.4. Temporal aspects............................................................................................................................................ 102 II Audiotactile interactions: Spatiotemporal factors and role of visual experience 6.5. Neural substrates ............................................................................................................................................ 104 PART II: Experimental Studies .................................................................................................................................. 109 Chapter 7. .................................................................................................................................................................. 109 The sensory dominance between touch and audition................................................................................................. 109 7.1. Introduction.................................................................................................................................................... 109 7.2. Experiment 1. The audiotactile Colavita effect in the frontal space............................................................... 110 7.2.1. Method ................................................................................................................................................... 112 7.2.2. Results .................................................................................................................................................... 114 7.3. Experiment 2. The audiotactile Colavita effect: The effect of spatial factors and the part of body stimulated ............................................................................................................................................................................... 118 7.3.1. Method ..................................................................................................................................................