Simulating Speed in Language: Contributions from Vision, Audition and Action
Total Page:16
File Type:pdf, Size:1020Kb
Simulating speed in language: contributions from vision, audition and action. Laura Jane Speed 2014 Department of Experimental Psychology University College London A thesis submitted for the degree of Doctor of Philosophy 1 Declaration I, Laura Jane Speed, confirm that the work presented in this thesis is my own. Where information has been derived from other sources, I confirm that this has been indicated in the thesis. Signature 2 Abstract Embodied theories propose that understanding meaning in language requires the mental simulation of entities being referred to. These mental simulations would make use of the same modality-specific systems involved in perceiving and acting upon such entities in the world, grounding language in the real world. However, embodied theories are currently underspecified in terms of how much information from an event is contained in mental simulations, and what features of experience are included. The thesis addresses comprehension of language that describes speed of events. Investigating speed allows embodied theories to be extended to a more complex feature of events. Further, speed is a fine-grained feature and thus testing an embodied theory of speed will reveal whether or not mental simulations include the fine details of real-world experience. Within the thesis four main methods of investigation were used, assessing simulation of speed with different types of speed language under different conditions: behavioural testing combining speed in language with speed in perception and action, eye-tracking investigating whether eye-movements to a visual scene are affected by speed in sentences, a psychophysics paradigm assessing whether speed in language affects visual perception processes, and finally, as a crucial test of embodiment, whether or not Parkinson’s patients, who have difficulty moving speedily, also have problems with comprehension of speed language. The main findings of the thesis are that: (1) speed, a fine-grained and abstract dimension, is simulated during comprehension, (2) simulations are dynamic and context-dependent, and (3) simulations of speed are specific to biological motion and can encode specific effectors used in an action. These results help to specify current embodied theories in terms of what the nature of simulations are and what factors they are sensitive to, in addition to broadly 3 providing support for the sharing of cognitive/neural processes between language, action and perception. 4 Acknowledgments I would firstly like to thank my supervisor Gabriella Vigliocco for accompanying me on my PhD journey. You have given me great advice and been a strong role model to me. Special thanks to David Vinson for always being so helpful and particularly for guiding me through the first data analysis of my PhD. I would also like to thank the rest of the Language and Cognition lab for their support and exciting discussions in lab meetings. Thank you to Aurelio Bruno for the collaboration and introduction to psychophysics. Also thanks to my collaborators in South Carolina: Rutvik Desai, Wessel van Dam and Carl Brzorad. I learnt so much during such a short visit and had lots of fun at the same time. Thank you to Chris Street and Julia Florentine for chapter comments and for being great colleagues and friends throughout my PhD. I am also very grateful for the student assistants that have helped to collect data over the years and of course the many eager volunteers from the participant pool. Importantly, I’d like to thank my family for being my number one motivation and my friends for making me happy. Thank you to mum for telling me nearly everyday how proud you are of me. Finally, thank you to Gary for supporting me, making me laugh and providing so many exotic locations to work from. 5 Table of contents Chapter 1 Introduction ............................................................................................ 20 1.1 The overarching debate: symbolic vs. embodied approaches to cognitive science .................................................................................................................... 21 1.2 The symbolic vs. embodied debate applied to semantic representations ...... 24 1.3 The Theoretical Debate ................................................................................. 30 1.4 Theories and hypotheses of embodied language processing ......................... 45 1.5 Assessing embodiment .................................................................................. 50 1.6 Chapter Summary ......................................................................................... 53 Chapter 2 Literature Review ................................................................................... 55 2.1 Aims .............................................................................................................. 55 2.2 Behavioural evidence .................................................................................... 57 2.3 Functional magnetic resonance imaging (fMRI) .......................................... 65 2.4 Electroencephalography (EEG) .................................................................... 75 2.5 TMS .............................................................................................................. 77 2.6 Patient Data ................................................................................................... 81 2.7 Summary ....................................................................................................... 84 6 Chapter 3 The Present Thesis .................................................................................. 86 3.1 Time, space and speed in language ............................................................... 87 3.2 Speed in the brain .......................................................................................... 92 3.3 Description of experimental chapters ........................................................... 95 3.4 Conclusion .................................................................................................... 99 Chapter 4 Perceptual speed and lexical decision .................................................. 100 4.1 Multimodal simulation ................................................................................ 101 4.2 The role of perceptual and verbal modality ................................................ 103 4.3 Experiment 4-1: No perceptual stimuli ....................................................... 105 4.4 Experiment 4-2: Visual speed (horizontal) and visual words ..................... 108 4.5 Experiment 4-3: Auditory speed (beeps) and visual words ........................ 111 4.6 Experiment 4-4: Visual speed (vection) and visual words .......................... 114 4.7 Experiment 4-5: Auditory speed (white noise) and visual words ............... 117 4.8 Experiment 4-6: Visual speed (vection) and auditory words ...................... 119 4.9 Experiment 4-7: Auditory speed (white noise) and auditory words ........... 121 4.10 Experiment 4-8: Auditory speed (footsteps) and auditory words ............. 123 4.11 Experiment 4-9: Auditory speed (footsteps) and visual words ................. 126 7 4.12 General Discussion ................................................................................... 127 4.13 Chapter conclusion .................................................................................... 139 Chapter 5 Do speed verbs affect speed discrimination? ....................................... 140 5.1 Experiment 5-1: Fast, slow and neutral verbs ............................................. 145 5.2 Experiment 5-2: Fast and slow verbs only .................................................. 156 5.3 Experiment 5-3: Replicating Experiment 5-2 ............................................. 159 5.4 General Discussion ..................................................................................... 161 5.5 Chapter conclusion ...................................................................................... 166 Chapter 6 The influence of speeded actions and perceptual speed on comprehending sentences about fast and slow events ............................................. 167 6.1 Experiment 6-1: Manipulating Auditory Speed with Footsteps ................. 170 6.2 Experiment 6-2: Manipulating Speed of Physical Action .......................... 195 6.3 General Discussion ..................................................................................... 216 6.4 Chapter conclusion ...................................................................................... 220 Chapter 7 Eye-movements and the mental simulation of sentence speed ............ 221 7.1 Experiment 7-1 ............................................................................................ 224 7.2 Experiment 7-2 ............................................................................................ 231 8 7.3 Experiment 7-3 ............................................................................................ 237 7.4 Experiment 7-4 ............................................................................................ 242 7.5 General Discussion ..................................................................................... 248 7.6 Chapter conclusion ...................................................................................... 252 Chapter 8 Is speed processing in language affected by deficits in the motor system? 253 8.1 Method .......................................................................................................