MODALITY-SPECIFIC SEMANTIC PROCESSING OF ACTION CONCEPTS: AN EYETRACKING INVESTIGATION By CHING-I HUNG A DISSERTATION PRESENTED TO THE GRADUATE SCHOOL OF THE UNIVERSITY OF FLORIDA IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF DOCTOR OF PHILOSOPHY UNIVERSITY OF FLORIDA 2015 © 2015 Ching-I Hung To my family ACKNOWLEDGMENTS I thank my chair and doctoral supervisor Dr. Jamie Reilly for his guidance. He has given me many opportunities to grow independently. It is through him that I realize what a research scientist is and learn to push my limits. I am very grateful to my committee members for their patience and feedback, Dr. Lori Altmann for her advice during my doctoral program, and the participants for their flexible schedule in the study. I wish to acknowledge my colleagues, members in the Cognition and Language Lab at the University of Florida, and members in the Memory, Concepts, Cognition Lab at Temple University. Life at UF would have been tough without my doctoral colleagues Hyejin, Audrey, Jonathan, Supraja, Nour, Lisa and Jimena. I appreciate the valuable time we spent together. It motivates me to be better both academically and personally. I also appreciate members at Temple University for their warm welcome as I relocated to a new environment. I would not have gone this far without their supports. Above all, I thank my family for their unconditional love and my mom for reminding me to stay faithful. I especially thank my best friend, Hao-yun for the incredible support when the tide was high in this journey. 4 TABLE OF CONTENTS page ACKNOWLEDGMENTS ...............................................................................................................4 LIST OF TABLES ...........................................................................................................................7 LIST OF FIGURES .........................................................................................................................8 ABSTRACT .....................................................................................................................................9 CHAPTER 1 INTRODUCTORY OVERVIEW ..........................................................................................11 2 LITERATURE REVIEW .......................................................................................................15 Semantic Processing – Access to Meanings ...........................................................................15 Action Concepts ......................................................................................................................22 Actions vs. Objects ..........................................................................................................23 Action Understanding ......................................................................................................26 Verbal and Visual Access to Action Concepts ................................................................29 Working Hypotheses ..............................................................................................................32 3 STIMULUS STANDARDIZATION .....................................................................................33 Overview .................................................................................................................................33 Single Word Norming .............................................................................................................33 Participants ......................................................................................................................33 Single Word Lexical Norming ........................................................................................34 Single Word Visual Norming ..........................................................................................36 Summary of Single Word Norming ................................................................................37 Action Pair Selection ..............................................................................................................38 Participants ......................................................................................................................38 Pair Selection ...................................................................................................................38 4 ACTION SEMANTIC JUDGMENT TASK ..........................................................................41 Overview .................................................................................................................................41 Participants .............................................................................................................................41 General Procedure ..................................................................................................................42 Semantic Task and Procedure .........................................................................................42 Motor Familiarization Task and Semantic-Motor Control Task .....................................44 Motor Task and Procedure ..............................................................................................45 Eyetracking Recording and Apparatus ...................................................................................45 Data Design and Analysis .......................................................................................................46 Behavioral Analysis .........................................................................................................46 5 Eyetracking Analysis .......................................................................................................47 Research Questions Remarks and Predictions ........................................................................48 5 RESULTS ...............................................................................................................................50 Data Trimming ........................................................................................................................50 Behavioral Results .................................................................................................................50 Response Accuracy .........................................................................................................51 Reaction Time .................................................................................................................52 Motor Rates .....................................................................................................................53 Behavioral Summary .......................................................................................................54 Interim Discussion of the Behavioral Results .................................................................56 Eyetracking Results ................................................................................................................59 Fixation Count .................................................................................................................59 Revisits ............................................................................................................................59 Fixation Duration .............................................................................................................60 Eyetracking Summary .....................................................................................................61 Interim Discussion of the Eyetracking Results ...............................................................63 Secondary Analysis ................................................................................................................64 Discussion of the Experimental Findings ...............................................................................65 6 GENERAL DISCUSSION .....................................................................................................81 Direct vs. Indirect Semantic Processing .................................................................................81 Motor Experience during Semantic Processing ......................................................................83 Action Semantic Processing – A Summary ............................................................................85 Limitations and Future Directions ..........................................................................................87 APPENDIX A INSTRUCTIONS FOR PSYCHOLINGUISTIC RATINGS .................................................91 B ACTION STIMULI IN SETS 1 AND 2 .................................................................................93 C NON-ACTION STIMULI ......................................................................................................95 LIST OF REFERENCES ...............................................................................................................96 BIOGRAPHICAL SKETCH .......................................................................................................106 6 LIST OF TABLES Table page 3-1 Mean characteristics of action stimuli ...............................................................................40 3-2 Mean and standard deviation of the 7-point Likert scale for the pair-relation survey .......40 5-1 Summary of F statistics for the ANOVA results examining response accuracy in action and object concepts via subject (F1) and item (F2) analysis ...................................68