An Integrative Framework of Time-Varying Affective Robotic Behavior
Total Page:16
File Type:pdf, Size:1020Kb
AN INTEGRATIVE FRAMEWORK OF TIME-VARYING AFFECTIVE ROBOTIC BEHAVIOR A Dissertation Presented to The Academic Faculty By Lilia V. Moshkina In Partial Fulfillment Of the Requirements for the Degree Doctor of Philosophy in Computer Science Georgia Institute of Technology May, 2011 Copyright © Lilia V. Moshkina 2011 AN INTEGRATIVE FRAMEWORK OF TIME-VARYING AFFECTIVE ROBOTIC BEHAVIOR Approved by: Dr. Ronald C. Arkin, Advisor Dr. Arthur D. Fisk College of Computing School of Psychology Georgia Institute of Technology Georgia Institute of Technology Dr. Tucker Balch Dr. Ayanna M. Howard College of Computing School of Electrical and Computer Georgia Institute of Technology Engineering College of Computing, Adjunct Georgia Institute of Technology Dr. Melody M. Jackson Date Approved: March 31, 2011 College of Computing Georgia Institute of Technology ACKNOWLEDGEMENTS This dissertation is a product of many years of work, and would not be possible without those who lent me their support, knowledge, and inspiration. First of all, my heartfelt gratitude goes to my husband, friend and colleague, Eric Martinson, for his unfailing encouragement, insightful ideas, hands-on help, and superior editing, not to mention taking excellent care of our young daughter Maya during the times I had to be away in order to conduct my experiments. My academic advisor, Professor Ron Arkin, has been instrumental in bringing this research to completion, and I am very grateful for his direction, support, both intellectual and financial, and his almost infinite patience. I would also like to thank my dissertation committee for their guidance in a number of related domains and their invaluable comments. My special thanks go to Professor Melody Jackson, who has been a source of encouragement for many years, and who inspired me to enter into the Ph.D. program at GA Tech in the first place. I would also like to thank Samsung Electronics Corporation for providing a two-year grant for implementation and testing of the TAME framework, as well as Hyunryong Jung and Chien-Ming Huang who were supported by this grant. I am especially grateful to Sunghyun Park, whose unflagging help and support went above and beyond duty and have been indispensable in completion of this dissertation. Finally, I would like to express my deep gratitude to my friends and lab mates. My dear friend Natalia Dyomina has always been there for me, whether I needed encouragement, fresh ideas or statistics help, and my early conversations with her provided the inspiration for the theme of my research. My friends Ernest Merrill, Mario Romero and Sergey Demin were willing participants in all my experiments, and gave iii much appreciated moral support and excellent advice. Last but not least, my lab mates from Mobile Robot Lab: Zsolt Kira, Alex Stoytchev, Yoichiro Endo, Will Wagstaff and Brian Lee could be counted on for deciphering software and hardware intricacies, and making the lab a livelier place. iv TABLE OF CONTENTS TABLE OF CONTENTS ACKNOWLEDGEMENTS ……….………………………………………………..… iii LIST OF TABLES ………...……………………….………………………………….. ix LIST OF FIGURES …………………………………………………………………. xii SUMMARY ………..………………………………………………………………… xviii 1 INTRODUCTION ……………………………………………………………… 1 1.1 Terminology ……………………………………………………………….. 3 1.2 Research Question ………………………………………………………. 4 1.3 Expected Contributions ………………………………………………….. 5 1.4 Dissertation Overview ……………………………………………………. 6 2 RELATED WORK ……………………………………………………………... 7 2.1 Socially Interactive Affective Robots ………………………………….. 10 2.2 Evaluating Socially Interactive Robots ……………………………….. 17 2.3 Affect for Enhancing Robotic Behavior ………………………………. 21 2.4 Affect in Virtual Agents …………………………………………………. 24 2.5 Summary …………………………………………………………………. 28 3 ARCHITECTURAL FRAMEWORK ………………………………………… 30 3.1 TAME Framework Overview ………………………………………….... 31 3.2 Personality Traits ………………………………………………………... 34 3.3 Emotions …………………………………………………………………. 50 3.4 Moods …………………………………………………………………….. 84 3.5 Affective Attitudes ……………………………………………………….. 99 v TABLE OF CONTENTS 3.6 Chapter 3 Summary …………………………………………………… 121 4 EXPLORATORY EXPERIMENTAL STUDY …………………………….. 124 4.1 Implementation ………………………………………………………… 124 4.2 Study Design and Analysis …………………………………………... 128 4.3 Summary and Lessons Learned …………………………………….. 150 5 SOFTWARE ARCHITECTURE AND IMPLEMENTATION ……………. 156 5.1 Software Architecture Design ………………………………………... 156 5.2 Implementation ………………………………………………………… 175 5.3 Online Survey on Recognition of Affective Robotic Behavior …… 188 5.4 Summary ……………………………………………………………….. 199 6 EVALUATING ROBOT AFFECT IN HRI EXPERIMENTS ……………… 200 6.1 Evaluating Expressions of Negative Mood and Fear in a Search-and- Rescue Scenario ……………………………………………………..… 201 6.2 Evaluating Expressions of Extraversion and Introversion in a Robot as a Guide Scenario ……………………………………………………… 235 6.3 Summary ……………………………………………………………….. 274 7 MEASURING SOCIAL RESPONSE TO AFFECTIVE ROBOTS ……... 278 7.1 HRI Measures Overview ……………………………………………… 278 7.2 HRI Metrics Employed in TAME Experiments ……………………… 284 7.3 Summary and Recommendations …………………………………… 306 8 CONCLUSION ……………………………………………………………… 311 8.1 Research Questions …………………………………………………… 312 8.2 Contributions …………………………………………………………… 323 APPENDIX A: CONSENT FORM FOR THE AIBO EXPLORATORY EXPERIMENTAL STUDY …………………………………………………………. 325 vi TABLE OF CONTENTS APPENDIX B: SCRIPT FOR EXPERIMENT ADMINISTRATORS ………….... 327 APPENDIX C: DEMOGRAPHICS QUESTIONNAIRE …………………………. 333 APPENDIX D: POST QUESTIONNAIRE FOR AIBO STUDY ………………… 335 APPENDIX E: A BRIEF BIG FIVE PERSONALITY QUESTIONNAIRE FOR PARTICIPANT ……………………………………………………………………… 337 APPENDIX F: A BRIEF BIG FIVE PERSONALITY QUESTIONNAIRE FOR ROBOTIC DOG .……………………………………………………………….…… 339 APPENDIX G: BRIEF MEASURE OF POSITIVE AND NEGATIVE AFFECT (MOOD) ………………………………………………………………………….…… 341 APPENDIX H: FREE-RESPONSE QUESTIONNAIRE SUMMARY ……….… 343 APPENDIX I: CODING INSTRUCTIONS FOR FREE-RESPONSE QUESTIONNAIRE GIVEN TO INDEPENDENT CODERS ……………………. 348 APPENDIX J: SAMPLE AFFECT CONFIGURATION FILE …………………… 350 APPENDIX K: SAMPLE STIMULI CONFIGURATION FILE …………………… 354 APPENDIX L: PSEUDOCODE FOR CHAPTER 5: SOFTWARE ARCHITECTURE AND IMPLEMENTATION ………………………………….… 356 APPENDIX O: CONSENT FORM FOR SEARCH-AND-RESCUE EXPERIMENT ………………………………………………………………………………………… 384 APPENDIX P: VIDEO RELEASE FORM FOR NAO EXPERIMENTS ……….. 387 APPENDIX Q: EXPERIMENTER SCRIPT FOR SEARCH-AND-RESCUE EXPERIMENT …………………………………………………………………….… 388 APPENDIX R: NAO’S SPEECH DURING THE SEARCH-AND-RESCUE TOUR ………………………………………………………………………………………… 390 APPENDIX S: PANAS-T WITH REGARDS TO NAO ROBOT FOR SEARCH- AND-RESCUE EXPERIMENT ………………………………………………….…. 391 APPENDIX T: POST QUESTIONNAIRE FOR SEARCH-AND-RESCUE EXPERIMENT ………………………………………………………………………. 393 APPENDIX U: DEMOGRAPHICS QUESTIONNAIRE FOR SEARCH-AND- RESCUE STUDY …………………………………………………………………… 396 vii TABLE OF CONTENTS APPENDIX V: SEMI-STRUCTURED INTERVIEW FOR SEARCH-AND-RESCUE STUDY ……………………………………….……………………………………… 397 APPENDIX W: FLYER FOR SEARCH-AND-RESCUE STUDY ……………... 399 APPENDIX X: DESCRIPTIVE STATISTICS FOR SEMANTIC DIFFERENTIAL SCALES, SEARCH-AND-RESCUE EXPERIMENT ……………………………. 400 APPENDIX Y: CONSENT FORM FOR EXTRAVERSION EXPERIMENT ….. 404 APPENDIX Z: MEMORY RETENTION TASK ………………………………… 407 APPENDIX AA: MATH PROBLEM ……………………………………………….. 415 APPENDIX BB: ROBOT’S GREETING FOR EXTRAVERSION EXPERIMENT ……...…………………………………………………………………………………. 418 APPENDIX CC: POST-QUIZ QUESTIONNAIRE ………………………………. 419 APPENDIX DD: POST-MATH QUESTIONNAIRE …………………………...… 421 APPENDIX EE: NASA TLX SCALE ……………………………………….………423 APPENDIX FF: ROBOT PERSONALITY QUESTIONNAIRE, TRAIT EXPERIMENT ………………………………………………………………………. 426 APPENDIX GG: POST-QUESTIONNAIRE FOR TRAIT EXPERIMENT …….. 429 APPENDIX HH: DEMOGRAPHICS QUESTIONNAIRE FOR TRAIT EXPERIMENT ……………………………………………………………………… 430 APPENDIX II: RELIABILITY AND DESCRIPTIVE STATISTICS FOR SEMANTIC DIFFERENTIAL SCALES (SCALE AND SUBSCALE LEVEL) ……………….. 431 9 REFERENCES …………………………………………………………….. 436 viii LIST OF TABLES List of Tables Table 1 : Robotic Systems with Multiple Affective Phenomena ......................................14 Table 2: Overview of Multi-Affective Systems ...............................................................28 Table 3 : Description and Facets of Big Five Personality Dimensions ............................36 Table 4: Examples of Traits Applied to Human-Robot Interaction .................................39 Table 5 : Means and Standard Deviations for Personality Scores for Adults ..................41 Table 6: Representative definitions of Emotions ...........................................................51 Table 7: Functions and Core Relational Themes for Select Emotions ...........................54 Table 8: Examples of applying emotions to Human-Robot Interaction ..........................59 Table 9: A sample mood/activation point dependency matrix ........................................75 Table 10 : Comparative View of Mood Levels ................................................................86 Table 11: Summary of time-varying aspects of the TAME components ....................... 123 Table 12: Normal distribution statistics for FFM personality scores ............................. 158 Table 13: A sample personality configuration for a highly agreeable and slightly neurotic robot ............................................................................................................................ 159 Table 14: Sample Personality/Emotion Matrix specifying the influence traits have