Effects of Virtual Humans' Facial Emotional Displays on Persuasion
Total Page:16
File Type:pdf, Size:1020Kb
Rochester Institute of Technology RIT Scholar Works Theses 6-15-2015 Effects of Virtual Humans’ Facial Emotional Displays on Persuasion Yuqiong Wang [email protected] Follow this and additional works at: https://scholarworks.rit.edu/theses Recommended Citation Wang, Yuqiong, "Effects of Virtual Humans’ Facial Emotional Displays on Persuasion" (2015). Thesis. Rochester Institute of Technology. Accessed from This Dissertation is brought to you for free and open access by RIT Scholar Works. It has been accepted for inclusion in Theses by an authorized administrator of RIT Scholar Works. For more information, please contact [email protected]. Effects of Virtual Humans’ Facial Emotional Displays on Persuasion by Yuqiong Wang A dissertation submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy in Computing and Information Sciences B. Thomas Golisano College of Computing and Information Sciences Rochester Institute of Technology Rochester, New York June 15, 2015 Effects of Virtual Humans’ Facial Emotional Displays on Persuasion by Yuqiong Wang Committee Approval: We, the undesignated committee members, certify that we have advised and/or supervised the candidate on the work described in this dissertation. We further certify that we have reviewed the dissertation manuscript and approve it in partial fulfillment of the requirements of the degree of Doctor of Philosophy in Computing and Information Sciences. Dr. Joe Geigel, Thesis Advisor Department of Computer Science, Rochester Institute of Technology Dr. Reynold Bailey, Committee Member Department of Computer Science, Rochester Institute of Technology Dr. Jonathan Gratch, Committee Member Institute for Creative Technologies, University of Southern California Dr. Andrew Herbert, Committee Member Department of Psychology, Rochester Institute of Technology Dr. Evelyn Rozanski, Committee Member Department of Information Sciences & Technologies, Rochester Institute of Technology Dr. Pengcheng Shi, Dissertation Defense Chairman Computing and Information Sciences, Rochester Institute of Technology Certified by: Dr. Pengcheng Shi, Director Computing and Information Sciences, Rochester Institute of Technology iv Acknowledgments Seven years ago, I came across a research paper in ACM SIGGRAPH. The paper talked about collaborations between artists and technolo- gists. I was intrigued. The first author was Dr. Joe Geigel, a professor at the Rochester Institute of Technology in Rochester NY, the United States. I traveled from Wuhan, China to Rochester, U.S.. I knocked on Dr. Joe Geigel’s door, told him (in broken English) that I read his paper, and asked whether he could be my Ph.D. advisor. To my great delight, he understood my English, and said yes. I consider myself extremely lucky to have found the most caring and supportive advisor. Through my (lengthy) graduate career, Dr. Joe Geigel has given me invaluable care, guidance and support. Dr. Joe Geigel not only taught me how to be a researcher, but also helped me grow as a person. He has my best interest at heart. Words aren’t enough to describe my gratitude... but I will still try. Thank you Joe for everything! One more thank-you for the Hello Kitty guitar! v I would also like to express my deep gratitude to Dr. Jonathan Gratch1, my dissertation committee member and supervisor at the Uni- versity of Southern California, Institute for Creative Technologies. Dr. Jonathan Gratch gave me the opportunity to escape from snow. He and his talented team (Thank you Dr. Gale Lucas and Dr. Peter Khooshabeh!) gave me many great inspirations, and kindly helped me improve my research skills. I could never have accomplished this dis- sertation without them. I am very grateful. Thank you Jon, Gale, and Pete! A thousand thanks from the bottom of my heart to Dr. Andrew Herbert, my dissertation committee member, for introducing me to Psychology, for being a role model, and for greatly improving the qual- ify of my work. My initial dissertation proposal was mediocre at best, yet Dr. Andrew Herbert enlightened me and patiently guided me. Dr. Andrew Herbert demonstrated how to be a real researcher. I was hum- bled many times by his knowledge, insight, and dedication. I was able to improve this dissertation thanks to all the constructive comments from him. Thank you Andy! I am very grateful to Dr. Evelyn Rozanski, my dissertation com- mittee member, for supporting me since Day 1, and for serving on my 1It turns out that my two supervisors, Dr. Joe Geigel and Dr. Jonathan Gratch, have the exact same initial (JMG). Future work is needed to determine the probability of a Ph.D. student having two committee members with the same first, middle, and last initials. Thank you JMG. vi committee after retirement (sorry it took so long). Thank you for of- fering a great course in the very first quarter of my Ph.D., and thank you for all the heart-warming conversations at your office. Thank you Evie! Many thanks to Dr. Reynold Bailey, my dissertation committee member, for providing valuable comments for my dissertation, and for making the lab a fun place to stay. Thank you Rennie! I would also like to thank Dr. Pengcheng Shi, director of the Com- puting and Information Sciences Ph.D. program, for letting me join the Ph.D. program, for being an excellent director, and for serving as my defense chair at the last moment. Thank you for everything you have done for all the Ph.D. students. Thank you for inviting us (homeless) international students to your house. You have no idea how much that meant to us. Thank you Pengcheng! I would also like to extend my thanks to my dear friends Hongmei Yuan, Emilie Zhao, Yuyu Xu, Chris Acheson, Mustafa Jaber, David Oleson, and Howard Chen for believing in me, standing by me, and investing in me. Thank you for being in my life. You are the treasures of my life! I wish to apologize to my parents for upsetting them so many times. Sorry for being stubborn, but I need to make my own decisions. I miss you a lot, and I love you more than you can imagine. I hope I can make you proud one day. vii Finally, thanks to all the poker players who had financially supported me during my Ph.D.. viii Abstract E↵ects of Virtual Humans’ Facial Emotional Displays on Persuasion Yuqiong Wang Supervising Professor: Joe Geigel This dissertation explores the e↵ect of virtual humans’ facial emo- tional displays in the context of persuasion. In a collaborative problem- solving game, participants received persuasive information from a vir- tual teammate. The first study demonstrated that a subservient vir- tual teammate’s facial emotional displays reduced his or her persua- sive capacity. The second study revealed that the e↵ect of a virtual human’s facial emotional displays was jointly determined by whether the observer was in power, and/or whether the observer considered it appropriate to express emotions. Emotional expressions undermined persuasion when the observer overpowered the virtual human, and/or when the observer perceived emotional expressions to be inappropriate. When both conditions were satisfied, emotional expressions reduced persuasion the most. The third study suggested that the amount of anger expressions predicted the outcome of persuasion. Photorealism, while on one hand enhanced the perception of anger expressions, did not moderate the e↵ect of emotional expressions. Findings from this ix study inform theories of emotional expressions in persuasion, and guide the design of persuasive virtual humans. x Contents Acknowledgments ......................... iv Abstract ...............................viii 1 Introduction ........................... 1 1.1 Emotion as social information . 2 1.2 Emotional expression and persuasion . 5 1.3 Virtual humans’ emotional expression and persuasion . 8 1.4 Facial expressions as a means to express emotions . 11 1.5 Research Questions . 14 1.5.1 Do virtual humans’ facial displays of emotion have an e↵ect on persuasion? . 14 1.5.2 What factors moderate the e↵ect of virtual hu- mans’ facial emotional displays in persuasion? . 15 1.6 Overview of Studies . 16 1.6.1 Disambiguation . 16 1.6.2 Study1 ....................... 23 1.6.3 Study2 ....................... 24 1.6.4 Study3 ....................... 25 1.7 Significance . 26 2 Background ........................... 30 2.1 The interpersonal e↵ects of emotional expressions in human- humaninteractions . 31 xi 2.1.1 Empiricalstudies . 31 2.1.2 Theories . 39 2.1.3 Summary ...................... 46 2.2 E↵ects of emotional expressions in human-virtual human interactions ......................... 47 2.2.1 Empiricalstudies . 48 2.2.2 Theories . 61 2.2.3 Summary ...................... 69 2.3 The Lunar Survival Task . 69 2.4 AmazonMechanicalTurk . 71 3 Study 1: E↵ects of Virtual Humans’ Facial Emotional Displays on Persuasion ............. 74 3.1 Literature review . 75 3.2 Design . 77 3.3 Method . 79 3.3.1 Participants . 79 3.3.2 Procedure and materials . 80 3.3.3 Dependent measure . 88 3.4 Resultsanddiscussion . 89 3.4.1 Manipulationcheck . 89 3.4.2 E↵ectsofemotionalexpressions . 91 4 Study 2: Emotional Display, Power, and Appropriateness ........................ 94 4.1 Literature review . 95 4.2 Design . 96 xii 4.3 Method . 99 4.3.1 Participants . 99 4.3.2 Procedure and materials . 100 4.3.3 Dependent measure . 101 4.4 Resultsanddiscussion . 101 4.4.1 Manipulationcheck . 101 4.4.2 Main e↵ects . 103 4.4.3 Interaction e↵ects.................. 105 5 Study 3: Photorealism and Emotional Display .....113 5.1 Literature review . 114 5.2 Design . 118 5.3 Image Ratings . 119 5.4 Method . 123 5.4.1 Participants . 123 5.4.2 Procedure and materials . 123 5.5 Resultsanddiscussion . 124 5.5.1 E↵ectsofphotorealism . 124 5.5.2 E↵ectsofemotionalinformation . 124 6 Conclusions ...........................128 6.1 Discussion . 128 6.2 Contributions ........................ 134 6.3 Limitations . 136 6.4 Future work . 137 References ..............................140 Appendices .............................150 xiii A Data analyses excluding participants who made no changes ..............................151 A.1 Study1 ..........................