THE WORLD IS NOT ENOUGH: TRUST IN COGNITIVE AGENTS by Ewart Jan de Visser A Dissertation Submitted to the Graduate Faculty of George Mason University in Partial Fulfillment of The Requirements for the Degree of Doctor of Philosophy Psychology Committee: Director Department Chairperson Program Director Dean, College of Humanities and Social Sciences Date: Summer Semester 2012 George Mason University Fairfax, VA The World Is Not Enough: Trust in Cognitive Agents A dissertation submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy at George Mason University By Ewart Jan de Visser Master of Arts George Mason University, 2007 Bachelor of Arts University of North Carolina at Wilmington, 2005 Director: Raja Parasuraman, University Professor Department of Psychology Summer Semester 2012 George Mason University Fairfax, VA Copyright c 2012 by Ewart Jan de Visser All Rights Reserved ii Dedication I dedicate this dissertation to Jos, Olga, Emmy, and Yorick. Your unconditional love and trust has kept me going. De raderen draaien...nog steeds. iii Acknowledgments My FILM 101 teacher told me on the first day of class that creating a movie constitutes a small miracle. It is no different with a dissertation. Many of the ideas, methods, and procedures contained in this work were conceived, adjusted, and refined after many discussions with a number of people. I have been blessed to be surrounded by such a group of loving, caring, and supportive people throughout my graduate studies, and I would like to mention these individuals here with a few words. Raja, you have supported me in every way possible throughout my Ph.D. I cannot imagine a better advisor. Thank you for your patience and support at all times and for setting the bar high. Thank you for showing me gradualness, modesty, and passion in science. I am also pleased to report that I have met your three unofficial Ph.D. goals, including eating your delicious spicy food, publishing one first authored publication in a respectable journal, and beating you once in pool. Frank, it was serendipity that you arrived at George Mason at the time of my proposal writings on trust. You have directed me to all the neuroeconomic aspects of trust, which has greatly enhanced my work. Thanks for all your crucial help in finalizing my proposal and critical comments on my work. I have enjoyed our many idea-filled meetings at the Jazzman cafe. Patrick, thanks for the many scientific and personal discussions on the philosophy of science, trust, and a host of other topics near the JC bike rack. Thanks for pushing me towards R and Latex in the end; my thesis rests on its structure. Tyler, thanks for all the daily soulful conversations, our frequent ice-tea runs with Delicious Dutch Delectables, frequent encouragement to emerge from the fog and get to the story, numerous visits to the man-cave, dissertation writing sessions at Bus Boys and Poets, award winning movie making, all the fun times in San Francisco, Miami, and Vegas, and for being a true friend. I would like to thank the following Arch Lab members and research assistants who have helped me with developing the materials for this dissertation, collecting and coding data, and helped me to develop the software to run the experiment. These include Steve Scheid, Stephanie Chalk, Laya Muralidharan, Rizhna Chener, Andreina Daza, Andres Arredondo, Saman Berenji, Lara Moody and Melissa Smith. I would also like to thank Peter-Paul van Maanen, Peter Twieg, Adam Emfield, and Bill Miller for assistance in programming the experimental task. Furthermore, I would like to thank Asaf Beasley for providing the AMP task and Kevin Young for providing the Balloon Task. To the The Fisherman Circle, you are my foundation. To my dad, thank you for keeping me on track to finish, reading my writings, and promoting my work to others. To my mom, thanks for the for reminders to get the PhD done, the wonderful time at the Ole Mink farm, frequent visits to the USA with dad, and even making my graduation goal your password (please change it now). To my sister, thank you for being a golden angel and keeping me iv on the ground. To my lil bro, thanks for providing humorous reviews, notifying me of the latest I-phone games on my breaks, and completing your awesome thesis ahead of me. To Jasmine, thank you for daily morning technical and emotional phone support, your mangos, melons, and lentils, our late night skype writing nights, and the fun classes we shared together for distraction and relaxation. To Minhaj, thank you for the writing sessions in the Caribou coffee shop in the early stages of my dissertation. Thank you for listening with enthusiasm to the formation of my ideas on this topic, showing and reminding me how to remain a Homo Ludens, and explaining to me how desis became the greatest nation on earth. Finally, I would like to thank the American Psychological Association (APA) for awarding me a Dissertation Research Award that supported this work and the grant FA9550-10-1-0385 from the Air Force Office of Scientific Research. The creation of something new is not accomplished by the intellect but by the play instinct acting from inner necessity. - Carl Jung v Table of Contents Page ListofFigures........................................ viii Abstract........................................... x 1TrustinCybersocieties................................ 1 1.1 Introduction.................................... 1 1.2 Human-Human Trust . 3 1.2.1 Threat Policy and Action . 5 1.2.2 Threat Assessment . 5 1.2.3 Trust Evolution . 7 1.3 Human-Automation Trust . 8 1.3.1 Agent Belief . 9 1.3.2 Agent Experience . 9 1.4 Human-Human Trust vs. Human-Automation Trust . 10 1.4.1 Agent Belief . 11 1.5 AgentExperience................................. 14 1.5.1 Task Difficulty .............................. 14 1.5.2 Etiquette ................................. 14 1.5.3 TrustRepair ............................... 15 1.5.4 Feedback ................................. 16 1.5.5 Level of Automation Assistance . 16 1.5.6 Trust Evolution . 17 1.5.7 Individual Differences .......................... 17 1.6 The Cognitive Agent Experiments . 18 1.6.1 TheCurrentStudy............................ 19 2 MethodsandResults ................................. 23 2.1 Experiment1 ................................... 23 2.1.1 Method .................................. 23 2.1.2 Results .................................. 30 2.1.3 Discussion................................. 38 2.2 Experiment2 ................................... 38 vi 2.2.1 Method .................................. 38 2.2.2 Results .................................. 39 2.2.3 Discussion................................. 47 2.3 Experiment3 ................................... 47 2.3.1 Method .................................. 47 2.3.2 Results .................................. 48 2.3.3 Discussion................................. 53 2.4 Experiment4 ................................... 54 2.4.1 Method .................................. 54 2.4.2 Results .................................. 55 2.5 Discussion..................................... 62 3 GeneralDiscussion................................... 64 3.1 Human-Human Trust and Human-Automation Trust are Different . 64 3.2 Agent Belief interacts with Agent Experience . 66 3.3 The Cognitive Agent Spectrum Is Useful for Categorizing Agents . 67 3.4 Limitations . 68 3.5 Implications for Design and Training . 69 3.6 FutureResearch ................................. 69 3.7 Conclusion .................................... 70 A BenevolentStories ................................... 72 A.1 Computer Agent Story: PC-Check . 72 A.2 Avatar Agent Story: TaxFiler . 73 A.3 Human Agent Story: Matt O’Shea . 75 B MalevolentStories ................................... 77 B.1 Computer Agent Story: SecCrack . 77 B.2 Avatar Agent Story: TXTSPY . 78 B.3 Human Agent Story: Matt O’Shea . 80 References . 82 vii List of Figures Figure Page 1.1 The Eastern Scheldt storm surge barrier (Dutch: “Oosterscheldekering”). 3 1.2 Advanced organizer: Trust threat taxonomy . 4 2.1 Theexperimentaldesign............................. 24 2.2 The Cognitive Agent Spectrum . 25 2.3 The Cognitive Agents . 26 2.4 The TNO Trust Task (T 3). ........................... 28 2.5 Agent by reliability for pattern recognition performance in Experiment 1. The red dashed line indicates chance performance on the task. 31 2.6 Agent by reliability for overall performance in Experiment 1. 32 2.7 Agent by reliability for compliance in Experiment 1. The red triangles indicate calibrated performance. 33 2.8 Agent by reliability for self-reliance in Experiment 1. 35 2.9 Agent by reliability for trust in Experiment 1. The red triangles indicate calibratedtrust................................... 36 2.10 Agent by time for perceived morality in Experiment 1. 37 2.11 Agent by reliability for overall performance in Experiment 2. The red dashed line indicates chance performance on the task. 41 2.12 Agent by reliability for compliance in Experiment 2. The red triangles indicate calibrated performance. 42 2.13 Agent by reliability for self-reliance in Experiment 2. 43 2.14 Agent by reliability for trust in Experiment 2. The red triangles indicate calibratedtrust................................... 44 2.15 Agent by Time for perceived security in Experiment 2. 45 2.16 Agent by Time for perceived morality in Experiment 2. 46 2.17 Agent by reliability for overall performance in Experiment 3. 49 2.18 Agent by reliability for compliance in Experiment 3. The red triangles indicate calibrated performance.