An Abstract of the Dissertation Of
Total Page:16
File Type:pdf, Size:1020Kb
AN ABSTRACT OF THE DISSERTATION OF Karina A. Roundtree for the degree of Doctor of Philosophy in Mechanical Engineering presented on August 21, 2020. Title: Achieving Transparency in Human-Collective Systems Abstract approved: H. Onan Demirel Julie A. Adams Collective robotic systems are biologically-inspired and exhibit behaviors found in spa- tial swarms (e.g., fish), colonies (e.g., ants), or a combination of both (e.g., bees). Collec- tive robotic system popularity continues to increase due to their apparent global intel- ligence and emergent behaviors. Many applications can benefit from the incorporation of collectives, including environmental monitoring, disaster response missions, and in- frastructure support. Human-collective system designers continue to debate how best to achieve transparency in human-collective systems in order to attain meaningful and insightful information exchanges between the operator and collective, enable positive operator influence on collectives, and improve the human-collective’s performance. Few human-collective evaluations have been conducted, many of which have only assessed how embedding transparency into one system design element (e.g., models, visualizations, or control mechanisms) may impact human-collective behaviors, such as the human-collective performance. This dissertation developed a transparency defi- nition for collective systems that was leveraged to assess how to achieve transparency in a single human-collective system. Multiple models and visualizations were evalu- ated for a sequential best-of-n decision-making task with four collectives. Transparency was evaluated with respect to how the model and visualization impacted human oper- ators who possess different capabilities, operator comprehension, system usability, and human-collective performance. Transparency design guidance was created in order to aid the design of future human-collective systems. One set of guidelines were inspired from the results and discussions of the single human-collective analyses and another set were based on a review of the biological literature. This dissertation can be used to aid designers achieve transparency in human-collective systems. The primary contributions are: 1. A transparency definition for human-collective systems that describes the process of identifying what factors affect and are influenced by transparency, why those factors are important, and how to design a system to achieve transparency. 2. An expansive set of metrics that successfully evaluated how transparency influ- enced operators with different individual capabilities, operator comprehension, system usability, and human-collective performance. 3. The recommendation that system transparency quantification requires evaluating the transparency embedded into the various system design elements in order to determine how they interact with one another and influence the human-collective interactions and performance. 4. Design guidance recommendations with respect to models, visualizations, and control mechanisms in order to inform designers how transparency can be achieved for human-collective systems. c Copyright by Karina A. Roundtree August 21, 2020 All Rights Reserved Achieving Transparency in Human-Collective Systems by Karina A. Roundtree A DISSERTATION submitted to Oregon State University in partial fulfillment of the requirements for the degree of Doctor of Philosophy Presented August 21, 2020 Commencement June 2021 Doctor of Philosophy dissertation of Karina A. Roundtree presented on August 21, 2020. APPROVED: Co-Major Professor, representing Mechanical Engineering Co-Major Professor, representing Mechanical Engineering Head of the School of Mechanical, Industrial, and Manufacturing Engineering Dean of the Graduate School I understand that my dissertation will become part of the permanent collection of Oregon State University libraries. My signature below authorizes release of my dissertation to any reader upon request. Karina A. Roundtree, Author ACKNOWLEDGEMENTS I completed this dissertation as a graduate research assistant supported by the US Of- fice of Naval Research Awards N000141613025 and N000141812831. While completing this work, I received technical assistance from Dr. Michael Goodrich, Dr. Jason R. Cody, Dr. Jamison Heard, Dr. Gina Olson, Dr. Gilberto Marcon dos Santos, Jennifer Leaf, Dagim Gebretsadik, Lorenzo Bermillo, Golden Rockefeller, Rene Martinez, Alan Sanchez, Anita Ruangrotsakun, and Gabriela Magana. Thank you all for your support. The most important person I must thank for helping me through my PhD journey is my partner, Dagim Gebretsadik. My success throughout this process is attributed to you. Thank you for being there every single step of the way, being my biggest fan, for always believing in me, and for your support, love, guidance, and help (especially when it came to coding). I love you dearly. This PhD belongs to both of us! The second set of people I want to thank are my advisors, Dr. Onan Demirel and Dr. Julie A. Adams. Thank you Dr. Demirel for your support, enthusiasm, and for always sparking my creativity. I was able to explore and learn about many domains thanks to your guidance. Thank you for always believing in me. Julie, there are no words that can express how thankful and blessed I am to have had the opportunity to work with you. The amount of professional and technical growth I have gained from you is tremendous. Thank you for giving me opportunities to grow and for constantly challenging me out of my comfort zone. I want to thank all the professors who aided me throughout this process. Thank you Dr. Michael Goodrich for being an amazing collaborator on the ONR grant and for your insights, guidance, and support on my research. Dr. Nancy Squires thank you always believing, supporting, and inspiring me. You are truly missed. Thank you Dr. Irem Tumer for helping find a PhD project I fell in love with. Thank you Dr. Ken Funk for entertaining my questions in class and supporting me. I learned so much from you. I also want to thank my committee, for their time, effort, and insights. There is a long list of OSU students I want to thank from the Mechanical Engineer- ing, Industrial Engineering, and Robotics program that supported me. Your friendship, kindness, and support helped me during this PhD journey. I want to thank my friends and sorority sisters, especially Daniella Roquett, for always cheering me on from afar and supporting my family and I through some hard personal times. The last and most important people I must thank is my family. Thank you for your love, patience, sup- port, encouragement, and for always believing in me. It has been a long journey. We finally made it, te quiero. TABLE OF CONTENTS Page 1 Introduction . 1 2 Literature Review . 5 2.0.1 Spatial Swarms . .6 2.0.2 Colonies . 15 2.0.3 Collectives . 20 2.1 Transparency . 22 2.1.1 Transparency Definitions . 22 2.1.2 Factors of Transparency . 23 2.2 Design to Achieve Transparency . 36 2.2.1 Providing Characteristics . 36 2.2.2 Using Design Principles . 47 2.2.3 Training . 55 3 Experimental Analysis . 62 3.1 Human-Collective Task . 62 3.2 Interface Environment . 63 3.2.1 Individual Agents Interface . 65 3.2.2 Collective Interface . 69 3.3 IA and Collective User Evaluations Experimental Design . 71 3.3.1 Independent Variables . 71 3.3.2 Experimental Procedure . 71 3.3.3 Participants . 73 3.4 Analyses of Transparency Experimental Design . 74 3.4.1 Research Questions . 75 3.4.2 Independent Variables . 76 4 Results and Discussions . 77 4.1 Visualization Analysis . 77 4.1.1 R1: Visualization Influence on Human Operator . 78 4.1.2 R2: Visualization Promotion of Human Operator Comprehension 93 4.1.3 R3: Visualization Usability . 103 4.1.4 R4: Visualization Influence on Human-Collective Performance . 115 4.1.5 Visualization Analysis Discussion . 122 TABLE OF CONTENTS (Continued) Page 4.2 Model with Visualization Analysis . 126 4.2.1 R5: System Design Element Influence on Human Operator . 127 4.2.2 R6: System Design Element Promotion of Operator Comprehension142 4.2.3 R7: System Design Element Usability . 164 4.2.4 R8: System Design Element Influence on Team Performance . 190 4.2.5 Model with Visualization Analysis Discussion . 200 4.3 Visualization and Model with Visualization Conclusions . 205 5 Design Guidance for Human-Collective Systems . 207 5.1 Design Guidance based on the Single Human-Collective Evaluations . 208 5.1.1 Human-Collective Visualization Design Guidance . 208 5.1.2 Human-Collective Model Design Guidance . 211 5.1.3 Human-Collective Control Mechanisms Design Guidance . 212 5.2 Biologically Inspired Design Guidelines . 214 5.2.1 Undesirable Emergent Behaviors . 214 5.2.2 Cohesion . 227 5.2.3 Timing to Maintain Cohesion . 232 5.2.4 Individual Collective Entities Roles . 235 5.2.5 Limited Communication Among Individual Collective Entities . 245 5.2.6 Collective and Subgroup Information ................. 254 5.2.7 Leadership . 264 5.3 Design Guidance Reliability for Real World Use Scenarios . 270 6 Conclusion . 274 6.1 Contributions . 276 6.2 Future Work . 279 Bibliography . 285 Appendices . 302 LIST OF FIGURES Figure Page 2.1 Human-machine system operating in an environment [1]. .5 2.2 Concept Map of human-machine system direct and indirect transparency factors [2]. 25 3.1 The Individual Agents (IA) interface two and half minutes into a trail, showing four collectives (rectangles with Roman numerals), and the six- teen discovered targets (rectangles with integers). The target’s value is represented by the green color, where higher values were brighter. The legend in the lower right corner identifies the individual collective entity state information and target range information. 66 3.2 The Collective interface mid-way through a trail scenario, showing the current locations of the four collectives (rectangles with Roman numer- als) and the locations of the discovered targets (green and blue squares with integer identifiers). The top half of each target indicates the target’s relative value (green) and the bottom half indicates the support of the highest supporting collective (blue). The legend in the upper left hand corner identifies the target range information.