OPTIMIZING PERFORMANCE AND SATISFACTION IN VIRTUAL REALITY ENVIRONMENTS WITH INTERVENTIONS USING THE DATA VISUALIZATION LITERACY FRAMEWORK Andreas Bueckle Submitted to the faculty of the University Graduate School in partial fulfillment of the requirements for the degree Doctor of Philosophy in the Luddy School of Informatics, Computing, and Engineering, Indiana University July 2021 Accepted by the Graduate Faculty, Indiana University, in partial fulfillment of the requirements for the degree of Doctor of Philosophy. Doctoral Committee _______________________________ Katy Börner, Ph.D. _______________________________ Patrick C. Shih, Ph.D. _______________________________ John A. Walsh, Ph.D. _______________________________ Robert D. Montoya, Ph.D. April 19, 2021 ii Copyright ©2021 Andreas Bueckle iii To Mama To Aga To Alex To Chris iv ACKNOWLEDGMENTS First and foremost, I want to express my deep gratitude to my advisor Katy Börner for hundreds, if not thousands of hours of mentorship, direct and indirect, and for her courage to challenge and be challenged together. Second, I would like to say a heartfelt “Thank You!” to my research committee, consisting of Patrick Shih, John Walsh, and Rob Montoya, for all the guidance, feedback, and support. Third, I gratefully acknowledge support from the following entities and people: the Department of Information & Library Science (ILS) for the ILS Leadership Award; the Cyberinfrastructure for Network Science Center (CNS) and the Department of Intelligent Systems Engineering (ISE) in the Luddy School of Informatics, Computing, and Engineering (SICE) at Indiana University for my employment as a research assistant and associate instructor as well as the ability to be part of large scientific endeavors; my stellar colleagues at CNS and in SICE; the German Academic Exchange Service (Deutscher Akademischer Austauschdienst, DAAD) for my graduate scholarship as a non-degree student in 2014/2015; and the Advanced Visualization Lab (AVL), especially Chauncey Frend, for tech support and the encouragement to pursue virtual reality as a research topic. Also, I would like to thank Klaus Gasteier at Berlin University of the Arts, who introduced me to interaction design while I was pursuing my Master’s degree prior to coming to IU. My thanks also go to Philip Beesley for allowing me to use his 3D model of Luddy Hall in my dissertation, and for the opportunity to collaborate with the world-class artists in his team. v Fourth, I would like to acknowledge the contributions of co-authors, co-creators, and colleagues. Like most interdisciplinary work, the research presented here is the product of collaboration between people with different skills. I gratefully acknowledge my co-authors Katy Börner and Michael Ginda for their work on the Data Visualization Literacy Framework (32), on which I built the theoretical core of this dissertation. Chapters 5 and 6 were co-written with Kilian Buehling, Patrick Shih, and Katy Börner, a great collaboration for which I express my deepest gratitude. Further, JangDong “JD” Seo from the Indiana Statistical Consulting Center (ISCC) provided his expert input on power analysis for determining the required number of subjects for the user studies presented in chapters 5, 6, and 7. Robert L. Goldstone assisted in developing the initial research design for chapter 5. Leonard E. Cross supported the development of the three setups tested in chapter 5 through active play-testing and discussions. Kristen Browne (Bioinformatics and Computational Biosciences Branch, Office of Cyber Infrastructure and Computational Biology, National Institute of Allergy and Infectious Diseases, National Institutes of Health) created the 3D model of the kidney used in chapters 5 and 6. Todd Theriault performed copy-editing and proof- reading for chapter 5. Perla Brown improved the figures in chapter 5. Eight colleagues at CNS served as pilot testers for the virtual reality (VR) setups developed for this dissertation. Lisel Record, Esmé Middaugh, and my brother Alexander Bueckle graciously served as friendly reviewers for my dissertation proposal in 2020. Further, I would like to acknowledge Sanjay Jain, M.D., Ph.D., and Diane Salamon, R.N., from Washington University in St. Louis School of Medicine for providing the pictures of dissected kidneys. vi Last but not least, I would like to thank my fellow sailors at the Sailing Club at Indiana University, where I had the honor of serving as Coach, Captain, and Marketing Chair throughout my graduate school experience at IU. Being on the water has always kept me full of energy for the road ahead. vii PREFACE Throughout my educational career, I have admired the transformative power of mediated experiences, or the fact that we can get excited by seeing shadows on the wall of a cave (or, in modern times, colorful pixels on a screen). Challenged and highly excited by “You Are Not A Gadget” (123), a provocative reflection on the internet, modern media, and their influence on society by VR pioneer Jaron Lanier, I found information science and interaction design as a perfect place to set up my new inner intellectual workshop. When I experienced proper VR for the first time at IU, I had a glimmer of an understanding what those people in the late 19th century must have felt when seeing that famous first film record of a train arriving at a station (174). According to an urban legend, when people watched this clip first the first time, many believed the train would dash out of the screen at any moment, simply because they had never experienced this kind of medium before. Being enchanted by the possibilities of a new medium never loses its power, whether it is in 1895 or in 2016. Combining VR with data visualization then, a field with rich theoretical and applied history, seemed like a dream project for a dissertation that is both theoretically sound and practically valuable, and that can help address real-world issues in a world increasingly reliant on our human ability to analyze data, identify patterns, and spot trends. I am grateful that this dissertation worked out the way it did, allowing me to combine data visualization for solving practical problems with the strength of the mediated experience that is VR. While collecting the data for this dissertation, I had the privilege of watching many dozens of people move and act in VR for the first time. While the VR applications viii described in this dissertation – owing to the nature of this research – lack the graphical fidelity, action, and larger-than-life environments of video games or similar products of entertainment, I witnessed that VR has an impact almost regardless of content. With immersive media becoming increasingly cheap, immersive content becoming ever more widespread, and producing immersive content getting easier by the day, I cannot wait to see where we can take the emerging field of immersive data visualization. As a tinkerer, I cannot wait to continue building. Andreas Bueckle Friday, July 16, 2021 ix Andreas Bueckle OPTIMIZING PERFORMANCE AND SATISFACTION IN VIRTUAL REALITY ENVIRONMENTS WITH INTERVENTIONS USING THE DATA VISUALIZATION LITERACY FRAMEWORK In the age of big data, interactive data visualizations are becoming increasingly prevalent. The ability to focus on subsets of data is essential for exploring large temporal, geospatial, topical, and network datasets. In order to categorize all parts of data visualizations, various attempts have been made to build frameworks for how to interpret, teach, and construct data visualizations while turning data into insights. This need for interactive data visualizations has sparked interest to classify key interaction types. The recent rise of affordable virtual reality (VR) has introduced yet more ways of interacting with datasets using our visual abilities while enabling user input beyond mouse and keyboard. In this dissertation, we apply the Data Visualization Literacy Framework (DVL-FW) in a series of three interconnected VR user studies with 152 subjects representing around 123 hours of face-to-face data collection. In the first study, we compare performance and satisfaction across two VR and one desktop implementation of the same 3D manipulation interface. We found that while VR users are about three times as fast and about a third more accurate in terms of rotation than desktop users, there are no significant differences for position accuracy. Building on this experiment, in the second study, we investigate quantitative differences between two user cohorts, where the experiment cohort gets to inspect their own manipulation performance data between trials in a VR setup and with a traditional 2D line graph, depending on their x assigned setup (“Reflective phase”). Our findings indicate that while there is no difference in performance between VR users across cohorts, the Reflective phase yields significant differences for desktop users and increases the satisfaction for VR users. Moreover, we identified behavioral metrics for VR users in the Reflective phase that have a favorable effect on performance in subsequent trials. Finally, in the third study, we asked users to travel to various points inside a virtual 3D model of Luddy Hall on the Indiana University campus in Bloomington, IN. We then tested whether the experiment cohort was able to devise faster movement strategies after a Reflective phase where they inspected their own navigation data in a VR visualization with a miniature model of the building. We found that users with a Reflective phase