FLUX VR: THE DEVELOPMENT AND VALIDATION OF A HEURISTIC CHECKLIST FOR VIRTUAL REALITY GAME DESIGN SUPPORTING IMMERSION, PRESENCE, AND FLOW A Dissertation by Daniel Smith Master of Arts, Wichita State University, 2018 Bachelor of Science, Cameron University, 2015 Submitted to the Department of Psychology and the faculty of the Graduate School of Wichita State University in partial fulfillment of the requirements for the degree of Doctor of Philosophy May 2021 © Copyright 2021 by Daniel Smith All Rights Reserved FLUX VR: THE DEVELOPMENT AND VALIDATION OF A HEURISTIC CHECKLIST FOR VIRTUAL REALITY GAME DESIGN SUPPORTING IMMERSION, PRESENCE, AND FLOW The following faculty members have examined the final copy of this dissertation for form and content, and recommend that it be accepted in partial fulfillment of the requirement for the degree of Doctor of Philosophy with a major in Psychology. Rui Ni, Committee Chair Barbara Chaparro, Committee Member Traci Hart, Committee Member Michael Jorgensen, Committee Member Lisa Vangsness, Committee Member Accepted for the College of Liberal Arts and Sciences Andrew Hippisley, Dean Accepted for the Graduate School Coleen Pugh, Dean iii DEDICATION To my partner, Jasmine – I could never express my appreciating for your support these past few years. This dissertation marks the end of an era and the start of our new lives. I wouldn’t want to share it with anyone else, I love you. To my Mother – Your emotional support has been unwavering for 28 years. I know you’ve wanted this just as much as me. Everything we went through motivated me to make it this far. My success is our success. We did it. To my Father – You’ve always pushed me to be better. I wouldn’t have made it this far if you hadn’t shown me that anything can be done with enough effort. iv ACKNOWLEDGEMENTS When I was in high school, I wanted to be a medical doctor. While sitting in the chair to enroll for my first semester of college, I decided I didn’t want to go to school for that long and elected to major in Psychology. About 2 years in, I decided counseling wasn’t for me and was very lost in what I wanted to do. Then, in my Psychology of Perception class, I heard my professor mention “engineering psychology”. I have always loved building things and regretted not making it a career, which made this very interesting to me. I asked him to explain the field to me more and the rest is history. I started my journey into human factors psychology and shortly after discovered that I could do research on whatever I wanted, including video games. This was the first step in fulfilling my childhood dream of working in video games, but there were so many more steps to be taken to get here today. It all started with my first advisor, Dr. Adam Randell. At my old university, there were no research labs, so he and Dr. Jeff Seger started the Research Club, where I became president simply because I wanted to go to graduate school. Even though he wasn’t my assigned advisor, Dr. Randell went to great lengths to help prepare me for graduate school by pushing me as far as he could with my writing in his classes, giving me responsibilities in research projects he had set up, and any other areas he could find. The effort given but not required by Dr. Randell and Dr. Seger will forever be the jumping point for my graduate education and I hope to offer the same help to anyone that could possibly benefit from it. Neither of you had to do what you did for me, and for that I am forever grateful. When I started at Wichita State, I left behind everyone and everything I knew to start a new life. Alongside me were the members of my cohort: Adam, Alexis, and Jasmine. By some crazy stroke of luck, we were all interested in video games, both casually and as a career, so we bonded immediately. We grew very close very quickly and that became a rock for me for the remaining years of graduate school. We shared stories and frustrations daily, supporting each other through all of the trials and tribulations that come with graduate school. Through internship rejections, stressful assignments, long nights, and v everything in between, I always felt like I wasn’t going through it alone because of you three. There exists no universe where I would have made it all the way through graduate school without you three, thank you for enriching my life. To S.W.A.R.M., my online community of friends, you were all a blessing throughout the COVID quarantine. I probably would have lost my mind if not for the consistent contact with people through our discord and game nights. To all of my fellow HF students, thank you. We’ve all stood together through so many things, both personal and academic. Kirsten, Tiffany, Monica, Bill, Dakota, Abbie, Dom, Inga, Will S., Will C., Shivani, JP, Brad, Jake, Christina P., Christina S., Brady, Taylor, and Kevin, I hold so many dear memories with all of you that make up many of the highs amongst the myriad of lows caused by graduate school. For those of you that will come after me, keep going, you’re going to make it. vi ABSTRACT Virtual reality (VR) gaming is a rapidly-growing field in both technology and adoption (eMarketer, 2019). However, because the technology is relatively new, there is a lack of guidance on what constitutes good design and effective evaluations measures for VR games. Because the goals of VR gaming are immersion and presence, it’s important that effective evaluation tools are available that may catch usability problems that interfere with these goals. This study utilized an eight-step method developed by Quiñones et al. (2018) to develop and validate a heuristic checklist to evaluate VR games. The experimental VR games heuristic checklist was compared to a control checklist developed by Muñoz, Barcelos, & Chalegre (2011) for evaluating virtual worlds. Two VR games, Apex Construct and Art of Fight, were evaluated by six experts, three using the experimental checklist and three using the control checklist. Experts were also given a questionnaire used to evaluate the checklist itself, and user tests were conducted on both games to uncover usability problems that might have been missed by both lists. Expert heuristic evaluations resulted in 55 total revisions to the list, including 50 changes to existing questions and the addition of 5 new questions. The expert judgment questionnaire did not result in any adaptations to the questions but did result in the change of three heuristic category importance ratings. User tests uncovered many overlapping usability problems when compared with the heuristic evaluations, but also uncovered unique problems which resulted in the creation of 11 new checklist questions. The final heuristic checklist consists of 145 questions and 15 heuristic categories. vii TABLE OF CONTENTS Chapter Page 1. INTRODUCTION ............................................................................................................................................. 1 2. LITERATURE REVIEW ................................................................................................................................. 5 2.1 Immersive Design in Virtual Reality .............................................................................................. 5 2.2 Immersion, Presence, and Flow ....................................................................................................... 6 2.3 Grounded Theory in Immersion ...................................................................................................... 8 2.4 Immersion, Presence, and Performance Model [IPP] ............................................................. 9 2.5 Flow & Gameflow ............................................................................................................................... 12 2.6 Virtual Reality Technology. ............................................................................................................ 17 2.7 Heuristic Evaluation (HE) ............................................................................................................... 19 3. THE CURRENT STUDY .............................................................................................................................. 25 3.1 Heuristic Checklist Generation and Validation Methodology ........................................... 26 3.2 Step 1: Exploratory stage. ............................................................................................................... 27 3.2.1 Method. ......................................................................................................................................... 27 3.2.2 Results. ......................................................................................................................................... 27 3.2.3 Discussion. .................................................................................................................................. 28 3.3 Step 2: Experimental stage. ............................................................................................................ 29 3.3.1 Method. ......................................................................................................................................... 29 3.3.2 Results. ......................................................................................................................................... 29 3.3.3 Discussion. .................................................................................................................................. 31 3.4