THE DEVELOPMENT AND VALIDATION OF THE GAME USER EXPERIENCE SATISFACTION SCALE (GUESS) A Dissertation by Mikki Hoang Phan Master of Arts, Wichita State University, 2012 Bachelor of Arts, Wichita State University, 2008 Submitted to the Department of Psychology and the faculty of the Graduate School of Wichita State University in partial fulfillment of the requirements for the degree of Doctor of Philosophy May 2015 © Copyright 2015 by Mikki Phan All Rights Reserved THE DEVELOPMENT AND VALIDATION OF THE GAME USER EXPERIENCE SATISFACTION SCALE (GUESS) The following faculty members have examined the final copy of this dissertation for form and content, and recommend that it be accepted in partial fulfillment of the requirements for the degree of Doctor of Philosophy with a major in Psychology. _____________________________________ Barbara S. Chaparro, Committee Chair _____________________________________ Joseph Keebler, Committee Member _____________________________________ Jibo He, Committee Member _____________________________________ Darwin Dorr, Committee Member _____________________________________ Jodie Hertzog, Committee Member Accepted for the College of Liberal Arts and Sciences _____________________________________ Ronald Matson, Dean Accepted for the Graduate School _____________________________________ Abu S. Masud, Interim Dean iii DEDICATION To my parents for their love and support, and all that they have sacrificed so that my siblings and I can have a better future iv Video games open worlds. — Jon-Paul Dyson v ACKNOWLEDGEMENTS Althea Gibson once said, “No matter what accomplishments you make, somebody helped you.” Thus, completing this long and winding Ph.D. journey would not have been possible without a village of support and help. While words could not adequately sum up how thankful I am, I would like to start off by thanking my dissertation chair and advisor, Dr. Barbara Chaparro. Thank you for being the candle that illuminates my path to Ph.D. success. Without you and Dr. Alex Chaparro, I would not have discovered the field of human factors psychology. Both of you have saved me from a career that might bring me a lot of money, but which brought me no joy. Dr. C., thank you for your endless amount of patience, guidance, and support. Thank you for always making yourself available whenever I needed you, and knowing precisely what to say whenever one of my dissertation fears kicked in. You are and always will be the heroine in my book! To the rest of my dissertation committee (Drs. Joseph Keebler, Jibo He, Darwin Dorr, and Jodie Hertzog) I extend my deepest gratitude for your time, support, and thoughtful comments/suggestions. Specifically, I would like to thank further Dr. Joseph Keebler for sharing his statistical knowledge and expertise. Dr. Keebler, thank you for your words of encouragement and continuous reassurance about the quality of my research—I always slept a little better at night after conversing with you! This dissertation would also not have been possible without a group of people who, despite being extremely busy, took the time to provide detailed feedback about my research. And for this, they will always have my gratitude and a special place in my heart—thank you Dr. vi Davin Pavlas, Dr. Rochelle Edwards, Dr. Amanda Smith, Dustin Smith, Neethan Siva, Gigi Phan, Peter Phan, Duc Nguyen, C.J. Randall, and Erin Gannon. I also would like to thank Drs. Wijnand IJsselsteijn and Frans Mäyrä for providing me a copy of their gaming scale/questionnaire. Next, I would like to send a heartfelt thanks to those who have helped me in sharing my gaming surveys as well as to the owners/administrators who have allowed me to share my surveys on their forums/websites. Particularly, I want to thank Dr. Charles Burdsal for making the WSU panels via Qualtrics possible. Additionally, I want to thank the following people who were instrumental in helping me recruit more respondents: Mr. Norman Bent, Christina Siu, Dominic Canare, Nicole Stahl, Dustin Smith, Neethan Siva, Seth Correa, Duc Nguyen, Peter Phan, and Jacqueline Huynh. I apologized that I could not list everyone’s name, but please know that your help is very much appreciated! Although they may not be directly involved in my dissertation, I would especially like to thank the people who were influential in motivating me to pursue an advanced education. Thank you, Mses. Vanessa Souriya-Mnirajd, Shukura Bakari-Cozart, and LaWanda Holt-Fields for believing in me and for encouraging me to join the McNair Scholars Program, which had become a life-changing experience. Another important mentor in my life that I would like to acknowledge is Dr. Deema de Silva. Thank you for taking me under your wings and showing me that I too can fly. I am also extremely thankful for my family for their unconditional love and support during this long journey. Thank you to my Ba and Ma for instilling in me the value of hard work and a good education early in life. I am forever grateful for their decision to abandon everything vii in their homeland and move to a foreign place in search of a better future for their children. Despite not knowing much about the HF psychology doctorate they have remained supportive of my “professional student” status for most of my grad school years. And they have only asked when I am going to get a real job every once in a while. In addition to my parents, I would like to thank my siblings, my sister-in-law, and my aunt and uncle for their constant support and encouragement. Like my parents, they did not ask (excessively) the grad-student-most- forbidden question, “When are you going to graduate?” Aside from my immediate family, I would like to thank all of my friends for their love and support, as well as patience during my years in graduate school. Thank you for being a source of joy and welcomed distraction whenever I needed a break from work. And thank you for being understanding when I could not hang out (for what felt like a century) due to school and work. Of the friends who I held near and dear to my heart, I especially want to acknowledge two individuals who I considered as my “brother and sister from another mother” and soon-to- be Drs., Jennifer Teves and Bobby Nguyen. Thank you for embarking on this Ph.D. adventure with me, and for always being there to lend your listening ears and shoulders to lean on when I felt stressed and tired. Most of all thank you for the much-needed laughter during the difficult times. I would not be able to make it this far without both of your company. I love both of you very much, and I cannot wait to celebrate your future successes! Finally, this dissertation would not have even begun if it were not for a video game that changed a young girl’s world 23 years ago. Thank you creators of Tetris for igniting my love for video games, and for showing me that a good video game can make all the difference. viii ABSTRACT Since the 1970’s, video games have grown to become a ubiquitous form of digital entertainment. As the demand for video games continues to increase, video game developers and designers are facing the difficult task of creating games that are not only enjoyable to play, but marketable in a highly competitive industry. In order to improve the success of commercial video games, some game companies have turned to using a variety of game evaluation methodologies to improve their game’s design. The most common form of game evaluation in the video game industry is called playtesting, which involves the players providing feedback about a game after they have played it for a period of time. However, meaningful results are difficult to obtain from a playtesting session without a properly constructed gaming questionnaire or scale. Thus, there is a need for a psychometrically validated, comprehensive, gaming scale that is appropriate for playtesting and game evaluation purposes. Following the current best practices of scale development and validation, this research employed a mixed-methods design that consisted of a five-step plan to develop and validate a new satisfaction scale for gaming. As a result, a new instrument measuring video game satisfaction called the Game User Experience Satisfaction Scale (GUESS) with nine subscales emerged. In general, the GUESS was demonstrated to have good content validity and internal consistency, as well as satisfactory convergent and discriminant validity. Furthermore, the GUESS was developed and validated based on the assessments of over 450 unique video game titles across a number of popular genres (e.g., Role-Playing, Action Adventure). Thus, it can be applied across many types of video games in the industry. ix TABLE OF CONTENTS Chapter Page 1. JOURNAL ARTICLE ..................................................................................................................... 1 Introduction .............................................................................................................................. 1 Usability Testing of Video Games ......................................................................................... 2 Heuristic Evaluation of Video Games ................................................................................... 4 Playtesting and Gaming Scales/Questionnaires ................................................................... 5 Research Purposes .................................................................................................................... 8 Step 1: Item Pool Generation and Truncation .........................................................................