User Preferences for Video Game Delivery A Case Study of Cloud Gaming Ole Kristian Brenne Richstad Master of Science in Computer Science Submission date: June 2015 Supervisor: Alf Inge Wang, IDI Norwegian University of Science and Technology Department of Computer and Information Science Abstract In the recent years there have been a number of video game streaming platforms emerging on the market. Cloud gaming services like OnLive and PlayStation Now may ultimately prove successful in tapping the Internet to deliver instant streaming video games. How- ever, there has been a limited research on these platforms, especially focusing on the end user preferences. In this thesis, preferences of different gamer groups (hardcore and casual) are studied in order to identify possible development directions for such services. A test has been designed and performed with users from the different groups. The commercial PC-based service OnLive has been used for testing the user experience and user acceptance of cloud gaming. A custom public game test has been conducted as well. Observation, user surveys and interviews have been used for data collection. The experiments done focuses on the users’ perception and response towards such a systems in terms of fidelity, usability, comparison to traditional game delivery and likelihood of buying into such a model. The thesis takes a holistic approach were not only the games themselves are tested, but the total experience in using the game service. At large, the results indicate that casual gamers are more positive to the cloud gaming service model than hardcore gamers. The findings suggest that one of the reasons are that casual gamers choose convenience over high fidelity in their game preferences. i Sammendrag I de siste årene har det kommet flere plattformer for strømming av videospill. Cloud gaming-tjenester som OnLive og PlayStation Now kan til sist vise seg å bli en suksess i å bruke internett for å levere umiddelbar strømming av videospill. Det har imidlertid vært begrenset med forskning på disse plattformene, spesielt med fokus på brukerpreferanser. I denne avhandlingen er preferanser for forskjellige grupper spillere (hardcore og ca- sual) studert for å identifisere mulige utviklingsretninger for slike tjenester. En test har blitt designet og utført med brukere fra forskjellige grupper. Den kom- mersielle PC-baserte tjenesten OnLive har blitt brukt for å teste brukeropplevelsen og brukerakseptansen til cloud gaming. En egenutviklet spilltest i offentlighet har også blitt utført. Observasjon, brukerundersøkelser og intervjuer har blitt brukt til å samle inn data. Eksperimentene som er gjort fokuserer på brukernes oppfatning og respons på slike systemer med hensyn til teknisk kvalitet, brukbarhet, sammenligning med tradisjonell spill-leveranse og sannsynligheten for å kjøpe seg inn i en slik modell. Avhandlingen tar en helhetlig tilnærming hvor ikke bare spillene i seg selv er testet, men totalopplevelsen ved å bruke spilltjenesten. I det store og hele indikerer resultatene at casual-spillere er mer positive til cloud gaming tjenestemodellen enn hardcore spillere. Funnene antyder at en av grunnene er at casual-spillere velger bekvemmelighet foran høy teknisk kvalitet i sine spillpreferanser. iii Preface This project was performed as a master thesis in the course TDT 4900 at The Norwegian University of Science and Technology (NTNU) and concludes my master de- gree in Computer Science. The thesis is a continuation of the depth study in the course TDT 4501 Computer Science, Specialization Project, at NTNU. The task was created and given by the Department of Computer and Information Science (IDI) at NTNU. I would like to thank the supervisor, professor Alf Inge Wang, for all help and ad- vice during the project. I would also like to thank Egil Fors and Kari Hoås Moen for their feedback and Erlend Fors for proofreading. I gratefully and sincerly thank Heidi Moen Fors for her support and patience during this period. Trondheim, June 2015 Ole Kristian B. Richstad v Contents I Introduction and research1 1 Introduction3 1.1 Project structure................................3 1.2 Motivation...................................4 1.3 Definitions....................................4 2 Research6 2.1 Research methods...............................6 2.1.1 Case study...............................6 2.1.2 GQM Approach............................6 2.2 Research goal and research questions.....................6 2.3 Evaluation approach..............................8 2.3.1 Practical issues.............................8 2.3.2 Ethical considerations.........................8 2.3.3 Data handling.............................8 2.4 Metrics.....................................9 2.4.1 System Usability Scale.........................9 2.4.2 Mean Opinion Score.......................... 10 2.5 Research process................................ 11 2.5.1 Literature study............................ 11 2.5.2 Test Setup............................... 11 2.5.3 Observation............................... 11 2.5.4 Questionnaires............................. 11 2.5.5 Interviews................................ 11 2.5.6 Data gathering............................. 11 2.5.7 Analysis................................. 11 2.6 Tools....................................... 12 II Prestudy 13 3 The Game Industry 15 3.1 Sales....................................... 15 vii 3.2 Digital distribution............................... 15 3.2.1 Steam.................................. 15 3.3 Player types................................... 16 3.4 Game types................................... 16 3.4.1 Casual games.............................. 16 3.4.2 Core games............................... 17 3.5 Current gaming trends............................. 18 3.5.1 Indie games............................... 18 3.5.2 Player created content......................... 18 3.5.3 Piracy and second-hand games.................... 19 3.5.4 Apps................................... 19 3.5.5 Free-to-play............................... 20 3.5.6 Browser games............................. 20 4 Cloud Gaming 21 4.1 Cloud computing................................ 21 4.2 Infrastructure as a Service........................... 22 4.2.1 Amazon Elastic Compute Cloud (EC2)............... 22 4.3 Cloud gaming.................................. 22 4.4 Latency..................................... 23 4.5 Visual Quality................................. 24 5 Cloud Gaming Services 25 5.1 OnLive...................................... 25 5.1.1 Platforms................................ 26 5.1.2 Pricing model.............................. 26 5.2 Gaikai/PlayStation Now............................ 27 5.2.1 Platforms................................ 27 5.2.2 Pricing model.............................. 28 5.3 GRID...................................... 28 5.4 Ubitus/GameNow............................... 29 5.5 GameFly/PlayCast............................... 29 5.5.1 Pricing model.............................. 29 5.6 G-cluster.................................... 30 5.7 Upcoming services............................... 30 5.7.1 Shinra Technologies.......................... 30 5.8 Summary.................................... 31 6 User Perspectives 32 6.1 Quality of Experience............................. 32 6.2 Latency and the human perception...................... 32 6.3 Consumer value................................. 33 6.4 User rights................................... 34 6.4.1 Privacy................................. 34 viii 6.5 Pricing models................................. 34 6.6 User acceptance................................. 35 III Own contribution 37 7 Testing 39 7.1 Test procedures................................. 39 7.1.1 Test environment............................ 39 7.1.2 Grouping the players.......................... 39 7.1.3 Participant selection.......................... 40 7.1.4 Avoiding bias.............................. 40 7.1.5 Game selection............................. 40 7.2 Test setup.................................... 41 7.2.1 Emulation tools............................. 41 7.2.2 Setting up a cloud gaming system.................. 41 7.3 Test execution................................. 48 7.4 OnLive - Online test.............................. 48 7.4.1 Games.................................. 49 7.5 Custom Cloud Streaming - Public test.................... 49 7.5.1 Grand Theft Auto V.......................... 49 7.5.2 Testbed................................. 49 7.5.3 Observation............................... 50 8 Results 53 8.1 Questionnaire - Online test.......................... 53 8.1.1 System Usability Test......................... 59 8.2 Questionnaire - Public test.......................... 60 8.3 Validity..................................... 63 8.3.1 Correlation............................... 63 8.4 Further comments and interviews....................... 64 9 Evaluation 65 9.1 Research questions answered......................... 65 9.2 Project reflection................................ 67 IV Summary 69 10 Conclusion 71 10.1 Summary.................................... 71 10.2 Further work.................................. 71 ix x List of Figures 2.1 GQM Tree....................................7 2.2 SUS questionnaire............................... 10 3.1 Wii Sports (Nintendo, 2006).......................... 17 3.2 Screenshot of Call of Duty: Black Ops (Activision, 2010).......... 18 3.3 Screenshot of Minecraft (Mojang, 2011)................... 19 4.1 Cloud computing logical diagram [11]...................