Personalized Game Reviews Information Systems and Computer
Total Page:16
File Type:pdf, Size:1020Kb
Personalized Game Reviews Miguel Pacheco de Melo Flores Ribeiro Thesis to obtain the Master of Science Degree in Information Systems and Computer Engineering Supervisor: Prof. Carlos Ant´onioRoque Martinho Examination Committee Chairperson: Prof. Lu´ısManuel Antunes Veiga Supervisor: Prof. Carlos Ant´onioRoque Martinho Member of the Committee: Prof. Jo~aoMiguel de Sousa de Assis Dias May 2019 Acknowledgments I would like to thank my parents and brother for their love and friendship through all my life. I would also like to thank my grandparents, uncles and cousins for their understanding and support throughout all these years. Moreover, I would like to acknowledge my dissertation supervisors Prof. Carlos Ant´onioRoque Martinho and Prof. Layla Hirsh Mart´ınezfor their insight, support and sharing of knowledge that has made this Thesis possible. Last but not least, to my girlfriend and all my friends that helped me grow as a person and were always there for me during the good and bad times in my life. Thank you. To each and every one of you, thank you. Abstract Nowadays one way of subjective evaluation of games is through game reviews. These are critical analysis, aiming to give information about the quality of the games. While the experience of playing a game is inherently personal and different for each player, current approaches to the evaluation of this experience do not take into account the individual characteristics of each player. We firmly believe game review scores should take into account the personality of the player. To verify this, we created a game review system, using multiple machine learning algorithms, to give multiple reviews for different personalities which allow us to give a more holistic perspective of a review score, based on multiple and distinct players' profiles. Keywords digital game, machine learning, player model, review system. iii Resumo Atualmente, a forma mais comum de avalia¸c~aode jogos ´eatrav´esde reviews de jogos. Estas s~aoan´alises cr´ıticas,com o objetivo de dar informa¸c~aosobre a qualidade dos jogos. Enquanto que a experi^enciade um jogo ´einerentemente pessoal e diferente para cada jogador, as abordagens atuais para a avalia¸c~ao desta experi^encian~aot^emem considera¸c~aoas caracter´ısticasindividuais de cada jogador. N´osacreditamos veemente que as pontua¸c~oes das reviews dos jogos s~aoinerentes `apersonalidade do jogador. Para verificar isto, n´oscri´amosum sistema de review de jogos, usando m´ultiplosalgoritmos de aprendizagem, que modela reviews para diferentes personalidades que nos permitem dar uma perspetiva mais hol´ısticade uma pontua¸c~aode review, baseado em m´ultiplose distintos perfis de jogador. Palavras Chave jogo digital, aprendizagem de m´aquina,modelo de jogador, sistema de review. v Contents 1 Introduction 1 1.1 Motivation............................................2 1.2 Problem..............................................3 1.3 Hypothesis............................................3 1.4 Contributions...........................................4 1.5 Document Outline........................................4 2 Related Work 7 2.1 Game Reviews..........................................8 2.1.1 Aggregators........................................ 10 2.1.2 Platforms for digital mobile distribution........................ 10 2.1.3 Platforms for digital computer distribution...................... 10 2.1.4 Entertainment websites................................. 11 2.1.5 Youtube channels..................................... 12 2.1.6 Game Reviews Summary................................ 12 2.2 Game Genres........................................... 13 2.3 Personality Models........................................ 14 2.3.1 The Five Factor Model................................. 14 2.3.2 Myer-Briggs Type Indicator............................... 14 2.4 Player Models........................................... 15 2.4.1 Bartle Player Types................................... 16 2.4.2 Quantic Foundry's Gamer Motivation Profile..................... 17 2.4.3 Demographic Game Design............................... 19 2.4.4 BrainHex......................................... 20 2.5 WEKA.............................................. 21 2.6 Explored Machine Learning Algorithms............................ 23 2.6.1 Instance-Based Algorithms............................... 23 2.6.2 Regression Algorithms.................................. 25 2.6.3 Tree Algorithms..................................... 28 2.7 Credibility............................................. 29 2.7.1 Bootstrap......................................... 30 2.7.2 Cross-validation..................................... 30 2.8 Discussion............................................. 31 3 Methods and Procedures 33 3.1 Measuring Tools......................................... 34 3.2 Approaches............................................ 34 3.3 Methodology........................................... 35 3.3.1 User Questionnaire.................................... 36 3.3.2 Game Filtering...................................... 37 3.3.3 Dataset Filtering and ARFF Preparation....................... 38 vii 3.3.4 Algorithm Filtering................................... 40 3.3.5 Review System...................................... 41 3.3.6 Review System Validation................................ 42 3.4 Metrics.............................................. 42 3.5 Discussion............................................. 42 4 Results 43 4.1 Demographic Results....................................... 44 4.2 Game Filtering Results...................................... 46 4.3 Dataset Filtering Results.................................... 48 4.4 Review System Training Results................................ 48 4.4.1 Linear Regression Algorithm Training Results..................... 48 4.4.2 K-Nearest-Neighbor Algorithm Training Results................... 52 4.4.3 Multilayer Perceptron Algorithm Training Results.................. 53 4.4.4 M5P Algorithm Training Results............................ 54 4.5 Review System Validation Results............................... 56 4.5.1 Linear Regression Algorithm Validation Results.................... 56 4.5.2 K-Nearest Neighbor Algorithm Validation Results.................. 56 4.5.3 Multilayer Perceptron Algorithm Validation Results................. 57 4.5.4 M5P Algorithm Validation Results........................... 57 4.5.5 Algorithms Validation Results Discussion....................... 58 4.6 Discussion............................................. 59 5 Conclusions 61 5.1 Discussion............................................. 62 5.2 Future Work........................................... 62 A Game Genres 71 B Selected Digital Games 73 C User Questionnaire 1 77 D User Questionnaire 2 85 viii List of Figures 1.1 GameRankings's Grand Theft Auto V review score of 96.33 in a scale from 0 to 100 (Game Rankings, n.d.)...........................................3 2.1 FIFA 18 review score of 7 by GameSpot specialists (GameSpot, 2017)............8 2.2 Pokemon GO review score of 7 by GameSpot specialists (GameSpot, 2016).........8 2.3 Pokemon GO user's review score of 4.1 on Google Play (Google Play, n.d.)......... 10 2.4 Grand Theft Auto V with 70% positive reactions and both graphs with all user's reactions since its release date and the past month (Steam, 2015).................... 11 2.5 Player interest graph with two coordinates creating four distinct quadrants: Achiever, Explorer, Socialiser, Killer (Bateman, 2009)........................... 16 2.6 Quantic Foundry's Gamer Motivation Model with the three high-level motivations and its sub-categories (Quantic Foundry, 2016)............................. 17 2.7 The four player typologies from Demographic Game Design Model: Conqueror, Manager, Wanderer and Participant, each one divided into casual gamers and hardcore gamers (Dias & Martinho, 2010)......................................... 20 2.8 BrainHex model with the seven archetypes, each related to a style of play (BrainHex, n.d.). 21 2.9 Attribute-Relation File Format (ARFF) file used in our work. The relation Grand Theft Auto V Review with the list of seven plus one numeric attributes and a list with some of the instances... 22 2.10 A K-Nearest-Neighbor (KNN) example with two distinct features where a new instance is being evaluated within the three nearest neighbors (Punch III et al., 1993)......... 24 2.11 A Linear Regression (LR) graph in a system with two dimensions, where the red line represents the value obtained from all instances represented as blue dots.......... 26 2.12 Sigmoid Function graph where is possible to see that is a function not completely differ- entiable in the middle values (Witten et al., 2016)....................... 27 2.13 Tenfold cross-validation repeated ten times where the white squares represent the training dataset and the orange squares the validation dataset..................... 31 3.1 Two approaches to give personalize review scores........................ 34 3.2 Flow chart of the six methodology steps. It starts from the User Questionnaire, followed by Game Filtering, and Dataset Filtering and ARFF Preparation. From it, appears the Algorithm Filtering process, and the Review System. Ending the pipeline methodology with the Review System Validation step............................. 35 3.3 Header part of an ARFF file. It contains the relation Grand Theft Auto V Review System, the list of seven plus one numeric attributes which are the seven archetypes of the user's BrainHex model and the game review score..........................