Proposal of a Protocol and a Computer Tool for Assessing the Believability of Virtual Players in Multiplayer Video Games Cindy Even
Total Page:16
File Type:pdf, Size:1020Kb
Proposal of a Protocol and a Computer Tool for Assessing the Believability of Virtual Players in Multiplayer Video Games Cindy Even To cite this version: Cindy Even. Proposal of a Protocol and a Computer Tool for Assessing the Believability of Virtual Players in Multiplayer Video Games. Computer Science [cs]. ENIB, 2019. English. tel-02015659 HAL Id: tel-02015659 https://hal.archives-ouvertes.fr/tel-02015659 Submitted on 12 Feb 2019 HAL is a multi-disciplinary open access L’archive ouverte pluridisciplinaire HAL, est archive for the deposit and dissemination of sci- destinée au dépôt et à la diffusion de documents entific research documents, whether they are pub- scientifiques de niveau recherche, publiés ou non, lished or not. The documents may come from émanant des établissements d’enseignement et de teaching and research institutions in France or recherche français ou étrangers, des laboratoires abroad, or from public or private research centers. publics ou privés. THESE DE DOCTORAT DE L’ECOLE NATIONALE D'INGENIEURS DE BREST COMUE UNIVERSITE BRETAGNE LOIRE ECOLE DOCTORALE N° 601 Mathématiques et Sciences et Technologies de l'Information et de la Communication Spécialité : Informatique Par Cindy EVEN Proposal of a Protocol and a Computer Tool for Assessing the Believability of Virtual Players in Multiplayer Video Games Thèse présentée et soutenue à Plouzané, le 28 Janvier 2019 Unité de recherche : Lab-STICC, CNRS UMR 6285 Thèse N° : (8) Rapporteurs avant soutenance : Odile LIMPACH Professor, Cologne Game Lab Daniel MESTRE Professeur des universités, Université Aix-Marseille Composition du Jury : Président : Prénom Nom Fonction et établissement d’exercice (9) (à préciser après la soutenance) Examinateurs : Odile LIMPACH Professor, Cologne Game Lab Daniel MESTRE Professeur des universités, Université Aix-Marseille Ronan CHAMPAGNAT Maître de conférences HDR, Université de La Rochelle Fred CHARLES Principal lecturer, Bournemouth University Dir. de thèse : Cédric BUCHE Professeur des universités, École nationale d'ingénieurs de Brest Co-encadrante : Anne-Gwenn BOSSER Maître de conférences, École nationale d'ingénieurs de Brest Invité(s) Julien SOLER Ingénieur de recherche, Virtualys PREAMBLE This thesis was carried out at the European Center for Virtual Reality (CERV) as part of a CIFRE agreement. The CIFRE (Conventions Indus- trielles de Formation par la Recherche) system was set up to contribute to the innovation process of French companies and to their competitiveness. It also encourages exchanges between public research laboratories and socio-economic circles. The CIFRE agreement involves four actors: — The PhD student who only intervenes on the research work (here: myself, student of the Ecole National d’Ingérieurs de Brest (ENIB) and the Doctoral School Ed-Mathstic), — The company that recruits the PhD student in order to entrust him with a strategic research mission (Virtualys), — The academic research laboratory which supervises the work car- ried out by the PhD student (Lab-STICC, CNRS - UMR 6285), — And the National Agency for Research and Technology (ANRT), which arranges the contract with the company. Virtualys is a software engineering and R&D company specialising in virtual reality, interactive 3D and new web technologies. Founded in 1997 by the association of four newly graduated engineers and 13 associates (mainly teacher-researchers), whose will was to enhance the research work of the CERV (European Center for Virtual Reality) which is the com- puter science laboratory of the National School of Engineers of Brest (ENIB). Draft : December 17, 2018 iii CONTENTS Contentsv List of Figures vii List of Tables viii Acronyms ix Glossary xi 1 Introduction1 1.1 Different Types of Bots ...................... 2 1.2 Believability of virtual players................... 8 1.3 Contribution............................. 9 1.4 Manuscript Organisation ..................... 10 2 Related Works 13 2.1 Defining Believability........................ 14 2.2 Assessing Believability ...................... 15 2.3 Assessment’s Characteristics Analysis............. 21 2.4 Discussion.............................. 29 2.5 Conclusion ............................. 31 3 Blinding the Judges 33 3.1 Model................................. 34 3.2 Implementation........................... 36 3.3 Experiment Methodology..................... 46 3.4 Results................................ 47 3.5 Discussion.............................. 48 3.6 Conclusion ............................. 49 4 Influence of the Judges’ Expertise 51 4.1 Model Modifications........................ 52 4.2 Experiment Methodology..................... 57 4.3 Results................................ 61 4.4 Discussion.............................. 66 Draft : December 17, 2018 v Contents 4.5 Conclusion ............................. 69 5 Reporting Suspected Cheaters 71 5.1 Model................................. 72 5.2 Experiment Methodology..................... 76 5.3 Results................................ 77 5.4 Discussion.............................. 80 5.5 Conclusion ............................. 82 6 Conclusion and Future Work 85 6.1 Conclusion ............................. 85 6.2 Future Work............................. 87 6.3 Publications............................. 88 A Unreal Tournament 2004 Tutorial (in French) 91 B Questionnaire for the Experiment No. 1 (in French) 95 C Final Questionnaire for the Experiment No. 1 (in French) 97 D Questionnaire to Evaluate the Level of Expertise in the Ex- periment No. 2 (in French) 99 E Material for the Experiment No. 3 (in French) 101 E.1 Pre-Experiment Questionnaire..................101 E.2 Screenshot of the Experiment in Process . 103 E.3 Post-Experiment Questionnaire . 104 Bibliography 107 vi LIST OF FIGURES 2.1 Illustration of the protocol used for the different versions of the BotPrize.................................. 18 2.2 Screenshot of the Knowxel mobile application used for the judging process in (Llargues Asensio et al., 2014)- Transla- tion: In your opinion, the player is a: Bot. Human. You do not know.................................... 26 3.1 Questionnaire translated from French ............... 37 3.2 System architecture of the UtBotEval system........... 39 3.3 UML class diagram of the UtBotEval Framework......... 40 3.4 Screenshot of one of the web application pages viewed by a participant................................ 45 3.5 Physical arrangement for the experiment ............. 46 3.6 A 100% stacked bar chart with the responses of the final ques- tionnaire................................. 49 4.1 Screenshot of the DM-Gael map from UT2004.......... 59 4.2 Bar plot of the match durations (in minutes) depending on the maps................................... 63 4.3 Bar plot of the humanness score for (a) bots and (b) humans depending on the maps. ....................... 64 4.4 Bar plot of the humanness score for (a) bots and (b) humans depending depending on the match durations (in minutes). 65 4.5 Correspondence analysis factor map................ 67 4.6 Correspondence analysis factor map................ 68 4.7 Multiple correspondence analysis plot for dimensions 1 and 2 . 68 5.1 Reporting form from the video game Fortnite........... 73 5.2 Ban notification from the video game Fortnite........... 73 5.3 Screenshot illustrating the position of the windows: on the left the video game UT2004, on the right the reporting form opened in a browser (see section E.2 for a larger image) . 74 5.4 Mean number of reports depending on the condition . 78 5.5 Mean humanness score for bots depending on the condition . 79 5.6 Negative correlation between the bots’ estimated humanness and the number of reports in the control group.......... 80 Draft : December 17, 2018 vii LIST OF TABLES 2.1 Framework for evaluating the believability of characters from (Hinkkanen et al., 2008)........................ 20 2.2 Comparison of the existing experiments.............. 22 3.1 Passing order of participants..................... 48 4.1 Competition results .......................... 62 4.2 Distribution of the devices usually used to play according to the level of expertise (in percentage)................ 65 4.3 Distribution of the type of players usually met in games ac- cording to the level of expertise (in percentage) ......... 66 viii ACRONYMS AFK Away From the Keybord 72 AI Artificial Intelligence2,9, 13, 15, 28, 32 ECA Embodied Conversational Agent 14 FPS First Person Shooter3,5, 16, 19, 23, 25, 30, 33, 69, 74, 83, Glos- sary: FPS LAN Local Area Network5, 45 MITM Man-in-the-middle attacks3 MMO Massively Multiplayer Online game4, Glossary: MMO MOBA Multiplayer Online Battle Arena4, Glossary: MOBA NPC Non-Player Character4–6,8, Glossary: NPC UT2004 Unreal Tournament 2004 15–17, 23, 34, 38, 39, 41, 46, 49, 53, 58, 72, 76 Draft : December 17, 2018 ix GLOSSARY chatbot or chatterbot, is a computer program designed to simulate con- versation with human users, especially over the Internet. 15 FPS A First Person Shooter is a video game genre where "first person" refers to the point of view : the player assumes the field of vision of the protagonist, so that the game camera includes the character’s weapon, but the rest of the character is not seen. "Shooter" refers to the action : the game interaction largely involves moving, aim- ing, and shooting a gun. First-person shooter games first became popular with the release of Doom in 1993.3, griefer (or grief player) A player who derives his/her enjoyment not from playing the game, but from performing actions that cause grief