Making the Game Wiki Obsolete Improving Approachability in Grand Strategy Games

Timmy Eklund

Department of Applied Physics and Electronics January 2020 Umeå University, Sweden [email protected] 2 Timmy Eklund

Abstract

The aim of this thesis is to explore and investigate the possibility of improve- ments in approachability and player retention, in Grand Strategy games. This is important for game developers to be able to develop games more suited to a wider audience. This study will investigate whether these improvements can be made by giving players access to information, traditionally found on game wikis, inside the game in a non-intrusive manner. This was investigated through a pre-study of current trends in gamers usage of information resources outside the games themselves. It was determined that wikis are the most common resource, and that they are most commonly used in the middle of play. Because of this, three mock-ups of user interfaces, which could supply such information to the player in the midst of playing, were designed. They were then evaluated by a panel of experts, consisting of game developers and experienced Grand Strategy players. The mock-up deemed by the experts to have the best potential, a windowed in-game encyclopedia, was implemented as a prototype in a small game. This game was developed to emulate the gameplay, and positive and negative aspects of Grand Strategy games. This prototype was then tested using A/B-testing. Despite a low number of participants, the results showed that access to informa- tion found on wikis in the game does lead to better approachability. It can also be concluded that players are closer to reaching a Flow state, which will lead to potentially higher player retention. Making the Game Wiki Obsolete 3

Sammanfattning

Syftet med denna uppsats är att utforska och undersöka möjliga förbättringar inom användarvänlighet och bevarande av spelares intresse, inom spel i genren Grand Strategy. Detta är viktigt för spelutvecklare, för att lättare kunna utveckla spel som passar en bredare publik. Denna studie kommer att undersöka huruvida sådana förbättringar kan uppnås genom att ge spelare tillgång till information, som traditionellt sett finns på wiki-sidor, inuti spelet, på ett sätt som inte stör spelaren. Detta undersöktes med hjälp av en förstudie av trender inom spelares använd- ning av information från resurer utanför spelen. Den vanligaste resursen visade sig vara wiki-sidor. De används framförallt under tiden som man spelar. Baserat på detta skapades tre förslag på användargränssnitt som kan tillhandahålla infor- mation till spelaren medan de spelar. De utvärderades därefter av en expertpanel, som bestod av spelutvecklare och erfarna spelare inom Grand Strategy genren. En encyklopedi i ett fönster inuti spelet blev det förslag som ansågs ha bäst potential. En prototyp av denna encyklopedi implementerades i ett litet spel. Detta spel hade utvecklats för att efterlikna spel inom Grand Strategy genren. Både de positiva och de negativa aspekterna efterliknades. Prototypen testades därefter med A/B-testning. Trots ett lågt antal deltagare visade resultaten att tillgång till information från wiki-sidor inuti spelet ledde till bättre användarvänlighet. Slutsatsen kan även dras att spelare var närmare att nå ett Flow-state medan de spelade, vilket potentiellt kan leda till att spelares intresse bevaras och vill fortsätta spela spelet. 4 Timmy Eklund

Acknowledgement

First and foremost, I would like to express my deep gratitude to everyone at Paradox Arctic for the warm welcome and inclusion they’ve shown me. I would especially like to thank Mikhail Gofman, Jesper Blomqvist and everyone else on the team that has been my home for the duration of my thesis. They’ve supported me, inspired me and helped me grow, both professionally and as a person, and really made me feel like part of their family. I could not have asked for a better place to write my thesis! Secondly, I would like to thank Simon Edman and Adam Nording for the great input and suggestions, and lively discussions during our peer reviews, which has significantly increased the quality of this thesis. I would also like to thank Tomas Nordström, my supervisor at the University, for his academic help on what makes a good thesis and how to write better papers. Furthermore, I would like to thank my family and friends, who have supported me during these months, either indirectly through words of encouragement, or more directly by participating in my surveys and tests. And last, but not least, thank you, Caroline, for encouraging me, and helping me see reason, during my most difficult and frustrated moments. Table of Contents

1 Introduction ...... 6 1.1 Background ...... 6 1.2 Paradox Interactive & Paradox Arctic ...... 7 2 Objective ...... 9 2.1 Purpose ...... 9 3 Theoretical Framework ...... 10 3.1 Grand Strategy Games ...... 10 3.2 Flow...... 10 3.3 Game Approachability Principles (GAP) ...... 12 3.4 Wikis...... 13 3.5 Tooltips ...... 14 4 Methodology ...... 16 4.1 Pre-Study ...... 16 4.2 Mock-up exploration ...... 16 4.2.1 Mock-up 1: Expanding tooltips ...... 17 4.2.2 Mock-up 2: Fullscreen in-game encyclopedia ...... 17 4.2.3 Mock-up 3: Windowed in-game encyclopedia ...... 18 4.2.4 Testing of mock-ups ...... 18 4.3 Implementation and Evaluation ...... 18 4.3.1 Pseudo-game ...... 19 4.3.2 Pseudo-game development ...... 20 4.3.3 Prototype ...... 21 4.3.4 User testing ...... 22 5 Results ...... 24 5.1 Pre-Study ...... 24 5.2 Mock-up exploration ...... 25 5.3 Implementation and Evaluation ...... 27 6 Discussion ...... 31 6.1 Limitations ...... 33 7 Conclusion ...... 35 7.1 Future Work ...... 36 8 Appendix ...... 40 A Pre-study - Survey Questions ...... 40 B Mock-ups ...... 42 6 Timmy Eklund

1 Introduction

The video-game industry is booming [1], and as the market grows, so does its diversity. Today, a myriad of different video-game genres exists, ranging from casual brainteasers, to intense war simulations, and engaging fantasy role playing games. This diversity is not limited to just genres, but also to difficulty. Some players desire difficult videogames that takes years to master, while others prefer games that can be picked up and enjoyed immediately [2]. But many of the game-developers of these tough games also want to invite new players to their games, without scaring them off with how challenging such games are to learn to play. Such an unfortunate scenario is, however, not uncommon, especially in a certain genre of games called Grand Strategy games [3]. These are games in which the player controls a nation and through various complex economic, military, and diplomatic means, seek to expand and help your nation prosper [4]. Such games can prove to be very intimidating to new players, due to their high complexity, which tends to result in poor approachability [3]. As a result, many players, both beginners and veterans, supposedly seek help and guidance from various internet resources when they play these games, but this results in a halt in the playing of the actual game, which has a negative impact on the player experience, according to game developers at a studio named Para- dox Arctic. These developers asked a question. Can the player experience be improved, by instead providing these types of resources inside the actual game experience? This thesis will be an exploratory study on whether providing such information resources in-game could potentially improve approachability, and as a result, retention of players.

1.1 Background Nearly a third of the world’s population plays video games, or have at least tried it out, as in 2016 it was estimated that approximately 2.5 billion people play games worldwide [1]. The video game industry is ever-expanding and is slowly but surely out-performing traditional media entertainment like the film industry. Just recently Avengers: Endgame by Marvel Studios set a new record for highest-grossing film of all time (not adjusted for inflation) by earning almost 2.8 billion USD [5]. Compare this to League of Legends, one of the world’s most popular video games, which in the same year earned 1.5 billion USD, a feat which they repeat yearly [6]. In 2019 the total global box office revenue was 42.5 billion USD [7]. The video game industry reportedly earned 120.1 billion USD in the same year [6]. Video games as a form of entertainment have become so prevalent and influential in society and pop culture that vintage video games hold the same prestige that only classic films used to [8]. Just like in the film industry, there are a great many types of different game genres, all of which attracts different types of players. Some prefer the skill ex- pression possible in First Person Shooters where careful precision with a mouse Making the Game Wiki Obsolete 7 or control stick is required to show off your mettle. Others prefer the brainteasers of puzzle games. And others still prefer immersing themselves in a compelling story of good and evil, knights and princesses, and dragons and caves in a story heavy Role Playing Game. But some players prefer the strategical mastery of military, economics and politics which are the core elements the genre of games called Grand Strategy. Simply put, a Grand Strategy game is a Real Time Strat- egy game where you as the player control the destiny of a nation or empire trying to take its place in the world [4]. For many the appeal of the games are the his- torical fantasies they have a chance to reenact. What if the Roman Empire never fell? What would have happened to the world if Germany proved unstoppable during World War 2? Or maybe you just want to see your favorite ancient coun- try rule the world. Whatever your motivation, the games have an almost infinite replayability as the games are based on systematic rules and mechanics which challenge the player, rather than a restrictive narrative which would force the player to follow a certain path. You can choose to do anything you please with your nation of choice [4]. This replayability means that the players who enjoy this genre tend to never really stop playing the games. They may play them more or less from time to time, but they always return. Grand Strategy games like Europa Universalis IV [9], Hearts of Iron IV [10], Civilization V [11] and Civilization VI [12] all hold a steady place in the top 50 games by player count on Steam [13]. And they are likely to stay there for a long time. Grand Strategy games are one of few game genres that require an extensive amount of memorized information, micro-management skills and lots of experi- ence in the chosen game in order to fully experience and enjoy the games. This can be quite an obstacle for new players to overcome. Some might be scared off before they even have a chance to see if they enjoy the game [3]. Is this something that can be remedied? Is there a way to help new players learn, at their own pace, by giving them the tools to look up the information they lack, without forcing them to rely on external help such as online wikis or other third-party resources?

1.2 Paradox Interactive & Paradox Arctic

Paradox Interactive is a Swedish game publisher, with several internal game development studios. The primary studio located in Stockholm, Paradox De- velopment Studio, was founded in 1997 and is one of the world’s most popular strategy game developers. They are responsible for widely known strategy fran- chises, such as Europa Universalis [9], Crusader Kings [14], Hearts of Iron [10] and Stellaris [15]. Paradox Arctic is a Swedish game development studio. The studio was founded in 2014 and is one of the internal studios owned by Paradox Interactive. Throughout the years the studio has done work on Magicka 2 [16], Pillars of Eternity [17], as well as minor work on all the major Paradox games such as Stellaris [15], Europa Universalis IV [9] and many more. The studio is currently focusing on creating 8 Timmy Eklund

Deep, Appealing and Personal games in the Management genre, with a flair of the complex strategy of traditional Paradox Interactive games. Making the Game Wiki Obsolete 9

2 Objective

In order to investigate whether the initial player experience in Grand Strategy games could be improved, by giving them better access to information on game mechanics and game rules in-game, the following primary and secondary research questions needed to be answered: Primary a) Investigate if providing detailed information about game mechanics—in-game, at the players leisure—will improve game approachability. Secondary b) Investigate if this access to detailed information could improve player reten- tion c) Determine the best way to provide this detailed information to the players in a non-intrusive way that does not frustrate or overwhelm the player.

2.1 Purpose The purpose of the paper was to investigate the possible advantages of providing detailed information on game mechanics in-game, rather than outside the game. The hypothesis was that it would improve approachability, that is, reducing the gate-keeping of new players by improving the learning curve, which in turn would lead to a higher player retention as frustrated players are more prone to quit the game. 10 Timmy Eklund

3 Theoretical Framework

In order to investigate the objectives of potentially improving approachability and player retention, information first had to be gathered on relevant subjects; Grand Strategy games and how they work, how to measure player enjoyment and approachability, and how the resources players currently use works.

3.1 Grand Strategy Games

A Grand Strategy Game, sometimes abbreviated as GSG, is a game, in the strategy genre of games, which focuses on the idea of grand strategy, which is a form of government planning and is an advanced way for a country or nation to plan for fulfilling a long-term goal [18]. In order to reach said goal it is necessary to identify a national objective and assess the resources available to the nation such as “military prowess, political leverage, diplomatic ability, and economic might”[18] and utilize those resources to their fullest potential. In other words, Grand Strategy Games lets the player take control of nation or country, real or fictional, and guide it as they see fit. The player can, among other things, declare war on neighboring nations, expand and develop cities, engage in diplomatic relations with other countries, research technology or build trade routes [4]. The end goal of the game is of the players choosing, as they are in full control of what they want their nation to do, but a common goal is world domination. Grand Strategy Games are often set during historic periods—such as the time of the roman empire, medieval times, or the Victorian era—in which the player is able to bend and shape history to their will [4]. Grand Strategy Games have a lot of overlap with other similar game genres such as the very closely related genre of 4X games—games in which the player takes control of a character or society which has to EXplore, EXpand, EXploit and EXterminate in order to survive in the world[19]—or the tangentially related genre management games—games which focuses on caring for and maintaining a system. For example, The Sims[20], where the player has to care for and help a household and its inhabitants flourish, or Prison Architect[21], where the player takes on the role of prison warden and has to build and maintain a successful maximum security prison. However, while overlap do exist, not all of these games belong to the Grand Strategy Genre, even though the lines which separate them may sometimes be blurry. Some examples of games that do belong to the Grand Strategy Game genre are: – Europa Universalis IV[9] – Hearts of Iron IV[10] – Civilization V & VI[11][12]

3.2 Flow

In his book, Flow: The Psychology of Optimal Experience, from 1991, Mihaly Csikszentmihalyi talks about the psychological phenomenon he refers to as Flow Making the Game Wiki Obsolete 11

[22]. Csikszentmihalyi explains that what he refers to as Flow is a mental state in which an individual experiences a strong intrinsic motivation and satisfaction with their current activity. This results in the individual feeling that the activity is so rewarding in and of itself that they desire to do it, without external rewards, even if the activity is highly difficult or dangerous. While in a state of Flow, the person experiencing the phenomenon loses their sense of reflective self-consciousness and experiences time subjectively different (in the sense that time may feel like it is passing by more quickly. “Time flies when you’re having fun.” [23]). The phenomenon is generally a positive experience as Csikszentmihalyi explains that experiencing Flow in everyday activities is the key to living a happier life [22]. However, while the subject has not been extensively researched, the Flow state may potentially also inadvertently lead to addiction to the activity the Flow is experienced in, due to the high level of control and rewarding feelings the person experiences. Video games could be called what Csikszentmihalyi refers to as an autotelic ex- perience; “from two Greek words, auto meaning self, and telos meaning goal.”[22]. The user does not perform the activity as to reach an external goal. The goal is the enjoyment of the activity itself; the main criteria for achieving Flow. In order to reach Flow, eight criteria needs to be fulfilled according to Csikszent- mihalyi, as compiled by Penelope Sweetser and Peta Wyeth[24]: – a task that can be completed; – the ability to concentrate on the task; – that concentration is possible because the task has clear goals; – that concentration is possible because the task provides immediate feedback; – the ability to exercise a sense of control over actions; – a deep but effortless involvement that removes awareness of the frustrations of everyday life; – concern for self disappears, but sense of self emerges stronger afterwards; and – the sense of the duration of time is altered. In 2005, Sweetser and Wyeth published a paper on an evaluation model for the enjoyment of video games, utilizing these eight criteria for Flow as heuristics [24]. They call this model GameFlow and showed that it can be used to measure players enjoyment of games. One of the heuristics is that “challenges in games must match the players’ skill levels”[24] even if the player’s skill increases. If there is a mismatch the player may experience frustration, if the difficulty is too high or boredom if the difficulty 12 Timmy Eklund is too low, compared to the player’s skill level. The balance between difficulty and skill is illustrated in Fig.1.

Fig. 1: The relationship between Flow, and user skill and activity difficulty.

Flow is also one of the main reasons players keep playing a chosen game, as con- cluded by Lee and Tsai in their 2010 paper “What Drives People to Continue to Play Online Games? An Extension of Technology Model and Theory of Planned Behavior” [25]. Their research was specifically on Online games, but could, in part, be attributed to other games as well. Aside from Flow, they also state that perceived enjoyment and perceived behavioral control (choosing to go through with an action or behavior, or not, based on whether it is believed it can be done successfully) are important factors in player retention.

3.3 Game Approachability Principles (GAP) The Game Approachability Principles, abbreviated as GAP, are a set of guide- lines or heuristics proposed by Desurvire and Wiberg, in their study on game approachability [26]. These guidelines are meant to help game developers im- prove approachability in their games, by providing a set of heuristics by which approachability can be measured. Desurvire and Wiberg define Game Approach- ability as the concept of how easily an inexperienced player can learn the tools Making the Game Wiki Obsolete 13 and techniques necessary to play a game, by keeping them “sufficiently moti- vated” [26], and in doings so making sure that the player gains enough knowledge to continue playing the game, but keeping the player curious in order to motivate continued interest. They make a point that the game audience is shifting. Play- ers, traditionally, used to consist of solely hardcore gamers. However, in recent times, games have become so mainstream and popular that game developers can no longer expect an expert end-user when designing their products [26]. More often than not the end-user will be a casual or inexperienced player which needs to be taught how to play the game in such a manner that the player does not lose interest due to game being perceived as having a too high skill floor [26][27]. One of the principles proposed by Desurvire and Wiberg is “Information On Demand and In Time, System Thinking” [26], which is a measurement of if the “player has access to answers regarding the game whenever needed when first coming across new material” [26]. This principle is based on the research by James Paul Gee. In his research article Learning by Design: Good Video Games as Learning Machines from 2005 Gee analyzes how and why those who play games are so intrinsically motivated to learn (in order to be able to play the game) and how it might be applied to traditional learning environments like schools [28]. One of the many principles he defines is the one called “Information ‘On Demand’ and ‘Just in Time’ ”[28]. This principle states that humans are quite inefficient at absorbing verbal information when received out of context and in large amounts. It is preferable to receive the specific information in the situation when it is needed. Gee draws a comparison to Game Manuals and how they “just like Science textbooks, makes little sense if one tries to read them before having played the game”. In other words, without the proper context and timing, the mind is unable to properly absorb verbal information and instruc- tions, as the mind is unable to insert the information in a proper mental model in which the information is supposed to be used. Gee exemplifies this with a comparison to a phenomenon where students who are straight A students in class and during tests are unable to apply what they have learned in practice [28].

3.4 Wikis

A wiki is an internet website which can “be modified or contributed to by users.” [29] The concept of a wiki was first created by Ward Cunningham, in 1995, when he created a website called WikiWikiWeb, a website which could be edited by its users. The name comes from the Hawaiian word “quick” [29]. The most commonly used wiki is most likely Wikipedia; an online encyclopedia based on the wiki concept of user-generated and user-moderated content [30]. Wikis are often faced with a lot of criticism due to their nature. Many call in to question the fact that anyone can edit the contents of a wiki page and argue that there will always me malicious users who will try to undermine the efforts of the rest of the editors by “introducing false or misleading content” [29]. However, the supporters of wiki software argue that the community as a whole is a stronger 14 Timmy Eklund force than the malicious few and as such the community will always correct any false information. And although the system is not in any way foolproof, it stands as a testimony to the goodness of people [29]. In his article, from 2014, Matthew Barr explains that wikis might be one the most significant social media platforms for gamers, because of the scholarly, almost academical, work required to maintain and add on to the sites [31]. This implies that with a large enough dedicated user base of contributors, a wikis articles are likely to reach a very high standard of quality. However, wikis suffer from a problem called participation inequality [32]; only a small number of users actually contribute to wikis. Sometimes also referred to as the “1% rule”, because of the distribution of users: 1% of users are active key contributors and 9% are intermittent contributors, but 90% of users are lurkers—users who only read and never contribute. This means that in order to have a wiki with up-to-date information, and a high-quality standard of articles, a very large user base is needed.

3.5 Tooltips

A tooltip is an element commonly found in most GUIs (graphical user interfaces) which displays a descriptive text of the element the mouse pointer is currently hovering over, in order to help the user understand a possibly ambiguous icon or button [33]. In order to solve the issue of their players having to rely on a third-party wiki website, Riot Games, developer of the video-game League of Legends, experi- mented with providing the more detailed type of information, that players were looking up on the wiki, through expandable tooltips [35]. The benefit of this was that the non-expanded tooltip could be made more simple in order to provide faster “In the moment” information; information you need to be able to read through and understand quickly [35], as can be seen in Fig.2a. At the same time, the expanded tooltip could provide players who needed more expert-level information the data they wanted, such as number data needed to make deci- sions for min-maxing [36], deciding which strategies to employ, planning on how to approach their next fight ahead of time, etc. [35], which can be seen in Fig.2b. Making the Game Wiki Obsolete 15

(a) Regular tooltip.

(b) Expanded tooltip.

Fig. 2: Tooltips for abilities (character skills) as seen in the video-game League of Legends (Riot Games, 2020). Screenshot by author. [34] 16 Timmy Eklund

4 Methodology

As not much is known on whether players actually use outside resources when they play Grand Strategy games, a pre-study was performed to gauge if players use such resources and how often they do it. Afterwards, three mock-ups were designed, based on current industry standards for how to supply players with information, such as tooltips and in-game ency- clopedias. These mock-ups were then evaluated by experts on Grand Strategy games. Based on the result of the expert evaluation, one of the mock-ups were selected for implementation and player testing. The player testing of this implementation was performed in a small game, de- veloped at the same time as the pre-study and mock-up exploration stages. The selected mock-up was implemented as a prototype in this game and then tested on two groups of players using A/B-testing, where one group was a control group without the prototype. The results of the test were evaluated based on whether the player could complete the game scenario or not, how quickly they did so, how often they utilized the prototype, as well as the testers own thoughts on how well they did and how useful they found the prototype.

4.1 Pre-Study

In order to properly evaluate the potential of improving in-game information sources available to players, the current state of players usage and need of such sources needed to be investigated. A survey was sent out measuring current trends and usage of outside resources—such as website, wikis, YouTube, strategy guides, etc.—during gameplay, as well as general statistics on enjoyment and approachability. The target audience of the survey was primarily gamers, but preferably with a focus on gamers who primarily play Grand Strategy games. The pre-study survey was published on a private Facebook group of university students, currently enrolled and alumni, in interaction design at Umeå Univer- sity, “ParadoxPlaza”[37], the sub-page dedicated to games published by Paradox Interactive on the social internet platform Reddit [38], as well as “SampleSize”[39] a sub-page dedicated to surveys on the same website. These platforms were cho- sen as they were most likely to produce participants in the target audience, but still on a broader spectrum of that audience, that is, not only experts of Grand Strategy games, but beginners as well.

4.2 Mock-up exploration

Before implementation and testing of a final prototype, the different ways of giving the player information needed to be evaluated. As it would be impossible to exhaustively test every single method of providing information to the player, Making the Game Wiki Obsolete 17 a few prevalent examples from other games were selected for prototyping and user testing. The methods chosen for prototyping and testing were: – Tooltip expansion to display additional information – Fullscreen in-game encyclopedia pausing play (with and without clickable tooltip links) – Windowed in-game encyclopedia during play (with and without clickable tooltip links) These approaches were chosen as they are, at the time of writing, the most commonly found solution for similar problems in games of other genres. The mock-ups were based on the design, layout and theme already present in Stellaris [15], the game which would serve as background for the mock-ups. Basic design was also based on industry standards for encyclopedias and tooltips. All mock-ups were designed in the prototyping tool Figma [40]. All of the mock-ups opted to not make use of the wiki way of having user generated content, as despite the many pros of a wiki website, the cons were simply too great in the context of being used in a paid product, such as a video- game. The fact that it is impossible for profanity and incorrect, and sometimes even malicious, information to not to sometimes slip through was simply to large to ignore.

4.2.1 Mock-up 1: Expanding tooltips

The first mock-up was based on the experiment done by Riot Games on providing players with additional information by allowing them to expand tooltips [35]. The first mock-up incorporates this ability to expand all tooltips to display more information that may be wanted by an expert player looking for detailed data or a beginner looking to learn more about the game mechanic they do not understand. This mock-up can be seen in Fig.12 and Fig.13 in Appendix B.

4.2.2 Mock-up 2: Fullscreen in-game encyclopedia

The second mock-up incorporates the traditional, and most common, solution for providing large amounts of information in-game: an in-game encyclopedia, also known as a codex, database, or compendium. This encyclopedia is a fullscreen view, which pauses the game to allow for full attention from the user, without making them stressed that they are missing out on things happening in the game. The second mock-up can be seen in Fig.14 in Appendix B. 18 Timmy Eklund

4.2.3 Mock-up 3: Windowed in-game encyclopedia

The third mock-up was designed as a variant on mock-up 2. This mock-up also implemented an in-game encyclopedia, but windowed, rather than fullscreen, as to allow the player to keep playing the game while the encyclopedia was open, to allow having a reference page open in the encyclopedia, while following its instructions to achieve a result in the game. For example; In order to build a farm, the player needs to a) choose or create a builder unit, b) move the builder unit to the desired location, c) command the builder unit to build the specified building (the farm). The reason this idea was chosen for prototyping was due to the results from the pre-study. The pre-study indicated that most players use third-party pages, like wikis, most often during gameplay, rather than before or afterwards. This indicated that players want the information at the same time as they play the game. They do not wish to look up the information afterward. They want to utilize the missing information, in real time, to be able to keep playing the game. As such, it seemed appropriate to allow the player to access the information, without interrupting them from actually playing of the game. The third mock-up can be seen in Fig.16 in Appendix B.

4.2.4 Testing of mock-ups

The mock-ups were all evaluated through user testing by experts. The test-group was a small group of experienced users, mostly consisting of game developers and players with a lot of time spent in Grand Strategy Games. The participants were first shown a typical game scenario and the issue that comes with it. They were then asked how they would like the issue to be solved. They were then shown the same scenario but featuring one of the three mock- ups. The participants were then asked what they liked about the mock-up, what they disliked about it, if the mock-up solved the problem, and how they generally felt about this solution, giving it a rating from 1-5 in efficiency, satisfaction, and approachability. This was then repeated for the last two mock-ups. Lastly the participants were asked to place the different mock-ups in their order of preference and give a final comment on the test and mock-ups in general. The order in which the mock-ups were presented to the participant was different for each of the participants to not contaminate the results due to possible bias from the participants based on the order of the mock-ups.

4.3 Implementation and Evaluation

In order to evaluate how well the mock-up, chosen for implementation during the mock-up exploration stage, would improve approachability and enjoyment, the mock-up needed to be evaluated in natura. In other words, it needed to be Making the Game Wiki Obsolete 19 tested on a player playing a Grand Strategy game for real. It could not be tested using a mock-up in a lab setting.

This meant that the chosen mock-up needed to be implemented as an actual prototype in a Grand Strategy Game. However, learning to work with, and implement into, the game engine used by the Grand Strategy games developed by Paradox Interactive would be nigh impossible, given the timespan of the research project. As such, a small game that as closely as possible emulated a Grand Strategy game needed to be developed, to be used as a platform for testing the prototype. This was done over the course of the entire research project.

The prototype was then evaluated, quantitatively, through participants play- ing the game once—unsupervised, at a time of their choosing, in their choice of location—and then filling out a survey and attaching the save file of their playthrough to their response. The save file contained a data log on what the participant did, and at what times, measuring milestones and the usage of the prototype.

4.3.1 Pseudo-game

Over the course of the project a pseudo-game was developed in Unity, a game engine, in order to provide a platform for the final implementation and the final tests of the research, as implementing the prototype in a real Grand Strat- egy Game would be too time-consuming. These games often have very complex systems and require a lot of time learning the engine in order to successfully implement solutions in them.

This pseudo-game was instead developed in order to mimic such a game, but pro- vide an easier platform for implementation, as the systems and engine would not be as complex. The pseudo-game was based on the open-source project Mostly Civilized Hex Engine [41], created by Github and Youtube user Quill18 [42].

The intention was for the pseudo-game to simulate the experience of a Grand Strategy Game, in order to evaluate the difference in what a player may expe- rience when playing such a game with or without the systems proposed by this research. Therefore, the pseudo-game was developed to provide an experience as close to a Grand Strategy Game as possible in the given development time. Due to various limitations and design choices the pseudo-game also shares a lot of similarities with a 4X game (EXplore, EXpand, EXploit and EXterminate). However, this was not a problem as many Grand Strategy Games belong to the 4X games genre as well, due to the overlap in the type of gameplay typical for the two genres. The pseudo-game should therefore, despite skirting the genre of 4X games, be considered a very valid platform for the research. A screenshot from the game can be seen in Fig.3. 20 Timmy Eklund

Fig. 3: Screenshot of the finished pseudo-game.

4.3.2 Pseudo-game development

The pseudo-game was, as previously mentioned, based on the open-source project Mostly Civilized Hex Engine. Initially the pseudo-game was developed very closely to the development tutorial for the project, but as the project got farther and farther into the development, the design choices wanted for the pseudo-game was diverging from the open-source project, and eventually the pseudo-games development was completely separate from the Mostly Civilized Hex Engine project.

The big components shared between the two projects were: a) a hex-grid based map, b) unit logic, c) unit pathfinding, and d) the mouse controls.

The key mechanics needed for the pseudo-game, which differs from, or expands upon, the Mostly Civilized Hex Engine, were: a) real-time gameplay (as opposed to turn-based), b) different nations visualized as different colors on the map, c) various systems and interfaces usually present in Grand Strategy games (such as diplomacy between nations, management of settlements, etc.)

The final version of the pseudo-game consisted of a gameplay experience, with a gameplay time of a maximum of 30 minutes, that simulated a small part of playing a Grand Strategy or 4X game. The player goal in the pseudo-game was to invade an enemy nation and claim one of their cities as their own. Sub-goals included, but were not limited to; building farms to produce food, recruiting armies (which cost food to recruit), make claims on foreign territory, declaring war on the enemy nation, and sending your armies into battle against the enemy. Making the Game Wiki Obsolete 21

The faster a player could succeed in the goal of the game, the more proficient at the game they were considered to have become.

4.3.3 Prototype

Based on the results of the mock-up exploration phase, Mock-up 3 was chosen as the best candidate for implementation as a prototype. However, with a few alterations. As suggested by testers of the mock-ups, the navigation of the win- dowed in-game encyclopedia was simplified, by adding a tab-system to the side of the window. This allowed for faster and more understandable navigation of the different categories in the encyclopedia. Icons were also added as image recog- nition allowed for easier digestion of the different categories and their content. Finally, tooltips were kept, but in the more traditional simple form, rather than the expanding versions tested with Mock-up 1. However, these tooltips contained highlighted clickable words, similar to hyperlinks on web pages. Clicking on one of these highlighted words would immediately open the in-game encyclopedia on the corresponding page to the link clicked. This solved a concern testers had expressed regarding the Mock-up 3, as they had liked the possibility of reading while playing, but had found the traversal of the tree of categories confusing and time-consuming. This prototype was then implemented in the pseudo-game. This implementation can be seen in Fig.4.

Fig. 4: The Windowed Encyclopedia prototype, here called Codex, implemented in the pseudo-game. 22 Timmy Eklund

4.3.4 User testing

A final test of the prototype, as implemented in the pseudo-game, was done on a group of players of mixed skill and experience. The test was performed using A/B-testing, comparing two groups of players; one group which played through the pseudo-game scenario, with the prototype, the windowed in-game encyclopedia, available to help them reach their goal, and one control group where players had to learn how the game worked without help from the in-game encyclopedia. The test was done without supervision, on the test participants own computers, in the comfort of their own home (or other personal choice of location), as online supervision likely would be extremely difficult, and physically meeting the participants would not be prudent as, at the time of research, a pandemic was spreading around the world. At the start of the game, the participant was randomly selected for one of the test groups. The participant was then given a short explanation of the game, its basic rules and mechanics, and the goal of the game. They were also informed that the game had a time limit of 30 minutes, after which they would fail at completing the main objective of the game, and as such lose the game. This was done to help the players stay focused on the task of completing the goal as fast as possible. This was desired, as the main measurement of success would be how fast a par- ticipant was able to complete the game in a satisfactory manner. The hypothesis was that if the testers, which belonged to the group which had the prototype to help them, on average performed better in the game, than the control group, then the addition of the prototype would indicate that the easy access to infor- mation allowed for players to more quickly learn the game. However, using how quickly a player won the game, as the only unit of measurement, was likely to not result in accurate data, as a huge number of participants would be required to fully eliminate the differences in experience with similar games the differ- ent participants may have. A more experienced participant would after all be more likely to quickly figure out how to play the game. As such, and additional requirement was added. Instead of only measuring how quickly a player could win the game, a second unit of measurement was added. The number of times the encyclopedia was opened on a relevant page, prior to the completion of one of the steps necessary to progress in the game. In other words, if a correlation between reading a relevant encyclopedia page and winning more quickly could also be found, then the hypothesis was likely to be correct. Finally, in order to also measure how the participants themselves viewed their efforts and level of success in the game, the participants were asked to fill out a survey after their first playthrough of the game. In this survey the participants had to answer questions regarding their enjoyment of the game, their level of commitment to fulfill the goal of the game, if they felt the game frustrating or Making the Game Wiki Obsolete 23 difficult to learn, and whether the in-game encyclopedia had been of use to them. The last question only being present if the participant belonged to the group which had the encyclopedia available. This results of this survey could help indicate if a participant, despite poor re- sults, still enjoyed the game and was helped by the prototype in a satisfactory manner, or if in the case of an opposite experience, a participant quickly suc- ceeded but felt that the help had been lacking or had not enjoyed the game. This third level of measurement would help further steer the test results to indicate whether the participants had felt that the game was approachable or not. 24 Timmy Eklund

5 Results

The results of the pre-study survey, which measured player tendencies to use outside resources while playing Grand Strategy games, indicated that a majority of players shift window focus to read wikis primarily while playing the game, as opposed to before or afterwards. Many of the participants also answered that they do not have access to all the in-game information they need when playing such games. The expert evaluation of the three different mock-ups showed that both tooltips and windowed in-game encyclopedias are plausible alternatives for displaying detailed information, traditionally found on wikis, in a player-friendly and use- ful manner. The windowed encyclopedia, however, showed better potential for scaling, as tooltips were likely to grow too big, if the information they would contain was too great. The evaluation also showed that it is important to have accessible and easy to use navigation for such encyclopedias, as otherwise the players curiosity might be dampened which might have a negative impact on the players experience. The user test of the implementation of the windowed encyclopedia suggested that having the encyclopedia as a resource in Grand Strategy games leads to lower levels of confusion and frustration for players, which in turn seems to lead to players performing better at the game, as they have a higher rate of victory and progress further in the game faster. Player perception of having access to the information they need also increased with access to the encyclopedia.

5.1 Pre-Study The pre-study survey, on players current usage of outside resources while playing Grand Strategy games, were answered by 107 participants, from all ages between 14-51, the majority of which identified as male. Two thirds of all participants spend between 1-4 hours per day playing videogames and, on average, 51-60% of that time is spent on Grand Strategy games in particular. Worth to note is that none of the participants spent 0% of their time on Grand Strategy games, so no complete beginners answered the survey. The most commonly played games (played by at least 50% of the participants) were Sid Meier’s Civilization V & VI, Crusader Kings II, Hearts of Iron IV, Europa Universalis, Victoria II and Stellaris. A clear majority of players completely agreed that they currently enjoyed playing Grand Strategy games, and the data is almost identical for their prediction on if they would continue to enjoy playing such games in the future. However, when asked about their enjoyment during the first 6 hours of playing such a game, while many still enjoyed it immensely, there were a small portion of players who did not necessarily enjoy themselves during this initial time. When asked whether the participant considered Grand Strategy games to be easy to play (in a difficulty sense), most answered that they thought it was neither Making the Game Wiki Obsolete 25 very easy, nor very difficult, however the number of participants who thought the games were at least slightly difficult were more significant than those who thought they were easy. A majority of participants also thought that Grand Strategy games are difficult to learn to play (in the approachability sense). As seen in Fig.5, a large majority of the participants felt that while playing they did not always have access to the information they desire or need.

Fig. 5: Pre-study - Player responses to the statement: I always have all the in-game information I need when playing Grand Strategy. (1: Completely Disagree, 5: Completely Agree)

On the topic of how often the participants shift window focus while playing—an action also known as alt-tabbing—a majority answered that they often shift focus in order to look up information they need. When asked on their usage of wikis, in relation to playing Grand Strategy games, no consensus could be reached on preference for reading wikis before playing games, but a larger portion of the participants did answer that they use wikis after playing the game. However, a strong majority of players reported that they use wikis in the middle of playing Grand Strategy games, as can be seen in Fig.6. Finally, when asked which specific internet resources they use when playing Grand Strategy games, almost 90% of the participants answered that they use wikis. 71% used YouTube, and 40% used strategy guides (printed or online). The survey questions in their entirety can be found in Appendix A.

5.2 Mock-up exploration

The expert evaluation of the three different mock-ups was performed together with six participants. All participants were male, with an average age of 31. Many of the participants were employed as video-game developers. The participants 26 Timmy Eklund

Fig. 6: Pre-study - Player responses to the statement: I use wikis related to the game I’m playing WHILE playing Grand Strategy. (1: Completely Disagree, 5: Completely Agree) all had a high level of experience with videogames in general and a mid-to-high level of experience with Grand Strategy Games in particular. The six participants were shown the mock-ups in different orders to eliminate any potential order bias. The participants had to give the different mock-ups a rating of 1-5, where 1 is bad and 5 is good, in three different categories: Effectiveness, Enjoyment, and Accessibility. That is, how effectively this mock-up helped the player understand more about the game, if this type of design was something they personally liked, and whether the design was easy to understand and use. The experts were then asked to rate all the mock-ups in order of preference, after having seen them all. Those ratings were used to calculate an aggregate rating. The results were, on average: Category Mock-up 1 Mock-up 2 Mock-up 3 Expanding Fullscreen Windowed Tooltips encyclopedia encyclopedia Effectiveness 4.1 3.8 4.1 Enjoyment 4.1 3.5 4.0 Accessibility 4.5 4.3 3.9 Aggregate rating 3.0 2.3 4.0 Total 15.7 13.9 16.0 (1: Very bad, 3: Neutral, 5: Very good) Apart from these metrics, the expert evaluation consisted first and foremost of an interview. In this interview, among other comments, some of the experts expressed concern regarding the Windowed encyclopedia mock-ups current form of navigating be- tween pages, and stated that if the navigation were better, they would have rated Making the Game Wiki Obsolete 27 said mock-up a lot higher. The problematic navigation can be seen in Fig.16 in Appendix B. Other comments included, but were not limited to: Suggestions for icons to be included in navigation trees for the encyclopedias to better be able to quickly recognize categories; the desire to have all three mock-ups implemented, in order to facilitate different needs; concern that tooltips do not allow you to deep-dive and read more about other related subjects, if you are feeling curious; concern that the Fullscreen encyclopedia interrupted gameplay too much. In conclusion, Expanding Tooltips and the Windowed encyclopedia were both very viable candidates, but provided the concerns in the comments could be addressed, the Windowed encyclopedia was deemed the best candidate for im- plementation.

5.3 Implementation and Evaluation

After implementation in the pseudo-game, the Windowed encyclopedia proto- type was tested. The quantitative test was performed on two groups. One group of 8, which had access to the Windowed encyclopedia, which we will call Group A, and one control group of 8, without it, which we will call Group B, for a total of 16 participants. Group A: Had Windowed encyclopedia Group B: Did not have Windowed encyclopedia In Group A, 88% of the participants were male, and 12% were female. In Group B, there was an equal many male and female participants, in other words, 50%- 50%. Group A’s participants had an average age of 34, while Group B had an average age of 26. In Group A, the participants played games, on average, 2:10 hours per day. Group B played games 1:50 hours per day. Finally, on a scale from 1-5, where 1 is never and 5 is always, on how often the participants played Grand Strategy or 4X Games, Group A answered 2.4, while Group B answered 2.3. In other words, Group A was likely slightly more accustomed to playing games similar to the pseudo-game, than Group B. After these initial general questions, the participants had to rate their experi- ences playing the scenario of the pseudo-game from 1-5, where 1 is Disagree Completely, 3 is Neither Agree nor Disagree, and 5 is Agree Completely. The results were: Question Group A Group B The game was fun to play. 3.8 3.6 The game was frustrating. 3.1 3.5 The game was difficult to learn. 3.3 3.0 I performed well in the game. 3.6 3.4 I put a lot of effort into winning the 3.6 2.3 game. 28 Timmy Eklund

As can be seen in Fig.7, players in Group A seemed to think the game was less frustrating, than players in the control group.

Fig. 7: Results - Levels of frustration for participants in both groups and corresponding predicted normal distributions (Group A: With Codex, Group B: Without Codex)

Afterwards, the participants were asked to, in the same way, rate their experi- ences with the Tooltips, Windowed encyclopedia and access to information. (The participants in Group B did not receive the questions related to the Windowed encyclopedia.)

Question Group A Group B I was often confused on what to do 2.6 3.6 next. I had access to the information I 3.4 2.4 needed while playing. The tooltips were useful. 3.4 3.1

I read the tooltips often. 3.1 3.3

The Codex (Windowed Encyclopedia) 4.1 was useful. I used the Codex (Windowed 3.6 Encyclopedia) often.

As can be seen in Fig.8, players in Group A also seemed to think the game was less confusing, than players in Group B, just like in the case of frustration.

Players in Group A were likewise more satisfied with the amount of information they had access to, than the control group players, as can be seen in Fig. 9. Making the Game Wiki Obsolete 29

Fig. 8: Results - Levels of confusion for participants in both groups and corresponding predicted normal distributions) (Group A: With Codex, Group B: Without Codex)

Finally, the participants had to upload a save-file of their playthrough of the game. This save-file contained a log of events as they occurred, while the partic- ipant was playing the game. From this save-file, whether the player won or lost the game, and how long it took them to complete the game, was extracted. The results were as follows: Data Group A Group B Win-ratio 88% 50% Time to complete 00:20:19 00:25:48 The time to complete had a limit of 30 minutes, after which a player would lose the game. Both groups individual times were also tested with a statistical analysis called a “Two sample t-test”. A two sample t-test is a statistical test that can be used to determine whether two different groups of samples has a different mean-value. In other words, it would determine if Group A’s lower average time of completion, 20 minutes and 19 seconds, compared to Group B’s, 25 minutes 48 seconds, were due to statistically significant differences, or whether it was likely to be a result of random chance. The result of the two sample t-test was that, with a significance level of 0.10 (90% confidence), it was determined that the two groups differences in time of completion were statistically significant. The result was, in other words, not due to random chance. The differences in time of completion and their calculated trends can be seen in Fig.10. 30 Timmy Eklund

Fig. 9: Results - Satisfaction of amount of information available for participants in both groups and corresponding predicted normal distributions (Group A: With Codex, Group B: Without Codex)

Fig. 10: Results - Time of completion for participants in both groups and corresponding linear trends (predicted averages) (Group A: With Codex, Group B: Without Codex) Making the Game Wiki Obsolete 31

6 Discussion

The results of the pre-study indicated that most players that use third-party resources, especially wikis, often did so during gameplay, rather than before or after. This likely meant that players felt the need for such information during gameplay and did not want to proceed with their play-session unless they could find the answer to the question they were seeking.

This was the basis for which the different types of mock-ups were chosen, as stated in the Mock-up exploration subsection of the Methodology section. The three different mock-ups chosen could represent three different levels of “depth of content” vs “ease of access”. The first mock-up, expanding tooltips, had the highest level of “ease of access” as they had very little interruption of gameplay when the player wanted to read them. However, due to the nature of the small size of a tooltip, it could not contain much information and therefore had a low level of “depth of content”. The second mock-up, fullscreen in-game encyclopedia, instead had the highest level of “depth of content” as it quite closely could emulate the nature and functionality of a wiki page. It could contain large amounts of information, navigation, and further reading. However, it instead suffered from the same problem as a real wiki page. It completely interrupted the gameplay. As such, it had a very low level of “ease of access”. Finally, the third mock-up, a windowed in-game encyclopedia, tried to take the best of both worlds, by allowing for greater amounts of information than a tooltip, but still was small enough of a window to allow the player to continue to play the game, while using the encyclopedia at the same time. This mock-up had a medium level of both “ease of access” and “depth of content”.

The results of the expert evaluation of these three mock-ups at first indicated that players would prefer both expanding tooltips and the windowed encyclope- dia over the fullscreen encyclopedia. However, upon further reading and analysis of the comments of the participants, while many liked the expanding tooltips, just as many of the participants liked the windowed encyclopedia, despite it hav- ing many more shortcomings, many of which were quite easily fixed. As such, the decision was made to move forward with the windowed encyclopedia, rather than the expanding tooltips, provided that the shortcomings of the design could be remedied.

Many of the test participants expressed that they really liked the windowed encyclopedia, but did not like the provided way of navigating it. Many expressed concern that they would not use it, despite its usefulness, as navigating the tree of categories was cumbersome, and detracted from gameplay, the one thing that this mock-up was trying to minimize. However, the same participants said that they wished they could quickly access the encyclopedia by immediately jumping to the page they wanted to read, without having to rely on navigating the navigation tree. These comments gave birth to the idea of clickable links in tooltips, which could open the correlating page in the encyclopedia. 32 Timmy Eklund

The test of the prototype, implemented in the pseudo-game, proved to have very interesting results, despite having few participants. Players who had access to the windowed encyclopedia performed better at the game, winning approx- imately 75% more often than those who did not. This may however be a bit misleading, as the low number of participants (only 8 in each group, for a total of 16) could not eliminate biases, such as players with more experience in simi- lar games being over-represented in one of the groups, etc. However, the results also showed that participants who used the windowed encyclopedia had lower levels of confusion and frustration, compared to the control group, which could be a sign a that approachability has been at least somewhat improved through making information, typically found on wikis, available to the players. Those in the control group reported a higher dissatisfaction with the amount of informa- tion available to them, compared to the test group which had the encyclopedia, which could indicate that access to even low amounts of additional information leads to an improved player experience. All of these results point to an improved level of approachability for the partic- ipants in the test group which had the windowed encyclopedia available to the them. This seems to be perfectly in line with one of the Game Approachability Principles defined by Desurvire and Wiberg; “Information On Demand and In Time, System Thinking” [26]. Presented simply, players having access to the in- formation they require, at the time they need it, is one of the most important aspects of good game approachability. Presenting tons of information initially, in a tutorial for example, might lead to players not being able to remember all instructions given, as they are not useful in practice immediately, which is an important step to learning information. Not getting the information at all is obviously worse, but having access to the information when you actually need it alleviates frustration with challenges and leads to players learning how to play much more effectively. This principle is only one of several presented in the Game Approachability Prin- ciples. The effectiveness of the presented solution of a windowed encyclopedia is, in a vacuum, hard to measure. Good approachability in videogames consists of many different aspects and cannot be tackled by one principle alone. As highlighted by Lee and Tsai in 2010 [25], one of the key factors to player re- tention is Flow. The reduction of frustration and confusion seen from the results of the test indicate that the game experience with a windowed encyclopedia is more likely to produce a flow-like state than an experience without it, as flow exists in the intrinsically motivated middle-point between boredom and frustra- tion, where a satisfying challenge lies. Lee and Tsai also indicate that perceived behavioral control, that is, choosing to perform an action based on the whether it is believed to succeed or not, is an important part of player retention [25]. In other words, players being able to make educated guesses on whether they will succeed or not, in their planned action in the game, directly influences whether they will continue playing it. As such, the sooner the player can have a fairly good understanding of how the game works, and how to plan their strategy ac- Making the Game Wiki Obsolete 33 cordingly, the more likely they are to keep playing the game. The participants in the test group, which had the windowed encyclopedia, had an increased satisfac- tion with the amount of information available to them. This may be a key step in players gaining perceived behavioral control over the game, as more knowledge leads to players making more educated decisions in the game. When asked whether test participants put a lot of effort into winning the game, the control group had a much lower score, than the group with the encyclope- dia. In other words, the control group answered that they put a lot less effort into winning the game, than the other group. There is not enough data to fully support any one reason for this. However, it might be possible that the low ap- proachability experienced by participants in the control group led to participants feeling an increasing level of frustration with not understanding the game, that they eventually stopped caring whether they won or lost. There was also a slight difference in opinion on if the game was difficult to learn, between the two groups. And strangely enough participants in the group with the encyclopedia, who on average performed better than the control group, believed the game was slightly more difficult than the control group. The amount of participants is too low to determine if this is only due to random chance. However, it could be a simple case of, what is commonly referred to as, the Dunning-Kruger effect [43]. The Dunning-Kruger effect, put simply, is a tendency in humans, to be excessively confident in how much they know about a subject, when they in fact know very little about it. Or, put another way: The more you know about a subject, the more you realize how little you actually know about it. It could therefore be possible that participants in the group with the encyclopedia learned more about the game, than the control group, and as such became more aware of the games difficulty. Finally, due to reasons outside of the projects control, the final test, and its results, are subject to some rather big limitations, which need to be considered.

6.1 Limitations

The biggest limitation of the project was the inability to physically meet with test participants, due to a currently raging worldwide pandemic. This meant that performing qualitative test became much harder. Especially those that involved some type of observation of the participants. This in turn lead to the final test being a remote quantitative test, measuring objective statistics, such as win- ratio, and player opinions, from a survey, as opposed to a qualitative observation test, where the game approachability principles could have been used as a tool of comparison and measurement, and where the test leader could have benefited from having the participants “think out loud” while playing. Another limitation was the low number of participants in the final test. 8 par- ticipants in each group for a total of 16, is not really enough to eliminate biases such as differences in experience, commitment to performing well, or general 34 Timmy Eklund video-game skill. A more reliable result might have been gained had the number of participants been larger. The final limitation of note was the time span of the project. Had the time available to the project been longer, the prototype could have instead been im- plemented in an already established Grand Strategy game, and a long-term study on actual player retention could have been performed, by, through A/B-testing, measuring the number of new players who kept playing, compared to how many that quit, based on access to the prototype, or lack thereof. Making the Game Wiki Obsolete 35

7 Conclusion

The aim of this study was to investigate potential improvements in approach- ability and player retention in Grand Strategy games. This was important due to an observed phenomenon of players of such games often pausing their games to use information resources outside the game itself, such as Youtube and wikis. This phenomenon was first investigated through a pre-study survey on trends in usage of such resources outside of games. The pre-study showed that players primarily use wikis, and most often does so in the middle of playing games. As such, three mock-ups on ways of presenting information traditionally found on wikis, in-game were designed. One of them featured tooltips, which could ex- pand. The other two featured an in-game encyclopedia. The first encyclopedia mock-up had a fullscreen window. The other encyclopedia mock-up had a much smaller window, which allowed the player to continue playing, while it was ac- tive. Through an expert evaluation of the mock-ups, the windowed encyclopedia mock-up was chosen to be implemented as a prototype in a small game, which could be tested on players.

The results highly indicate that player frustration and confusion were lowered by the inclusion of a windowed encyclopedia. This was likely due to an improve- ment in the level of approachability in the game, by giving players access to the information they need, at the exact time they need it. as they can procure it themselves, when they feel the need, which according to the game approachabil- ity principles is an important factor in game approachability.

The improvements in players satisfaction with the amounts of information they have access to, in times of need, leads to players more confidently being able to make decisions. From previous research, it is known that confidence in decision making is one of the factors for improved player retention in online games. This may also hold true for non-online games. The lower levels of frustration and confusion also indicates that players are closer to a Flow state when they have access to the information presented in the windowed encyclopedia. This is very important as being in a Flow state is a key factor in player retention.

This study sought to investigate three potential improvements: approachability, player retention, and a non-intrusive design. The results showed that while po- tential bias could not be completely ruled out, the results highly indicate that providing detailed information about game mechanics—in-game, at the players leisure—will improve game approachability. The access to detailed information is likely to have lowered the levels of confusion and frustration that is common for new players, in similar games. Reducing frustration is necessary to be able to reach a Flow state, which is a very important aspect of player retention in video games.The windowed encyclopedia proved to be a useful way of providing this detailed information to the players in a non-intrusive way, as the players themselves chose when they wished to be presented with information, and the encyclopedia did not hinder the player from further play at the time it was used. 36 Timmy Eklund

The windowed encyclopedia was able to improve approachability and suggest an improvement in player retention. It can therefore be concluded that the win- dowed encyclopedia is a satisfactory possible solution to the problem of players looking for information elsewhere, when they do not having enough information, when playing Grand Strategy Games.

7.1 Future Work

In future studies on approachability in games, it would be recommended to use a high number of participants, in order to eliminate biases, like experience, skill and commitment. Different people play games differently, and, in order to get conclusive data, a larger sample size is necessary. On the topic of different solutions to the problem, other than the one presented in this thesis, it would also be prudent to investigate solutions presented for similar problems in areas other than videogames, as this study based its mock- ups on systems already established in the gaming industry; tooltips and in-game encyclopedias. Perhaps applications for the medical or industrial industry has found novel ways of solving similar problems, etc. Making the Game Wiki Obsolete 37

References

[1] WePC, “2020 video game industry statistics, trends & data,” Jan. 2020, [Accessed 2020-02-13]. [Online]. Available: https://www.wepc.com/news/ video-game-statistics/ [2] B. Reeves, “Do gamers really like hard games?” 2014, [Accessed 2020- 05-31]. [Online]. Available: https://www.gameinformer.com/b/features/archive/ 2014/09/25/why-we-like-hard-games.aspx [3] J. Walker, “I really wanted to be able to play stellaris,” May 2016, [Accessed 2020-01-29]. [Online]. Available: https://www.rockpapershotgun.com/2016/05/ 16/i-really-wanted-to-be-able-to-play-stellaris/ [4] A. Hailes, “The rhythms of digital history: A guide to historical grand strategy games,” Aug. 2019, [Accessed 2020-02-12]. [Online]. Available: https://25yearslatersite.com/2019/08/11/ the-rhythms-of-digital-history-a-guide-to-historical-grand-strategy-games/ [5] IMDb, “Top lifetime grosses,” 2020, [Accessed 2020-02-13]. [Online]. Available: https://www.boxofficemojo.com/chart/top_lifetime_gross/?area=XWW [6] E. Martinello, “League of legends generated $1.5 billion revenue in 2019,” Jan. 2020, [Accessed 2020-02-13]. [Online]. Available: https://dotesports.com/ league-of-legends/news/league-of-legends-generated-1-5-billion-revenue-in-2019 [7] J. Rifkin, “Global box office reaches record $42.5b in 2019,” Jan. 2020, [Accessed 2020-02-13]. [Online]. Available: https://www.boxofficepro. com/global-box-office-2019-record-42-5-billion/ [8] K. Webb, “The $120 billion gaming industry is going through more change than it ever has before, and everyone is trying to cash in,” Oct. 2019, [Accessed 2020-02-13]. [Online]. Available: https://www.businessinsider.com/ video-game-industry-120-billion-future-innovation-2019-9?r=US&IR=T [9] Paradox Interactive, “Europa Universalis IV,” Online (Steam), 2013, [Accessed 2020-05-30]. [Online]. Available: https://store.steampowered.com/app/236850/ Europa_Universalis_IV/ [10] ——, “Hearts of Iron IV,” Online (Steam), 2016, [Accessed 2020-05-30]. [Online]. Available: https://store.steampowered.com/app/394360/Hearts_of_Iron_IV/ [11] Firaxis Games, “Sid Meier’s Civilization V,” Online (Steam), 2010, [Accessed 2020-05-30]. [Online]. Available: https://store.steampowered.com/app/8930/Sid_ Meiers_Civilization_V/ [12] ——, “Sid Meier’s Civilization VI,” Online (Steam), 2016, [Accessed 2020-05-30]. [Online]. Available: https://store.steampowered.com/app/289070/Sid_Meiers_ Civilization_VI/ [13] Valve Corporation, “STEAM & GAME STATS,” Feb. 2020, [Accessed 2020-02-12]. [Online]. Available: https://store.steampowered.com/stats/ [14] Paradox Interactive, “Crusader Kings 2,” Online (Steam), 2012, [Accessed 2020-05-30]. [Online]. Available: https://store.steampowered.com/app/203770/ Crusader_Kings_II/ 38 Timmy Eklund

[15] ——, “Stellaris,” Online (Steam), 2016, [Accessed 2020-05-30]. [Online]. Available: https://store.steampowered.com/app/281990/Stellaris/ [16] Pieces Interactive, “Magicka 2,” Online (Steam), 2016, [Accessed 2020-05-30]. [Online]. Available: https://store.steampowered.com/app/238370/Magicka_2/ [17] Obsidian Entertainment, “Pillars of Eternity,” Online (Steam), 2015, [Accessed 2020-05-30]. [Online]. Available: https://store.steampowered.com/app/291650/ Pillars_of_Eternity/ [18] R. Sibii, “Grand strategy,” Jan. 2017, [Accessed 2020-02-20]. [Online]. Available: https://www.britannica.com/topic/grand-strategy [19] A. Emrich, “Sneak preview: Masters of orion,” Computer Gaming World, no. 110, p. 92–93, Sep. 1993, [Accessed 2020-02-20]. [Online]. Available: http://www.cgwmuseum.org/galleries/index.php?year=1993&pub=2&id=110 [20] Maxis, “The Sims,” PC CD, 2000, [Accessed 2020-05-30]. [Online]. Available: https://sims.fandom.com/wiki/The_Sims [21] Double Eleven & Introversion Software, “Prison Architect,” Online (Steam), 2015, [Accessed 2020-05-30]. [Online]. Available: https://store.steampowered. com/app/233450/Prison_Architect/ [22] M. Csikszentmihalyi, Flow: The Psychology of Optimal Experience. New York, NY: Harper Perennial, March 1991. [Online]. Available: http://www.amazon. com/gp/product/0060920432/ref=si3_rdr_bb_product/104-4616565-4570345 [23] Wiktionary Contributors, “Time flies when you’re having fun,” Sep. 2006, [Accessed 2020-03-31]. [Online]. Available: https://en.wiktionary.org/wiki/time_ flies_when_you’re_having_fun [24] P. Sweetser and P. Wyeth, “Gameflow: A model for evaluating player enjoyment in games,” Comput. Entertain., vol. 3, no. 3, p. 3, Jul. 2005. [Online]. Available: https://doi-org.proxy.ub.umu.se/10.1145/1077246.1077253 [25] M.-C. Lee and T.-R. Tsai, “What drives people to continue to play online games? an extension of technology model and theory of planned behavior.” International Journal of Human-Computer Interaction, vol. 26, no. 6, pp. 601–620, 2010. [Online]. Available: http://search.ebscohost.com.proxy.ub.umu. se/login.aspx?direct=true&db=buh&AN=51095004&site=ehost-live&scope=site [26] H. Desurvire and C. Wiberg, User Experience Design for Inexperienced Gamers: GAP – Game Approachability Principles. London: Springer London, 2010, pp. 131–147. [Online]. Available: https://doi.org/10.1007/978-1-84882-963-3_8 [27] Google/Kantar TNS, “Why casual gamers keep playing,” 2018, [Ac- cessed 2020-02-24]. [Online]. Available: https://www.thinkwithgoogle.com/data/ why-casual-gamers-keep-playing/ [28] J. P. Gee, “Learning by design: Good video games as learning machines,” E-Learning and Digital Media, vol. 2, no. 1, pp. 5–16, 2005. [Online]. Available: https://doi.org/10.2304/elea.2005.2.1.5 [29] M. A. Dennis, “Wiki,” Aug. 2018, [Accessed 2020-02-12]. [Online]. Available: https://www.britannica.com/topic/wiki Making the Game Wiki Obsolete 39

[30] The Editors of Encyclopædia Britannica, “Wikipedia,” Dec. 2018, [Accessed 2020-02-12]. [Online]. Available: https://www.britannica.com/topic/Wikipedia [31] M. Barr, “Learning through collaboration: video game wikis,” Int. J. of Social Media and Interactive Learning Environments, vol. 2, pp. 119 – 133, 01 2014. [32] J. Nielsen, “The 90-9-1 rule for participation inequality in social media and online communities,” Oct. 2006, [Accessed 2020-02-13]. [Online]. Available: https://www.nngroup.com/articles/participation-inequality/ [33] P. Christensson, “Tooltip,” 2006, [Accessed 2020-03-31]. [Online]. Available: https://techterms.com/definition/tooltip [34] Riot Games, “Tooltips for abilities in League of Legends,” 2020, [Screenshot by Author]. [Online]. Available: https://www.leagueoflegends.com/ [35] Riot Reinboom, “Tooltips Update,” Jul. 2018, [Accessed 2020-03-18]. [On- line]. Available: https://boards.na.leagueoflegends.com/en/c/developer-corner/ yIBUoyaP-tooltips-update [36] Giant Bomb contributors, “Min-maxing (concept),” [Accessed 2020-03-18]. [Online]. Available: https://www.giantbomb.com/min-maxing/3015-128/ [37] Reddit.com, “Paradox plaza subreddit,” [Accessed 2020-05-30]. [Online]. Available: https://www.reddit.com/r/paradoxplaza/ [38] ——, “Reddit,” [Accessed 2020-02-18]. [Online]. Available: https://www.reddit. com/ [39] ——, “Sample size subreddit,” [Accessed 2020-05-30]. [Online]. Available: https://www.reddit.com/r/SampleSize/ [40] Figma, “Figma,” 2016, [Accessed 2020-06-16]. [Online]. Available: https: //www.figma.com/ [41] Quill18, “Mostly Civilized Hex Engine,” 2017, [Accessed 2020-03-17]. [Online]. Available: https://github.com/quill18/MostlyCivilizedHexEngine [42] ——, “Quill18 - Youtube channel,” [Accessed 2020-03-17]. [Online]. Available: https://www.youtube.com/user/quill18 [43] J. Kruger and D. Dunning, “Unskilled and unaware of it: How difficulties in rec- ognizing one’s own incompetence lead to inflated self-assessments,” Journal of Personality and Social Psychology, vol. 77, pp. 1121–34, 01 2000. 40 Timmy Eklund

8 Appendix

A Pre-study - Survey Questions

Section 1 1. What is your age? – 0-120 2. What is your gender? – Male, Female, Prefer not to say, Other 3. How many hours per day do you play video games on average? – 0-24

Section 2 1. How much of your time playing video games is spent playing Grand Strategy Games? – 0-10%, 11-20%, ... , 91-100%. 2. Which Grand Strategy Games have you previously played? (Select all that apply) – Sid Meier’s Civilization III, IV, V, VI, Crusader Kings, II, Hearts of Iron II, III, IV, Europa Universalis III, IV, Victoria II, Stellaris, Imperator: Rome, Age of Civilizations II, Other

Section 3 1. I currently enjoy playing Grand Strategy Games. – Scale 1-5 (1: Disagree completely, 5: Agree completely) 2. I enjoyed playing my first Grand Strategy Games the first 6 hours I played of it. – Scale 1-5 (1: Disagree completely, 5: Agree completely) 3. I will probably enjoy playing Grand Strategy Games in the future. – Scale 1-5 (1: Disagree completely, 5: Agree completely) 4. I often play multiplayer when playing Grand Strategy Games. – Scale 1-5 (1: Disagree completely, 5: Agree completely) Making the Game Wiki Obsolete 41

5. I think Grand Strategy Games are easy to play. – Scale 1-5 (1: Disagree completely, 5: Agree completely) 6. I think Grand Strategy Games are easy to learn. – Scale 1-5 (1: Disagree completely, 5: Agree completely) 7. I always have all the in-game information I need when playing Grand Strat- egy Games. – Scale 1-5 (1: Disagree completely, 5: Agree completely) 8. Do you have any other comments?

Section 4 1. I often alt-tab to look up information while playing Grand Strategy Games. – Scale 1-5 (1: Disagree completely, 5: Agree completely) 2. I read/watch tutorials on Grand Strategy Games BEFORE playing. – Scale 1-5 (1: Disagree completely, 5: Agree completely) 3. I read/watch tutorials on Grand Strategy Games AFTER playing. – Scale 1-5 (1: Disagree completely, 5: Agree completely) 4. I read/watch tutorials on Grand Strategy Games WHILE playing. – Scale 1-5 (1: Disagree completely, 5: Agree completely) 5. I use wikis related to the game I’m playing BEFORE playing Grand Strategy Games. – Scale 1-5 (1: Disagree completely, 5: Agree completely) 6. I use wikis related to the game I’m playing AFTER playing Grand Strategy Games. – Scale 1-5 (1: Disagree completely, 5: Agree completely) 7. I use wikis related to the game I’m playing WHILE playing Grand Strategy Games. – Scale 1-5 (1: Disagree completely, 5: Agree completely) 8. Which internet resources do you use when playing Grand Strategy Games? – Wikis, Youtube, Strategy guides (online or printed), Other 9. Do you have any other comments? 42 Timmy Eklund

B Mock-ups

(a) Regular tooltip (b) Expanded tooltip

(c) Fullscreen encyclopedia

(d) Windowed encyclopedia (e) Windowed encyclopedia navigation

Fig. 11: Mock-ups Making the Game Wiki Obsolete 43 Regular tooltip. Fig. 12: Mock-up 1: Expanding tooltips 44 Timmy Eklund Expanded tooltip. Fig. 13: Mock-up 1: Expanding tooltips Making the Game Wiki Obsolete 45 Fullscreen encyclopedia Fig. 14: Mock-up 2: Fullscreen in-game encyclopedia 46 Timmy Eklund Windowed encyclopedia Fig. 15: Mock-up 3: Windowed in-game encyclopedia Making the Game Wiki Obsolete 47 Navigation Fig. 16: Mock-up 3: Windowed in-game encyclopedia