Social Behaviour in Non-Player Characters for Role-Playing Games Engenharia Informática E De Computadores

Home , Dungeon Siege

Social Behaviour in Non-Player Characters

for Role-Playing Games

Nuno Miguel Pires Afonso

Dissertação para obtenção do Grau de Mestre em Engenharia Informática e de Computadores

Júri

Presidente: Doutor Mário Rui gomes Orientador: Doutor Rui Filipe Fernandes Prada Vogais: Doutor Nelson Zagalo

Setembro de 2007

Resumo

Com a contínua expansão da indústria de jogos de computador, os programadores de jogos começaram-se a focar na Inteligência Artificial como um dos factores diferenciadores, começando a desenvolver personagens muito credíveis. Contudo, embora estas personagens sejam credíveis por elas próprias, elas não são credíveis num contexto social, o que se traduz num ambiente social fraco, não conseguindo atingir uma “ilusão de vida” social. Por outro lado, existe um tipo de jogo que possui esta componente social, o Massively Multiplayer Online Game (MMO or MMOG, i.e. um jogo online com centenas ou até milhares de jogadores a jogar ao mesmo tempo), que continua a atrair milhões de jogadores. Os MMOs providenciam aos jogadores uma experiência social rica que estes não conseguem encontrar em mais nenhum jogo de computador. Um género de jogo no qual as interacções sociais são muito importantes é o Role-Playing Game (RPG), daí que seja natural que alguns dos MMOs mais jogados sejam RPGs. Contudo, existem pessoas que continuam interessadas numa experiência de jogo em modo single player (i.e. só um jogador), que são, especialmente no caso dos RPGs, experiências mais focadas numa história, mas que continuam sem conseguir atingir uma experiência rica em interacção social como a que se encontra nos MMOs de hoje em dia. Nós acreditamos que através da criação de uma interacção social rica, não só entre o jogador e os Non-Player Characters (NPCs, i.e. personagens controladas pelo computador), mas também entre os próprios NPCs, poderemos criar uma experiência de jogo melhor para os utilizadores. Esta ideia é também suportada por um inquérito realizado por nós a duzentos e cinquenta jogadores de RPGs. Nós propomos um modelo para enriquecer as interacções sociais com, e entre, NPCs, centrada no conceito de relacionamento social. Além disso, apresentamos um caso de estudo que tem por base um jogo que se desenrola num dos cenários mais comuns dos RPGs, o mercado. Os resultados da experiência que realizámos com este jogo mostram que o modelo sugerido tem um efeito positivo na experiência de jogo dos utilizadores.

Palavras Chave: comportamento social, role-playing game, non-player character, jogo de computador, personalidade, relacionamento, memória, theory of mind, conversação, transmissão de informação.

i ii

Abstract

As the computer game industry keeps on growing, game developers start to focus on Artificial Intelligence as one of the differentiating factors between games, creating very believable characters. However, although these characters are believable by themselves, they are not believable in a social context, which translates into poor social environments, failing to attain a social “illusion of life”. On the other hand, there is a type of game that possesses this social component, the Massively Multiplayer Online Game (MMO or MMOG, i.e. a game with hundreds or even thousands of users playing at a time), which continues to attract millions of users. MMOs provide the users with a rich social experience that they cannot find on any other computer games. One genre of game in which social interactions are very important is the Role-Playing Game (RPG), which makes it natural that some of the most played MMOs are RPGs. However, there are still people interested in single player gaming experiences, which, especially in the case of RPGs, tend to be more story-driven, but still lack the rich social interaction experience provided by today’s MMOs. Therefore, we believe that by creating a richer social interaction, not only between the player and Non-Player Characters (NPCs), but also amongst NPCs, the player’s gaming experience could be improved. This belief is supported by a survey conducted by us with two hundred and fifty RPG players. We propose a model to enrich the social interactions with, and between, NPCs centred on the concept of social relationships. Furthermore, we present a case study with a game that takes place on one of the most common scenarios of a RPG, a market. Results of an experiment conducted with this game show that the suggested model has a positive effect in the players’ gaming experience.

Keywords: social behaviour, role-playing game, non-player character, computer game, personality, relationship, memory, theory of mind, conversation, information transmission.

iii iv

Acknowledgements

I would like to thank my parents since they paid for all this! Thankfully I was fortunate enough to have parents that supported my decisions throughout my academic “existential crisis”. I would also like to thank all of my family and friends who put up with me for my first quarter of a century. Finally I would like to thank my supervisor, Rui Prada, who guided me throughout this last year, and my colleagues at GAIPS, namely Tiago Alves, Rita Simões, André Pereira, Iolanda Leite, Carlos Gomes, Guilherme Raimundo, João Dias, Rui Figueiredo, João Gonçalves, António Brisson, Pedro Sequeira, Carlos Martinho, Mário Rui Gomes, and Ana Paiva. Although they are all important, I will point out Guilherme Raimundo for his precious help with the revisions of this document, and Tiago Alves, my office word guru, without whom this writing process would have been much more painful!

v vi

Contents

1 Introduction ...... 1 1.1 Motivation ...... 1 1.2 Problem ...... 2 1.3 Hypothesis...... 3 1.4 Outline ...... 3

2 State of the Art ...... 5 2.1 Introduction ...... 5 2.2 Why Social Behaviour in RPGs? ...... 6 2.3 RPG Analysis ...... 7 2.3.1 Neverwinter Nights...... 8 2.3.2 Knights of the Old Republic II ...... 10 2.3.3 World of Warcraft ...... 11 2.3.4 Dungeon Siege II ...... 12 2.3.5 Fable ...... 13 2.3.6 The Elder Scrolls IV: Oblivion ...... 15 2.3.7 RPG Comparison...... 16 2.4 Academic Research ...... 17 2.4.1 Avatar Arena ...... 18 2.4.2 PsychSim...... 19 2.4.3 Social Role Awareness...... 21 2.4.4 The µ-SIC System...... 22 2.4.5 Demonstrate!...... 26 2.4.6 Comparison ...... 27 2.5 RPG Gamer Survey...... 28

3 Conceptual Model...... 31 3.1 Introduction ...... 31 3.2 Relationships...... 31 3.3 Personality...... 32 3.4 Conversation ...... 33 3.5 Memory...... 34 3.6 Theory of Mind ...... 35 3.7 Information Transmission...... 36 vii

3.8 Conclusions...... 38

4 Implementation ...... 39 4.1 Introduction ...... 39 4.2 AI Architecture ...... 40 4.3 Relationships...... 42 4.4 Personality...... 43 4.5 Conversation ...... 43 4.6 Memory...... 43 4.7 Theory of Mind ...... 44 4.8 Information Transmission...... 45 4.9 Conclusions...... 47

5 Case Study: The Market ...... 49 5.1 Introduction ...... 49 5.2 Story ...... 49 5.3 Characters...... 50 5.4 First Version ...... 51 5.5 Second Version...... 52

6 User Evaluation...... 55 6.1 Introduction ...... 55 6.2 Dialog Evaluation...... 55 6.3 Game Evaluation ...... 56 6.4 Results...... 56

7 Conclusions...... 59 7.1 Introduction ...... 59 7.2 Future Work ...... 60

A RPG Gamer Survey ...... 65

B Memory Example ...... 69

C Case Study Questionnaire ...... 73 C.1 Personal Information Questions ...... 73 C.2 Questions for the first version evaluated...... 74 C.3 Questions for the second version evaluated ...... 75

viii

List of Figures

Figure 2.1 - Screenshot from Neverwinter Nights...... 9 Figure 2.2 - Screenshot from KOTOR II...... 11 Figure 2.3 - Screenshot from World of Warcraft...... 12 Figure 2.4 - Screenshot from Dungeon Siege II...... 13 Figure 2.5 - Screenshot from Fable...... 14 Figure 2.6 - Screenshot from The Elder Scrolls IV: Oblivion...... 16 Figure 2.7 - Example of an unbalanced cognitive configuration [44]...... 19 Figure 2.8 - Screenshot from Psychim's interface [38]...... 20 Figure 2.9 - Architecture proposed for the PPAs [27]...... 23 Figure 2.10 - Psychological models used in the µ-SIC System [28]...... 24 Figure 2.11 - Screenshot from a simulation situation [28]...... 25 Figure 2.12 - Abraham Maslow's Hierarchy of Needs Pyramid [52]...... 26 Figure 2.13 - Screenshot from Demonstrate [43]...... 26 Figure 3.1 - Conversation between two NPCs...... 34 Figure 3.2 - Level 1 of theory of mind...... 35 Figure 3.3 - Level 2 of theory of mind...... 35 Figure 3.4 - Constrains of information transmission...... 36 Figure 3.5 - Personality vs Relationship plot...... 37 Figure 3.6 - Representation of Character A...... 38 Figure 4.1 - Unity Technology Diagram...... 39 Figure 4.2 - Screenshot from Unity's Editor...... 40 Figure 4.3 - Base AI architecture...... 41 Figure 4.4 - AI architecture with conceptual model...... 42 Figure 4.5 - UML diagram of the Memory...... 44 Figure 4.6 - UML of the theory of mind about the player (see Figure 4.5)...... 45 Figure 4.7 - Information transmitted by a Neutral personality...... 45 Figure 4.8 - Information transmitted by a Friendly personality...... 46 Figure 4.9 - Information transmitted by an Unfriendly personality...... 46 Figure 5.1 - Two characters having a conversation...... 51 Figure 5.2 - Player having a conversation with an unfriendly merchant...... 53

ix x

List of Tables

Table 2.1 – Comparison of the social relationships of the NPCs across multiple RPGs...... 17 Table 3.1 - Possible combinations of the sum of the relationships...... 37 Table 5.1 - Personality of each character...... 50 Table 5.2 - Relationships between the characters...... 52 Table 6.1 - Difference between the relationships in version 1 and 2 in the merchants and the mechanic...... 57

xi xii

List of Equations

Equation 3.1 – Formula of the information transmitted...... 37

xiii xiv

List of Abbreviations

AI – Artificial Intelligence ANN – Artificial Neural Network BDI – Beliefs, Desires and Intentions FFM – Five Factor Model FSM – Finite State Machine LOD – Level of Detail MMO – Massively Multiplayer Online MMORPG – Massively Multiplayer Online RPG MOD – Modification NPC – Non-Player Character PPA – Proactive Persistent Agent RPG – Role-Playing Game

xvi

Chapter 1

Introduction

"Our individual lives cannot, generally, be works of art unless the social order is also." Charles Horton Cooley (1864-1929)

1.1 Motivation

In the last decades the personal computers became common devices, used every day by everyone, especially in younger peoples’ lives. As time passed by, it naturally evolved beyond a working machine, becoming also a form of entertainment. This evolution grew exponentially in the last years with the giant leap in processor and graphical capabilities, and with prices continuing to drop. Like with any market, when the demand is high, there will be more and more people interested in supplying, so it is not surprising that the number of games released every year keeps on growing. The game business has been so successful that even consoles, which are basically computers that are specifically designed to play games, became mainstream devices. However, despite this growth of game development, one component – which is extremely important for Role-Playing Games (RPGs) – has continued to be neglected: the social component. The RPG is, like it says, a role-play experience – the players take on the role of portraying those characters, and have to go through the game world resolving tasks that are given to them by Non-Player Characters1 (NPCs). The player needs to interact with NPCs in order to improve her/his character and progress in the story, making the RPG a type of game that has a very big social component. RPGs have been mostly focused on aspects like storytelling, sound and graphics, which, although important, are not enough to create game worlds that feel alive, with the NPCs having their own lives, not being there simply to serve the player. Usually the NPCs are frozen in time, repeating the same action – if any – over and over again.

1 Computer controlled characters. 1

If we could make the NPCs act more like human players – like they were also playing the game and not being there simply to satisfy the player’s needs – we think that by taking into account the social component, by giving a “illusion of life” [3] to the NPCs, the game worlds will have a whole new depth and the player will have an improved gaming experience. This “illusion of life” could be attained by creating richer social interaction, not only focusing on the social behaviour between the player and the NPCs, but also amongst NPCs, which would have their own families and friends, and would search for the satisfaction of their needs.

1.2 Problem

RPGs are usually developed without much focus on social behaviour. Until very recently Artificial Intelligence (AI) in RPGs has been reduced to simple behaviours – like path-finding, obstacle avoidance, and scripted behaviour -, but with the graphical quality becoming very high and mainstream, game developers started to focus on AI as the differentiating factor of the RPGs. Two recent examples of RPGs that started to care about AI are Fable [24] and The Elder Scrolls 4: Oblivion [46]. But although recent RPGs do a great job in creating individually believable characters – especially Fable, which has characters with great animations and very good voice acting -, they are not believable as a society. For example, it is common for the characters to share the same personality and the same behaviour towards not only the player but also other characters. The only characters that stand out are the ones associated with quests. Usually they all share the same memory as well, which means that either everyone knows something or no one does. There are exceptions, but usually work as simple repositories of some sort of information (e.g. one character that seems to appear quite often is one that knows where every place is in a village). This means that in today’s games, although there is an “illusion of life” in individual characters, that same illusion is not present in the society comprised by those characters. It is at least awkward to have multiple different characters act exactly the same way towards us, or stand still day and night waiting for us to interact with them. And even if they move around, they seem to lack purpose, not looking like they have a life of their own. The biggest factor that enlarges the shallowness of the characters is the fact that if we interact multiple times with them, it will always be like they never interacted with us before. Interactions between characters are simple isolated events, which do not have any influence in the story. This is especially awkward when we team up with various characters that follow us around and no matter what we do they will always back us up. Some of the more elaborate team systems may have an influence system that will condition what our team members say to us, but it is not natural if we team up with an animal loving character that continues to follow us around while we keep clobbering baby seals.

In conclusion, social interactions with/between NPCs in RPGs are not rich enough to achieve a rich role-play experience, like a survey conducted by us with RPG gamers confirms (see section 2.5).

1.3 Hypothesis

Recent RPGs create social interactions between the NPCs, with “conversations” spawning between them. However, these interactions are not as rich as the ones we encounter in the real world. And when a simple interaction occur, like in the case of a spontaneous conversation, it is random and without further depth. The notion of family and friends is really shallow or not existent, the fact that they live in the same house does not mean that they have social bonds, and we do not see behaviour like a friend going to someone else’s house for a group dinner. We believe that by enriching the social interactions with/between the NPCs, the social behaviour will become more immersive, and the player’s gaming experience can improve. To improve social interactions we propose adding inter-personal relationships between the characters.

1.4 Outline

This document is divided into seven chapters including this one. On the second chapter we describe the state of the art, with examples of how social behaviour is implemented in some of the latest and more acclaimed RPGs, followed by a review of academic research in the area of social behaviour that may be of interest towards our objective. The second chapter ends with the analysis of questionnaire directed towards RPG gamers to see how they think that this genre of game can improve their gaming experience. The third chapter has a detailed description of the conceptual model that we propose to attain the social behaviour in NPCs. The fourth chapter contains a description of the implementation of our conceptual model. The fifth chapter describes a scenario that was implemented on top of the conceptual model to test with gamers in order to see if they consider that the hypothesis presented in section 1.3 improves their gaming experience, followed by the description and analysis of this user evaluation in chapter six. In the seventh and final chapter we present some conclusions and future work.

Chapter 2

State of the Art

2.1 Introduction

“A role-playing game is a type of game in which the participants assume the roles of characters and collaboratively create stories. Participants determine the actions of their characters based on their characterization, and the actions succeed or fail according to a formal system of rules and guidelines. Within the rules, they may improvise freely; their choices shape the direction and outcome of the games.” Wikipedia [51]

As we can see by the definition above, a RPG is a type of game in which the immersion of the player should be very important. The player creates a character and becomes that character, starting to define it based on her/his choices. In fact, her/his choices can affect the story of the game, which means that the player has an active part in the outcome of the game (or at least on some parts of it). This means that the player can, not only create a big connection with hers/his character, but also should be granted a huge degree of freedom, which could even affect the whole story. This type of game first appeared in its “pen-and-paper”2 version in the 1970s, with the first commercial version, Dungeons & Dragons [50] [54], being released in 1974. With the appearance of computer games, it was only natural for this type of game to be adapted to the digital format, in which a person did not need to simply imagine the environments – s/he would “be in them”! There are two views of RPGs [49]: a western (American/European) view that is based directly on the rules and settings of “pen-and-paper” RPG systems, and an eastern (Japanese) view3. In the western view there is usually a lot of character customization. We can choose gender, race, physical characteristics (e.g. hair style/colour) and class (e.g. warrior, rogue, sorcerer). This view is also very characterized by an extensive set of player attributes. Usually a

2 Also known as tabletop. 3 Commonly referred to as console RPGs. 5 western RPG does not have an ending (i.e. it is not unusual to have sequels with the same main character). The eastern view’s games are usually associated with one main character, meaning that we do not have the customization of the western view. The player’s attributes system is often much simpler – which is not odd since these games are very closely tied with consoles and broader audiences (i.e. to attract younger people). Usually an eastern RPG has an ending, so sequels are often only tied by the names, with completely different stories (e.g. the Final Fantasy saga). In the last few years we have seen a specific type of RPG gain fans – the Massively Multiplayer Online Role-Playing Game (MMORPG). A Massively Multiplayer Online Game (MMOG or MMO) game is played by hundred (sometimes thousands) of players on each server. Some of these games have a monthly fee, which means that the game developers have to find ways to keep the players subscribing. One of the strong points towards this goal is to supply a great social interaction between the players [10].

2.2 Why Social Behaviour in RPGs?

In this type of game, the social interaction is crucial since the player’s evolution and story flow depend on it. Players usually roam throughout giant worlds, populated by various civilizations, with which they have to interact for their characters to progress or simply to complete some task. The richer the social behaviour, the bigger social immersion will be, which is one of top features of MMORPGs. Real people populate them, making the social experience as rich as it can get. But if MMORPGs are so rich when it comes to gaming experience, why not simply go towards a view of full human popularized worlds? There are basically two reasons: first, not everyone likes to play games online, there is always room to play single player games, and second because fully human worlds can create situations that degrade the gaming experience (e.g. spamming) [9]. In addition, people usually do not like to take on secondary roles, which are many times associated with doing repetitive tasks, and people find ways to avoid these situations (e.g. by creating programs that automate these tasks). We believe that there will always be room for NPCs, since there will always be some tasks that the user will not find fun to do in games. We think that with rich environments, in which the NPCs have their own lives, and perhaps even take part in the direction of the story, the player will have a better gaming experience. We want to make every NPC feel unique, and this means that the player will not know what will happen if s/he interacts with her/him, or what will be unleashed if s/he performs a specific action. NPCs will have their own lives, completely autonomous from the player, which will give an illusion of something bigger happening, making the world feel like it has a life of its own and is not simply there revolving around the player.

By looking into several RPGs, we determined some features related to social behaviour that we think that can improve the “illusion of life” [3]: • Personality – if a NPC is truly an individual, if s/he is unique, and reacts in her/his own way to events (e.g. rude, polite, grumpy, definitions of right and wrong). Personality is one of the important factors to give the idea that the world is alive, not all of us react the same way to events; • Emotions – if a NPC transmits emotional states (e.g. happiness, sadness). Emotion is very tied to the “illusion of life” [3], being considered one of the most important factors towards character believability; • Pro-Activeness – if a NPC does not need any human intervention, decides which actions to take on her/his own and dynamically (i.e. not just through scripted events) adjust to the environment. If a character chooses her/his goals dynamically, it will feel like s/he has a mind of her/his own, and that the world is not inhabited by mindless drones; • Individual Memory – if a NPC has her/his own memory/knowledge (i.e. no collective knowledge). This also aims towards a character’s uniqueness, not everyone chooses the same options because they have different information; • Social relationships with the player – if a NPC chooses to interact with the player (i.e. not just talk back when the player talks to her/him), if s/he can have ties with the player (e.g. friend, family, enemy); • Social relationships with the other NPCs – if a NPC chooses to interact with other NPCs, if s/he can have ties with other NPCs. No environment feels truly alive if there is no interaction between its inhabitants. Having identified these features, we analyzed some of the latest RPGs to see how they encompass them.

2.3 RPG Analysis

In order to explore our research problem, we first need to see what is out there in the market, and, since our work focuses RPGs, we picked up some of the more recent RPGs in order to see what they implement when it comes to social behaviour. The games we decided to analyze were: • Neverwinter Nights; • Knights of the Old Republic II; • World of Warcraft; • Dungeon Siege II; • Fable; • The Elder Scrolls IV: Oblivion. These games were chosen because they are some of the latest and best-rated RPGs.

We first started by reading the reviews from well-known game review sites [15] [18] [21], in order to collect the opinion of the game review experts. Afterwards we tested these games ourselves, not only to see what the reviewers meant in their analysis but also to get a better focus on the subject at hand. Most of the reviews do not focus on social behaviour, and in some of the older ones it was not even mentioned, because at the time there was not anything on social behaviour implemented in games. To simplify this analysis we will focus on the latter features – social relationships with the player and social relationships with other NPCs – since the other features are pretty much the same on all of the games, and, as we will see in section 2.5, it is an area where gamers believe that the RPGs could improve. The Personality in these games is pretty much attained through the dialogs that the characters have, especially if they have voice-acting and not just text. Another place where we usually see it is in cut-scenes, but we do not consider the cut-scenes as a “part of the game” since these are pre-generated scripted scenes. It would be good to see some sort of personality traits in characters that are just walking around (e.g. stop and smell a flower, kick that annoying chicken) and not simply in conversations. Emotions are also very tied with conversations or scripted events (e.g. a character that has a quest attached to it, and will stay there crying day and night until we resolve it). We do not see characters that are happier, sadder, more depressed, etc. Most of the newer games try to give an idea that the characters are Pro-Active, that they sense the world around them and then determine what action to do. And if we run past some of these characters, we might get this idea, but if we pass by them enough times or simply stop to watch them for a moment, they are usually walking back and forth from point A to point B. Some of the more outrageous examples are Neverwinter Nights, World of Warcraft – in which they either stand still or walk around back and forth – and Dungeon Siege II – in which they stand still or, even worst, do the same task for all eternity. And finally, the Individual Memory is not present in any of these games. Sometimes there is even no memory, we do some action and afterwards they will treat us like nothing happened, and some other times there is a collective memory (e.g. in Fable, every character has the same opinion about the player’s reputation, and if the player accomplishes something everyone automatically knows about it).

2.3.1 Neverwinter Nights

“Neverwinter Nights (NWN) is a computer game set in a huge medieval fantasy world of Dungeons and Dragons. This role-playing game (RPG) puts you at the center of an epic tale of faith, war, and betrayal. As a player, you are able to choose what skills and abilities you will develop as you voyage through the complex and dangerous fantasy world of Forgotten Realms. Be a deadly and dangerous Rogue moving through the shadows using stealth and secrecy, be a scholarly Wizard and wield powerful magic against your enemies, be a hulking

Barbarian whose lust for battle is matched only by his terrible rage, be an armor-clad Paladin who protects the innocent and vanquishes his foes, be a crusading Cleric who heals the sick and defends the helpless… be all this and more.” BioWare [4]

Neverwinter Nights is a RPG that was released by BioWare in 2002 and was considered one of the most acclaimed games of this type. It follows the rule sets of Dungeons & Dragons 3rd edition, so it was a typical western RPG. One other aspect that made this game very popular was the ability that the users had to create their own worlds and adventures (i.e. a MOD4), thanks to BioWare’s Aurora Toolset.

Figure 2.1 - Screenshot from Neverwinter Nights.

2.3.1.1 Social Relationships with the Player

There is no great interaction between the player and the NPCs. They just interact on two occasions: when we talk to them explicitly or when some kind of “special NPC” wants to get our attention and keeps on repeating the same phrase until we engage in conversation with her/him. Other than that, there are no real social relationships between the NPCs and the player. And the biggest flaw with this is that, although we are always walking with our henchman following us, we have no interaction her/him. S/he is like a complete servant, never disagreeing, or even giving an opinion to the player. This can only be seen as a master-slave relationship.

2.3.1.2 Social Relationships with other NPCs

When it comes to social relationships between the NPCs, the game tries to create an illusion that some of them interact with each other. We can find some simulated conversations, but even those are very shallow interactions. They simply stay there facing each other like they were having conversations through all the game, never moving.

4 Partial or total modification of a game. 9

If we approach a NPC that is “talking” with another one, that conversation stops and s/he gives us her/his full attention, while the other one just stays there waiting for your conversation to finish. There are some plot related relationships, like NPCs that claim that they are lovers (although we do not see them ever together), or some expressions of hate that one could see as a reflection of an enemy relation. But all is expressed through conversations.

2.3.2 Knights of the Old Republic II

“The Sith Lords is the next chapter to Knights of the Old Republic; winner of more than 35 ‘Game of The Year Awards’ featuring an all-new and perilous story line set approximately 4,000 years before Star Wars Episode I. Choose the light or the dark side of the Force based on your characters actions as you progress through the story. The choices you make not only affect your character, but also those in your party, and those who may join you in your quest.” LucasArts [26]

Knights of the Old Republic II (KOTOR II) was released in late 2004. It was a highly anticipated release since it was the sequel of the first KOTOR and, like its predecessor, won various awards. It uses the Star Wars’ universe as background, which only by itself attracted a very big audience. As we examine the various features in the NPCs, we will focus on the NPCs of the player’s team. The player can have as much as two NPCs at a time in her/his team that following her/him around, which the player can choose to control explicitly – and probably will do so in order to survive in battles. We focus on these because the others are really simple and most of them really dumb. The “primary” NPCs – the ones that influence the story / quests – are simple. They just stand there, waiting for us to interact with them. The “secondary” NPCs are just plain dumb! Not even taking into account what they say when the player interacts with them (they repeat the same messages in a loop and, even if the 3D model is different, the voice is always the same, depending on the species/gender), but it goes to extreme cases that if we talk with most of these NPCs, they “forget” about the path route they were following – not that it mattered much, because they just go back and forth from one place to another – and stay there still… for all eternity! Not even if we reload the game they will start moving. Basically we end up with cities populated by statues.

Figure 2.2 - Screenshot from KOTOR II.

2.3.2.1 Social Relationships with the Player

Between the player and her/his team, there is an Influence System. Basically, if we execute some actions that follow the “moral alignment” of our team member, s/he will be more inclined to give us new information. This information is tied with story/back-story. But this system is also a little limited in its implementation. If we have a team member whom does not approve of our actions, s/he will still help us fulfil them. We might get a disapproval speech afterwards, but that is about it. And this also brings up another meter that the game implements – the Moral Slider. We have a moral alignment based on what actions we decide to execute. It is quite a simple scheme: if we do an evil/unpleasant action we will gain points towards the bad (dark) side, and if we do a good/pleasant action we gain points towards the good (light) side. Sometimes, without really noticing we end up choosing an option in a conversation that influences our moral alignment when we did not intend to. This defines if our character is good or bad, and everyone will see us the same way.

2.3.2.2 Social Relationships with other NPCs

The only bits of social relationships between the NPCs are presented in cut-scenes or dialogs. Which is comprehensible since our team always following us around, and the rest of the NPCs are basically standing still.

2.3.3 World of Warcraft

“World of Warcraft is an online role-playing experience set in the award-winning Warcraft universe. Players assume the roles of Warcraft heroes as they explore, adventure and quest across a vast world. Being ‘Massively Multiplayer,’ World of Warcraft allows thousands of players to interact within the same world. Whether adventuring

together or fighting against each other in epic battles, players will form friendships, forge alliances, and compete with enemies for power and glory.” Blizzard Entertainment [6]

World of Warcraft is a MMORPG developed by Blizzard Entertainment and was released on late 2004. The NPCs in this game are some of the most simple we will ever find. They either do not move or go back and forth on the same path all the time. So what makes millions of gamers want to play this game? The answer is simple: the human factor. MMOs are characterized by having hundreds (or even thousands) of players in the same server, which means that we can interact with many people and share their gaming experience. In order to attain some objectives we will be obligated to associate ourselves with other players. This way people will end up making friends, which will be a big factor towards the longevity of the game. And because we already had this social factor from the players, the NPCs were transformed into immobile sources of information and items, which will always known how and where to find.

Figure 2.3 - Screenshot from World of Warcraft.

2.3.4 Dungeon Siege II

“The first Age of Aranna was ruled by an overlord with absolute control of magic itself, and ended in a cataclysm that consumed both he, and the amassed armies of the rebellion. In the millennium since that fateful day, the magic has flowed freely to any who would command it, but the tide has finally begun to wane, heeding instead the call of its ancient legacy. From the Plain of Tears a new heir beckons, and with each day his strength

grows. Consuming those who submit, eradicating those who resist. With none to oppose his ascendance, the circle will be complete, and an ancient age will begin anew. What fate will befall the Second Age? The time for judgement is nigh, and in Dungeon Siege II, the sequel to the best-selling RPG from Gas Powered Games, you have been called upon to wield sword and spell as the last hope of Aranna.” Gas Powerd Games [19]

Dungeon Siege II was developed by Gas Powered Games and was released on August of 2005. This game does not have really that much when it comes to NPCs, it is quite like World of Warcraft. Nearly all of them are statues, and the ones that are not walk back and forth or are repeatedly doing the same task (how long does it take for that NPC to actually nail those two boards together?!). The NPCs that have something useful to say to us have a distinctive icon above their heads (e.g. question mark for quests) or have a distinctive name (e.g. Helpful Guard). Other than that, the “evildoers” are always in the same zone, walking around until they spot some enemy, after which they will start to rush towards her/him, and, with some luck, we can isolate and kill them one by one since they do not interact with each other.

Figure 2.4 - Screenshot from Dungeon Siege II.

2.3.5 Fable

“Fable™ is a ground-breaking role-playing adventure game from Peter Molyneux, in which your every action determines your skills, appearance, and reputation. Create your life story from childhood to death. Grow from an inexperienced adolescent into the most powerful being in the world. Choose the path of righteousness or dedicate your life to evil. Muscles expand with each feat of strength; force of will increases with each work of wit.

Obesity follows gluttony, skin tans with exposure to sunlight and bleaches bone-white by moonlight. Earn scars in battle and lines of experience with age. Each person you aid, each flower you crush, each creature you slay, will change this world forever. Fable: Who will you be?” Lionhead Studios [24]

Fable is an adventure RPG developed by Lionhead Studios and was released on September 2004 for the Xbox. An extended version of it – Fable: The Lost Chapters – was released in 2005 for Windows, and afterwards was ported to Mac OS X. Fable is the only one of the RPGs that we evaluated that has a more eastern feel to it. We do not have character customization (in the sense of choosing race, gender, although we can choose clothes, tattoos, hair cuts) and it does not force you into choosing a side (i.e. moral alignment) or class (e.g. warrior, sorcerer) like most of the RPGs for personal computers. Instead it moulds our character based on our actions and choices. For example, if we keep stealing or hurting innocent people we will gain points towards the “evil” side, and NPCs will react to us accordingly (i.e. like in KOTOR II).

Figure 2.5 - Screenshot from Fable.

2.3.5.1 Social Relationships with the Player

Social relationships between the player and the NPCs are created and maintained through simple interactions, which consist of simple actions like standing in a heroic pose or doing a little dance. These interactions act on a love/hate axis, and according to our actions, the NPCs’ relationship with the player will slide up or down. However these actions do not have that much of an effect on the story or even how the NPCs see us. The biggest influence that the relationships may have on our gaming experience (i.e. not on the story itself!) is help us get a wife, with whom we will not have that much of an interaction afterwards.

When we interact with a NPC, s/he will completely mess up whatever task s/he was doing. For example, if we interact with the waitress of the inn – whom goes around filling up people’s glasses – she will loose her jar, which will magically reappear on the counter. Another example is the box carriers, whom will drop the boxes if we interact with them (which by the way breaks and magically disappears, not even making him at least be angry for making him break his box). And the NPCs that do not stop doing the task when we interact with them, will stop when we get close to them, and will give us their complete attention (e.g. blacksmith). This usually happens with all NPCs – they notice us. By this we mean that when we pass by them, we hear them talk to/about us, and with the great animation system that the game has, we can see really fluid reactions like a character clapping her/his hands while cheering for us or something as simple as looking at us. But still the lack of individuality makes it fell like something is missing. For example, if we break into someone’s house they will call the guards, and treat us in a “hostile” way, but right after we pay the fine they will see us like everyone else.

2.3.5.2 Social Relationships with other NPCs

Unfortunately, when it comes with relationships between the NPCs, it is not as important as with the player. NPCs do not usually notice each other. Some of them, like merchants or blacksmiths, do not interact with the other NPCs like they do with the player. There are some scripted events which can give a nice illusion, like the conversations between the NPCs or when the NPCs are in their houses and sit down at the table to eat their meals. Other than that, there is not that much of social interactions between them, even when it comes to family.

2.3.6 The Elder Scrolls IV: Oblivion

“Bethesda Softworks LLC, a ZeniMax Media company, today announced that The Elder Scrolls IV: Oblivion™ is currently in development for PC and future generation consoles. In development since 2002, Oblivion is another leap forward in role-playing with its combination of freeform gameplay and cutting-edge graphics. Oblivion is the sequel to the best-selling, award-winning role-playing game, The Elder Scrolls III: Morrowind®, named 2002 RPG and Game of the Year for both PC and Xbox.” Bethesda Softworks [46]

The Elder Scrolls IV: Oblivion was developed by Bethesda Softworks and was released on March 2006. Like it says in the title, Oblivion is the fourth release of this saga. We were expecting a much richer social environment from what the developers said prior to the game’s release. The conversations were one of the features they focused a lot, but they feel very shallow (we will explain them better in the next sub-sections).

Figure 2.6 - Screenshot from The Elder Scrolls IV: Oblivion.

2.3.6.1 Social Relationships with the Player

There is not much to say about social relationships between the NPCs and the player. There is a persuasion system, which means that the player can use her/his “conversational skills” to get more information out of the NPCs, although s/he can simply use a charm spell or bribe them. NPCs notice the player; they look and greet her/him when s/he is close. However, sometimes it is awkward. When the NPCs go to sleep at night, they get in bed even if the player is there right beside them, or if we break into their houses there is a good chance that if we wake them up they will just talk with us like nothing happened, and when the conversation ends they go right back to sleep.

2.3.6.2 Social Relationships with other NPCs

When it comes to social relationships between NPCs, there is not much going on. The developers do try to simulate a social behaviour by creating conversations between two NPCs, but it looks like it is made by a simple rule – if the NPC has not talked to another one in some random amount of time, starts a conversation. Once we even saw two NPCs ending a conversation and as they were walking away they started talking to each other again. Unfortunately the conversations are dull. They clearly sound like hints directed to the player. Although the ability to eavesdrop on NPCs having a conversation is a nice feature, but if we get right in their faces they do not even notice us. We can even push them away from each other, and they will keep on talking 10 meters apart from each other.

2.3.7 RPG Comparison

Table 2.1 shows a summary of the features analyzed in the previous sub-sections. The legend is as follows: • Never – no NPC has this feature; • Occasionally – we see it happen on some random characters, not really a pattern;

• Only on primary NPCs – when it occurs with NPCs that have a bigger part in the game (e.g. main characters or NPCs responsible for quests); • Always on all NPCs – all the NPCs have this feature. Social Relationships with Social Relationships with Game the Player other NPCs Neverwinter Nights Occasionally Occasionally Knights of the Old Republic II Only on primary NPCs Never World of Warcraft Never Never Dungeon Siege II Never Never Fable Always on all NPCs Occasionally The Elder Scrolls IV: Always on all NPCs Occasionally Oblivion

Table 2.1 – Comparison of the social relationships of the NPCs across multiple RPGs.

As we can see in Table 2.1, game developers are more focused in social relationships between the player and the NPCs than amongst the NPCs. When it comes to social behaviour between users and the NPCs, we believe that only Fable and Oblivion do anything substantial about it. It is our opinion that although Fable came out a long time before Oblivion, it does a much better work in social behaviour and even in the actual believability of their characters, having a great body language that does a great job in expressing the characters’ emotions.

2.4 Academic Research

The academic research in social behaviour is basically separated into two views: the NPC- to-NPC interaction and Player-to-NPC interaction. The Player-to-NPC interaction had the objective to create characters that look and fell more real when they interact with the player. Researchers started to take a look at other sciences, like psychology and philosophy, trying to duplicate characteristics like personality and emotion, in order to make NPCs feel more alive to the users [3]. The other view, the NPC-to-NPC interaction began by focusing on the communication between NPCs [13], usually associated with simple tasks like trading agents – which started to get some attention from the community with the generalization of the Internet. However, only recently researchers started to take a bigger interest in more complex NPC- to-NPC interaction, where the believability of the social interactions is important. We will now describe some of the work that has been done in this area.

2.4.1 Avatar Arena

2.4.1.1 Summary

Avatar Arena [44] is a system in which a task is delegated to an autonomous character (i.e. usually referred to as an agent). The task in this case is a negotiation between the characters (i.e. avatars5). The idea was to create a system that resembled a human-human negotiation, in which not only rational argumentation but also the social context and the personalities of the negotiating parties need to be taken into account. The authors focused on how social relationships could impact a character’s behaviour. In order to build the Avatar Arena, the authors defined that the following five features would have to be implemented in an avatar: 1. Understanding of the domain; 2. Awareness of the personal preferences of their users; 3. Personality and affective behaviour; 4. Social relationships with the other avatars; 5. Conversation and negotiation skills. To model the characters’ personality they based on the Five-Factor Model (FFM) [30]. The FFM is a descriptive model with five dimensions: Extraversion, Agreeableness, Conscientiousness, Neuroticism and Openness. However, to simplify, they chose to use only three of these dimensions – Extraversion and Neuroticism because they can be found on most of the theories, and Agreeableness because it is important when dealing with social relationships. As for affective behaviour, they did not implement it at the time, although they were planning to use a framework for modelling affective states of an agent that was being developed by Fiorella Rosis’s [45] group at University of Bari. To model social relationships between the characters, they based in socio-physiological theories of cognitive consistency, specifically in the Heider’s Balance Theory [20] and Festinger’s Theory of Cognitive Dissonance [12]. The Balance Theory establishes a relationship between two persons and an object – that can be an item, another person, etc. -, which as these persons perceive the world may stay in a balanced state or become unbalanced, making the persons re-evaluate it.

5 An avatar is a representation of a user, usually seen in Internet forums, chat-programs and games. 18

Figure 2.7 - Example of an unbalanced cognitive configuration [44].

In Figure 2.7 we see that the Parrot gives importance to the Genie and to the career, so the Parrot assumed that the career was also important to the Genie (i.e. balanced state). But when talking to the Genie, the Parrot realizes that the Genie has no interest in issues related to the career, making the Parrot’s cognitive configuration of that relationship become unbalanced, and forcing him to re-evaluate it in order to attain balance. Rist and Schmitt observed that the characters took into consideration the social distances to their partners as well as the beliefs about their values and the other character’s values when engaging in dialogues, although they admit that their work had a very simplified model of the underlying theories.

2.4.1.2 Conclusions

This system uses an interesting idea: the Balance Theory. It creates a Theory of Mind, which means that one agent has some beliefs about another but also creates a mental model of what s/he thinks that are the beliefs of the other. Then the relationships are adjusted towards the idea of balance. One point that we think that could be explored is the creation of a historical record of these relationships. For example, character A likes B but when C appears he starts to like C and dislike B. Then, if C disappears, the previous relation that A had with B could be retrieved (or not) based on their historical records of their relationships. Like many of the systems evaluated, it does not focus on computation intensive environments like a computer game; it focuses on a very limited and controlled scenario.

2.4.2 PsychSim

2.4.2.1 Summary

PsychSim is a multiagent-based simulation tool for modelling interactions and influence among groups or individuals [38] [39]. It is based on the fact that the message exchanged between two agents, like in human communication, depends not only on its content but also on the way we see our communicator. Therefore, our actions take into consideration our view of the communicator and also the reaction we expect from her/him.

Figure 2.8 - Screenshot from Psychim's interface [38].

This means that PsychSim revolves around the notion of Theory of Mind. Each agent has her/him own decision-theoretic model of the world including hers/his beliefs about the environment and a recursive model of the other agents. With such a model, the agent has the ability to judge the degree of credibility of a message in a subjective fashion, just like humans do. Each agent has represented the following features: • State – the real state of the agent, objective facts about the world that he may not even know (e.g. strength of the agent); • Actions – a set of actions that the agent can choose to perform; • Goals – a set of goals that represent its incentives (or not) for a certain behaviour; • Beliefs – subjective view that the agent has about the world; • Policies – rules to select an action or message based on the agent’s goals and beliefs; • Messages – information that can used by an agent or the user to influence the beliefs of the recipients; • Mental Models – representation of the mental models that the agent believes the others have (includes goals, beliefs and policies). Of these, the most “interesting” is that an agent has mental models of the other agents, what s/he “thinks” that is true. This means, for example, that an agent might have a belief about another agent’s beliefs. These models may be influenced, and make the agent change its beliefs. This influence is attained through the messages exchanged by the agents. In a survey of the social psychology literature, the authors identified four key factors important to determine a possible influence that one agent may have on another: • Consistency – people expect, prefer, and are driven to maintain consistency and avoid cognitive dissonance;

• Self-interest – the way we analyze information and the inferences we draw are biased by self-interest; • Speaker’s Self-interest – if the sender of a message benefits greatly from the recipient believing it, there is often a tendency to be more critical and for the influence to fail; • Trust, Likability, Affinity – the relation to the source of the message, whether we trust, like or have some group affinity for him, all impact whether we are influenced by the message. In order to demonstrate this tool they implemented a bullying scenario. The objective was to see how the bully would act depending on various factors (bully type, whom was punished, model of the students, model of the teacher) after having bullied the victim the first time. The outcome was successful if the bully did not choose to act out violently towards the victim the second time.

2.4.2.2 Conclusions

Although PsychSim has some very good ideas, like the recursive theory of mind, it seems that this system is too computationally intensive for real-time applications like games. Especially if we have a big depth of the recursive model, the number of agents has to be drastically reduced (basically their scenario can seen as only three agents – the bully, the teacher, and the group of students, since students act the same way). Another interesting idea is the influence system, which is tied to the theory of mind. What the agent believes about the others affect what s/he will choose to do. The other way around is also possible; the influence that a character has on another may be used to persuade her/him to do (or believe in) something.

2.4.3 Social Role Awareness

2.4.3.1 Summary

Social Role Awareness was the work of Prendinger and Ishizuka [37], in which they propose that animated agents should have a notion of social roles when interacting with other agents, and adjust their behaviour accordingly. Humans do not act always the same way; they choose their actions according to their personality but also to the context where they are inserted. For example, a secretary may address the cleaning woman in a whole different way than she addresses her boss. The goal of the authors work was to create autonomous agents that would be much more interesting conversational partners for language conversation training. These agents have a standard theory of reasoning about emotions – the model of Ortony, Clore and Collins [35], also known as the OCC model – and a simple model of personality. They implemented Moulin and Rousseau’s [33] approach to model and simulate conversations.

Moulin and Rousseau’s model distinguishes three levels of communication: communication level, conceptual level and social level. Prendinger and Ishizuka’s system integrates the latter two – conceptual and social levels. As for personality, they considered only two dimensions from the FFM model – which according to André et al. [1] are essential for social interaction – to keep things simple: Extraversion and Agreeableness. The core of the authors’ work is basically a social filter. Before choosing a particular action (e.g. express an emotional state), it will go by the social filter and will determine the agent’s behaviour based on her/his personality and social role. The agents do not know about the others’ beliefs or mental state, allowing this way the agents to cheat in their behaviour by expressing misleading emotions. To test this system the authors implemented a coffee shop scenario, and showed it to the users to see if they could identify the roles of the agents and if the responses of these agents were believable. The authors only tested it with five users that easily identified the roles. Just on the side note, the users did not find the agents without the social role awareness (i.e. only reasoned about personality and emotions) to be unbelievable. In the conclusion, the authors affirm that it is their belief that this approach adds value for three reasons – believability, social communication and explanatory power – although there is not a very strong user evaluation to back this up.

2.4.3.2 Conclusions

This work does not seem to have a great user evaluation to backup their conclusions. We cannot really say if it makes a difference to the users or not. The idea seems interesting but we think that it should probably be seen in a different way, not that much related to the social roles but to the context. Perhaps this context would take into account the social roles, but would have much more detail (e.g. if our boss is not polite with us, we will probably act different than if he was). The system as it is looks suitable for very strict hierarchic models. Probably the attributes for personality should be carefully reconsidered, since it is common to see Neuroticism in most of the psychological theories [44].

2.4.4 The µ-SIC System

2.4.4.1 Summary

The µ-SIC System [28] was developed as a part of a bigger project, an Agent Architecture for Proactive Persistent Non Player Characters [27], where Mac Namee and Cunningham propose a “new” architecture for agents. In fact, the authors do not call them agents but Proactive Persistent Agents (PPAs). The main difference between PPAs and regular agents is that they should always be persistent (i.e. active). Usually, the agents are in a “stasis”, and when an event is triggered they wake up. What they propose is that the agents should always be active. But the fact that agents are always

22 active arises the old philosophical question “Does a tree falling in the forest make a sound if there is no-on there to hear it?”, which means that should the agents be doing something if the player is not there to see it? What if the agent is doing something that is important for the player to understand the storyline? The authors mention that a level of detail (LOD) of AI may help control this issue. The LOD of AI is just like the LOD of geometry in computer graphics; agents that are closer to the camera (i.e. player) have a more profound AI reasoning than agents that are further away. They mention that the PPAs should have the following properties: • Autonomy; • Reactivity; • Pro-activeness; • Social ability. The first three are common in today’s agents, so their main suggestion is to introduce social ability to the agents. The architecture proposed for the PPAs is very similar to Hybrid Agents6 (see Figure 2.9).

Figure 2.9 - Architecture proposed for the PPAs [27].

As we can see, one of the additions is the Social module, which although it is reactive, the authors chose to keep it separate from the Behaviour System module (i.e. Reactive module) because some applications may not require it. The other addition is a Schedule module, which is basically a schedule of the agent (e.g. says when they go to work, home, socialize). Except for the Social module, all the others do not have anything new to them. The Social module is the µ-SIC System [28]. It consists of and Artificial Neural Network (ANN) that will be responsible for determining which social interactions the character should engage with another one. Basically it determines

6 The agents are both reactive and deliberative. 23 the character’s social behaviour and maintains her/his relationships with both players and other NPCs. The µ-SIC System uses three psychological models: (A) the Eysenck personality model [11], (B) the Lang mood model [22] and (C) the relationship model (see Figure 2.10).

Figure 2.10 - Psychological models used in the µ-SIC System [28].

The Eysenck model (A in Figure 2.10) plots the character’s personality across two orthogonal axes, introversion-extraversion and neuroticism-stability. It is worth noting that psychologists generally agree that two axes are not enough to accurately model the whole gamut of human personality types. The Land model (B in Figure 2.10) maps the mood also in two axes, valence (i.e. whether the mood is positive or negative) and arousal (i.e. intensity of the mood). The relationship model (C in Figure 2.10) has been used in other entertainment projects (e.g. Oz Project [42], TALE SPIN [31], UNIVERSE [23]) and has its psychological basis on the work of Wish et al [53]. Traditionally, the model uses four values to characterize the relationship of one character to another, however to facilitate conversation they extended the model with a value that indicates how interested a character is in another (i.e. Interestedness). Like mentioned above, the system uses some simplifications to model the characters, like in the case of the personality model. The reason for this is that the authors’ architecture is created towards real game use, not just simple demonstration projects like the other works. Computer games are really greedy applications, and we need to create solutions that are fast in their processing. This and simplicity – we need systems that the developers can easily understand and use to create their agents – are the reasons why some options might not have been used (e.g. a more complex personality model, like the FFM [30]. The big problem with ANNs is that we need to get the data to train it. The problem is that o such data exists for something like this, and they needed to create this data on their own. The

24 network achieved an accuracy of 85%, and the authors were satisfied with it because in the cases of incorrect prediction it was rarely significantly incorrect (e.g. the system produced a chat interaction instead of a joke interaction, but it would never produce an assault interaction instead of a kiss interaction).

Figure 2.11 - Screenshot from a simulation situation [28].

Figure 2.11 is a screenshot of the application created to test the authors’ work. It is a bar and the agents roam around interacting with each other. The authors considered the system to be quite successful although there was not any real user evaluation on whether or not this would improve the game experience. The biggest limitation that the authors identified in their system was that the interactions were a simple action-reaction system, for example a character flirts with another character and then the other one flirts back, ending there the interaction. To improve their system, the authors proposed inserting a new node to the ANN indicating a context, so that multiple interactions could be coherently connected (e.g. after flirting they started telling jokes to each other).

2.4.4.2 Conclusions

This system is the first one to truly be focused on games. It focuses on the architecture as a whole, considering for example the reactive layer. The agents proposed by the authors are hybrid, not like the other platforms that were more focused on deliberative behaviour. Although we are very reticent about the social module being exclusively reactive; this module should probably have two layers, one reactive that does things like say hello to a character that he recognizes, and another one deliberative that can create more elaborate social interactions (e.g. be nice to the boss because the character wants to marry his daughter and take his money). Their system focuses on simply choosing the right interaction.

2.4.5 Demonstrate!

2.4.5.1 Summary

Demonstrate! is the work developed by Rhalibi, Baker and Marebti [43], and also focuses in computer games. The goal was to create an agent architecture that would be able to represent teamwork dynamics through the representations of emotions, beliefs, and relationships. The authors’ architecture was based on Abraham Maslow’s Hierarchy of Needs [29] (see Figure 2.12).

Figure 2.12 - Abraham Maslow's Hierarchy of Needs Pyramid [52].

The lower levels – Physiological and Safety – are reactive. The three upper levels are involved in the agent behaviour when more complex interaction is required. With this architecture, the authors developed a game entitled Demonstrate!, which consists in a political street demonstration.

Figure 2.13 - Screenshot from Demonstrate [43].

The agents for this game had two “separate” reasoning states: The lower level – default agent reasoning, which is focused on the reactive level of the Hierarchy of Needs – and a higher level – the team-based agent reasoning, which is focused on the three upper levels of the Hierarchy of Needs. The emotional reasoning was based on the OCC model [35], comprising four emotions which values ranged from 0 to 1: Fear, Love/Belonging, Pride/Shame, and Happiness. The authors then defined a set of functions that control the decay of emotions based on personality types (e.g. hero, normal, coward). The agents’ interactions and relationships were modelled based on the Iterated Prisoner’s Dilemma [2]. When the leader – which is controlled by the user – proposes an action, the agents will choose to either cooperate or defect, based on their personality and emotions.

2.4.5.2 Conclusions

This system is also more directed towards games. They do however focus on a very limited scenario; it was even implemented with a 5 states Finite State Machine (FSM), unlike the µ-SIC System, which was a general architecture. They clearly consider the social interaction – although a little limited, since the agents just choose to cooperate or defect, there is nothing as interesting as the emergence of a new leader (it was always the player) – to be in a deliberative level, which in our opinion makes more sense towards the creation of richer social behaviours.

2.4.6 Comparison

In the research we reviewed, only the two latter systems seem to have a clear goal in creating a platform for real-time multi-agent games. Of course that all of them add some very good ideas towards the creation of a better social behaviour for agents, but the problem is that they are usually applied to more simple and controlled environments, not to something as open and complex as a RPG. Most of them seem to understand the need to have a model of the character’s personality, however it is not unanimous which of the dimensions to model – except for Extraversion. There is not also a consensus if the social layer should be deliberative or reactive, but most of them implemented it in a deliberative level (except for the µ-SIC System). It is our opinion that a rich social behaviour will always need to be – at least to some extent – in a deliberative level. The idea of theory of mind and how it may influence our – or be used to influence another one’s – interactions is also very interesting; the fact that we act based not only on ourselves but also on how we think that the other ones see us.

2.5 RPG Gamer Survey

To search for evidence that players, indeed, expect for better social behaviour from NPCs a pilot study was performed. This study consisted in a questionnaire that was published in the Internet and disseminated in several gamers’ forums [5] [7] [14] [16] [17] [25] [32] [34] [36] [47]. The questionnaire was removed one month and a half later with 250 complete answers collected. It can be found in appendix A. One of the aims of the study was to find if there was any significant difference between what players expect from the NPCs and what they found on the RPGs they play. Thus, players were asked to rank (from 1 - 4) the features mentioned on section 2.2 that we believe to be relevant for NPCs’ believability, regarding to what they desire (e.g. how important they feel they are for game play) and what they found from their gaming experience. Just to recall, the features inquired were: 1. NPCs’ Personality; 2. NPCs’ Emotions; 3. NPCs’ Pro-activeness; 4. NPCs’ Individual Memory; 5. NPCs’ ability to form social relationships with players; 6. NPCs’ ability to form social relationships with other NPCs.

When comparing the answers, there was a significant difference between what players desire and what players found regarding all features – Personality (T=1556.50, p<0.001, r=- 0.44), Emotions (T=1629, p<0.001, r=-0.36), Pro-activeness (T=1519.50, p<0.001, r=-0.40), Individual Memory (T=1639, p<0.001, r=-0.39), Relationships with the Players (T=1346, p<0.001, r=-0.43) and Relationships with other NPCs (T=1546, p<0.001, r=-0.43). From the results above, we can see that there is still a lot to improve regarding NPCs believability. In addition, the difference seems to be higher (r > 0.4) relatively to personality, and the ability to establish social relationships with both players and other NPCs. This supports the importance of our ideas of enhancing the social behaviour of NPCs. In the questionnaire, we also asked for some characteristics of players (e.g. gender), some information about their gaming habits (e.g. how often they play, what kind of games they play and if they ever playing tabletop RPGs) and, finally, if in the overall they believed that a more active social behaviour from NPCs would improve their gaming experience. From what we observe, players (mostly male – 93%) play several times a week. They like to play RPGs (90,4%), Strategy games (52,8%), Adventure games (48,8%), and Action games (40,4%). These results are natural since most of the forums where the questionnaire was disseminated were related to RPGs. Looking at the correlations between some of the questionnaire’s items, we found some interesting results. Players that played tabletop RPGs seem to identify more flaws in computer RPGs regarding characters’ personality (N=250, r=0.14, p<0.05) and pro-activeness (N=250,

28 r=0.14, p<0.05). Also, players that play more often expected more from characters’ pro- activeness (N=250, r=0.16, p<0.01) and believe more on the importance of improving the social abilities of NPCs to improve the role-play experience (N=250, r=0.13, p<0.05).

Chapter 3

Conceptual Model

3.1 Introduction

As we could see in the previous chapter, there is still much to be done to attain believable social behaviour in computer games. Not just our analysis of games tells us this, but also gamers say that they would prefer NPCs with richer social behaviour. As mentioned in section 1.3, we propose to improve the social interaction by introducing relationships between the characters. However, we consider that there have to be some additional concepts to help create rich social interactions between the characters: personality to obtain individual believability, conversation to be able to communicate with each other, memory to retain information from their interactions, theory of mind to be able to have a mental representation of each character, and information transmission to propagate information. In the following sections we will explain all these concepts and why we believe that all of them are necessary to improve social behaviour.

3.2 Relationships

We consider relationships the most important concept because it represents our social connection with each person, it influences how we interact with each other, and without them there is no real social behaviour. In RPG we only have a glimpse of relationships in scripted events, like cut-scenes, which, besides being pre-defined, are scattered along the story and do not create a continuous social behaviour. Some of the recent RPGs try to give a notion of relationships. One common case is characters living in the same house, although we do not see behaviour such as the man, which we assume that is the husband, kiss the woman before going out. The basis of our relationship system is based on Heider’s Theory of Balance [20], in which the relationships are a ternary association between two persons and a concept (see Figure 2.7). However, since we are not focusing on the relationships being dynamic at the moment, we will

31 not take into account the effect that an unbalanced state may have on a character’s relationships. For the representation of relationship we consider a simplification of the relationship model proposed by Wish et al [53], which may be extended in a future version. We only considered one dimension, Likes/Dislike axis, since we considered it to be one of the most expressive and easily understandable dimensions. A high value in this axis can be easily understood as a positive relationship between two characters (i.e. NPC A likes NPC B), and the opposite for a low value. We acknowledge that only this axis is not sufficient to accurately map the gamut of relationships, but our focus at this moment is not to accurately map relationships but instead to see if a minimum of relationships will affect the gaming experience of RPG players.

3.3 Personality

In real life, not everyone acts or reacts the same way. For example, if a bully starts beating up a kid in school, it is not sure that everyone will back off or stay and see the fight. Some might also engage in the fight, others might go and find help. Furthermore, not everyone will see the event the same way, some will stay on the bully’s side, some will stay on the victim’s side, and others will not even care about the incident. Of course there are other factors that influence where we stand, but our personality is at the core of how we might react in a given situation. Also in games, personality is one of the main differences between the primary (e.g. plot/quest related) and secondary characters. It is not common to find the same personality that a primary character has on another character, but the personalities of secondary characters appear quite often replicated (e.g. all the merchants have the same personality). This means that the personality is the core of our individuality, and to create believable social behaviour, we need to create believable individual behaviour, which has its basis in personality. We based our concept of personality on a simplified version of the FFM [30], only considering one of its dimensions, Agreeableness. We chose only this dimension for the same reasons that we chose only one dimension for representing relationships, we want a dimension that is easy to understand and measure, and because our focus is not accurately represent all of human personalities, but see if just by adding a simple notion of this concept, the RPG players will believe that their experience is improved. We do not consider that only this dimension is suitable for representing personality, and we believe that it would have to at least include the dimensions present in Eysenck model [11] (see A in Figure 2.10), Extraversion and Neuroticism, since there seems to be a consensus that they are fundamental in implementing relationships [1] [28] [44].

3.4 Conversation

In a RPG, the player needs interact with the NPCs in order to progress in the game.To obtain the information that will allow her/him to progress, s/he needs to have conversations with the NPCs. Conversations are the simplest way to transmit information from the NPCs to the player, however we propose that this concept is extended to all characters, allowing for conversations between the NPCs. We do not consider that two NPCs facing each other are conversations, there needs to be actual information transfer between them (see section 3.7), since it will allow for the propagation of information throughout the game world, which will create emergent behaviour. We consider that a conversation is a group of messages exchanged between two characters, which provide a flow of information between them (the information may be something as simple as two characters saying hello to each other). A message has the following fields: • Sender – who sent the message; • Receiver – to whom is the message; • Performative – what kind of message it is; • Text – the text shown to the player in-game; • Object – an object associated with the message. Some of the fields are self-explanatory, like the sender, the receiver and the text. The performative is a way to tell what is the meaning of the message. For example, one performative could be “buy-item”; when received, the receiver would know that the sender wants to buy an item from him. The object is the “focus” of the performative. Sometimes, the performative focuses on an object, which can be an item, a place, a character, information, etc., and we must have a way to convey that information in our message. Taking the previous example, the object of the performative “buy-item” would be the name of the item that we want to buy. The conversations also have to allow the initiation to be made by a message with any performative. For example, when addressing to a merchant we could start a conversation by greeting the merchant, or skip that part and tell him right away that we want to buy an item. It is important to allow a conversation to start anyway we want, because this kind of behaviour also has an influence on the relationships between the characters (e.g. if we do not start a conversation by greeting someone, we might be seen as being rude). Messages can be grouped and sent together. If messages are sent only one at a time, alternating between each speaker, it is an action-reaction system, with the second speaker being an passive element. By allowing to send two or more messages at a time, we are able to create situations like NPC A asks NPC B about X, and then NPC B answers about X but also asks Y to A (see Figure 3.1).

Figure 3.1 - Conversation between two NPCs.

A conversation also has to maintain a context, which may be something just as simple as the history of messages during a conversation. When a character is talking, s/he will take into account not just the message s/he just received, but also messages received previously during that conversation. Finally, we believe that this is not a final approach to conversations. We believe that this concept should be extended at least to provide two additional features: multiple character conversations, and the ability to eavesdrop on conversations. The multiple character conversation is not just to be able to have conversations between three or more characters, but also being able to intrude on conversations. For example, two characters are talking and we start talking to one of them, which will introduce us to the other intervenient. The eavesdropping feature is simply the idea that any character, not just the player, will be able to hear conversations, in which they are not taking any part of. This feature is already present in some RPGs [46], although only the player can eavesdrop on NPCs.

3.5 Memory

The memory is a concept that serves as a support for all of the other concepts, except for personality. The memory is simply a repository of information. Each character has its own memory, and the only way to populate the memory is through information transmission between the characters (see section 3.7). The information stored in the memory can be searched and deduced, transforming the simple bits and pieces of information into something with more value, knowledge.

For example, if we have a representation of a character (e.g. name, age, gender) that contains a reference to a profession, we need to be able to go from that character’s info and get to the character info of all that have the same profession. The individual memory is important because it reflects the way we interact with each other. For example, if we talk with a person we will remember her name, and when we see her again we will remember that we have already talked to them, conversations are not isolated events; we create a mental representation of each person, which we will explore in section 3.6.

3.6 Theory of Mind

A theory of mind (see section 2.4.2) is when we have a mental representation about another person’s beliefs. In our conceptual model we only have the first level of theory of mind, which means that characters have beliefs about the other characters’ beliefs (see Figure 3.2). For example, we may believe that Peter likes Mary.

Figure 3.2 - Level 1 of theory of mind.

The second level of theory of mind is what we believe that a person believes about another person’s beliefs, for example, we may believe that Peter believes that Mary likes John (see Figure 3.3).

Figure 3.3 - Level 2 of theory of mind.

Although this is an interesting idea, we do not believe to be viable at this moment to use the second level of theory of mind in every character, nor that it is important to achieve our goal. But the first level of theory of mind is necessary, since it is important for the characters to be aware of each other, and only then can they make interesting emergent social behaviour.

3.7 Information Transmission

Information transmission is the main reason to make all the characters have “real” conversations, instead of just creating an illusion that a conversation is happening (e.g. two characters facing each other and making some facial animations). In conjunction with the fact that every character has its own memory, this will allow for the information to flow in an emergent way through the game world. The only way for a character to know something is if s/he is present when it happens, or if another character tells her/him in a conversation. We propose that the information transmitted should be based on the personality, the relationship with the receiver of our message and the relationship with the object (which can be a concept, an item, a person, etc.) of our message (see Figure 3.4).

Figure 3.4 - Constrains of information transmission.

However, we need to take into account that our model is designed specifically for games, and this means that there needs to be a simple way for game developers to define what information with each combination of these variables. To resolve this problem, we propose that the three variables are transformed into two in order to be able to map them in a two- dimensional graphic that is easy to visualize and define. The two variables are the personality and the sum of both relationships (see Figure 3.5).

Figure 3.5 - Personality vs Relationship plot.

We can understand better the sum of the relationships if we define values to them. For example, if we consider that a relationship is a value comprised in [-1; 1], the sum of a dislikes relationship (-1) with a dislikes relationship (-1) would result in a dislikes relationship ( max("1+ ("1),"1) = max("2,"1) = "1, since the result is clamped between the boundaries of the relationship values; if the result was positive, the minimum function with 1 as the second argument would be used instead). The possible combinations are presented in Table 3.1. ! Likes (1) Neutral (0) Dislikes(-1) Likes (1) Likes Likes Neutral Neutral (0) Likes Neutral Dislikes Dislikes (-1) Neutral Dislikes Dislikes

Table 3.1 - Possible combinations of the sum of the relationships.

Basically the information transmitted can be seen in the formula presented in Equation 3.1.

"1# relationshipreceiver+object = relationshipreceiver + relationshipobject #1

InformationTransmitted =f (personality,relationshipreceiver+object )

Equation 3.1 – Formula of the information transmitted. ! The implementation of f depends on specific game at hand.

! 37

3.8 Conclusions

This chapter presented the conceptual model that we propose to reach our hypothesis, that if we enrich social behaviour through the creation of relationships with/between the NPCs, the player will have an improved gaming experience. Although our focus is the relationships with/between the NPCs, we believe that the concepts personality, conversation, memory, theory of mind, and information transmission also need to be implemented in order to achieve rich social behaviour. In Figure 3.6 we can see a representation of a character, which contains the concepts of personality, memory, relationships and theory of mind, and how they are connected.

Figure 3.6 - Representation of Character A.

The conceptual model proposed is not meant to accurately map social behaviour such as humans have, but it is a model to see if just the basis of this social behaviour will have an influence on the players’ gaming experience. If the players prefer a system with this basic social behaviour, the next step is to expand this model, especially in terms of creating dynamic social behaviour. Although it is not the focus of our conceptual model, we believe that this conceptual model also allows for the creation of non-linear events, since that if we change some of the parameters (e.g. relationship between two characters), we will have a different set of events. For example, if we have two characters that love each other, one objective could be to ruin their relationship, or if they hated each other the objective could be to bring them together. This is especially helpful to resolve one of the missing features of today’s RPGs: quest randomization. It would add a much bigger replay value to the RPGs, since the experience could vary much more every time they played. However, such a system or emergent behaviour would have to be carefully analysed, since it could ruin the storyline. For example, if the main storyline was that the player had to protect the princess, and we decided to make every character have the ability to kill the princess, such a system would have to constrain any assassination attempts to situations where the player would have the ability to stop them.

Chapter 4

Implementation

4.1 Introduction

Since we are focusing on games, we decided to do the implementation of our conceptual model in a game engine. After testing a few options we chose to go with Unity [48]. Unity is a game engine for Mac OS X – although a Windows version is being developed – that is based on the .NET platform (see Figure 4.1).

Figure 4.1 - Unity Technology Diagram.

The reasons for choosing this game engine were its quality, its ease of use, being available for Mac OS X, a great art workflow, and its great prototype focused development environment. Unity itself, since it is an editor (see Figure 4.2), does not need to be recompiled each time we make a change to our project; only our code will be recompiled, improving the general development workflow and the experimenting and tweaking of our solution.

Figure 4.2 - Screenshot from Unity's Editor.

In the following sections we will explain the AI architecture developed to support our conceptual model and some of the implementation details of our conceptual model.

4.2 AI Architecture

Because we want to implement such a system in a game focused environment, we need to create a system that is not very computationally intensive, since we need to be able to process multiple characters at a time. In order to develop such architecture we started by looking at the basis of multi-agent systems. We started with and hybrid agent architecture (see Figure 4.3).

Figure 4.3 - Base AI architecture.

Five components compose this architecture: • World – the environment itself; • Perception – the character’s mechanism to perceive the world; • Knowledge Base – the character’s information about the world (including her/himself); • Actuator – the character’s mechanism to act upon the world; • Reasoning – the character’s “thinking” process. As mentioned, the architecture is hybrid. This means that it is both reactive and deliberative. The reactive layer encompasses the perception and actuator components, and the deliberative layer is the reasoning component. This architecture was implement via Beliefs, Desires and Intentions (BDI) architecture [8] [40] [41]. The beliefs represent what the character knows about the game world, the desires represent the high level objectives that the character has, and the intentions represent how the character fulfils each desire. It was chosen because it is conceptually very easy to understand. This architecture was then adjusted to encompass our conceptual model (see Figure 4.4).

Figure 4.4 - AI architecture with conceptual model.

The memory is contained in the beliefs. The other component worth mentioning is the message manager, since it is the one responsible for the eavesdropping of conversations.

4.3 Relationships

Like mentioned in section 3.2, the relationships are based on the work of Wish et al [53]. Our model only has one of its dimensions, the Like/Dislike axis. We chose to create a simple implementation of this concept, using three discreet values to represent it. The three values that the relationship can have are: • Likes – if a character has a positive relationship towards another (1); • Neutral – if a character is neutral towards another (0); • Dislikes – if a character does not like (i.e. has a negative relationship towards) another (-1). We chose only three values because they are easy to understand; a continuous representation of the axis, from -1.0 to 1.0 would not be easy to understand because we would have to know how to distinguish a 0.8 value of a 0.85.

We acknowledge that this does not represent all the possible relationships, however we just want to prove that a minimum of relationships will have a positive influence on the player’s gaming experience.

4.4 Personality

Like in the case of relationships, we chose to implement a discreet personality, which can be one of the following three values: • Friendly – if the personality has a positive agreeableness value (1); • Neutral – if the personality is in the middle of the agreeableness axis (0); • Unfriendly – if the personality has a negative agreeableness value (-1). This personality will determine how the characters react, specifically how he talks in our case. For example, if a character has a friendly personality he will be much more friendly when he talks.

4.5 Conversation

The restrictions made for the conversations were the inability of creating composed messages (see section 3.4), and the inability of eavesdropping on a conversation. The composed messages were not implemented to simplify the interaction between the characters. Although the eavesdropping mechanisms were implemented, they were not used in practice. Despite the fact that we do not apply any restrictions to how a conversation system should be implemented, we opted for what we consider to be a better implementation than the way the conversation is implemented in today’s RPGs. Today’s RPGs have the conversation implemented as state machines; if we choose a given message, it will pass to a different state and we will have the message options from that state. Since we chose a BDI architecture as our base, each character has its own intentions, which could be for example to find a specific character. By using this in conjunction with the memory, specifically the theory of mind, we are able to create the messages that a character may choose from dynamically, which in our opinion make for a much interesting conversation system. However, we do acknowledge that such a system would have to be done with some precautions, since if we had many intentions we could end up with an unusable conversation interface (i.e. too many message possibilities to choose from).

4.6 Memory

The memory was able to retain the following information about the game world (this information is based on the case study’s scenario, presented in chapter 5): • Characters; • Occupations;

• Items; • Places. All these concepts were connected with each other (see Figure 4.5). For example, if a character was a merchant, besides adding the character’s information to the pool of characters, we also established a connection between the occupation merchant and that character.

Figure 4.5 - UML diagram of the Memory.

There was an exception for the representation that the other character make of the player, which we will explain in section 4.7. An example of a memory serialization file of a character can be found in appendix B.

4.7 Theory of Mind

The theory of mind is simply having each representation of a character in our memory have its own memory (see Figure 4.5), which will be the beliefs that a character has about the other characters. To make the conversations feel more natural, we added a concept called interest. An interest is something that a sender of the message wants and the receiver of the message cannot comply. For example, if the player wants to buy an item, which s/he thinks that a character has, s/he will ask to buy the item, but if that character does not have it, the character will retain that the player has an interest about that specific item. This will allow for s/he to remember that the player wanted it and that the next time they speak, if the character already has that item, s/he will supply us with that information right up front, without the player having to ask again. The notion of interest was only implemented on the player (see Figure 4.6), since there was no need for the other characters on our case study’s scenario (see chapter 5). But we think that the notion of interest is essential for all characters in the implementation of more complex scenarios. 44

Figure 4.6 - UML of the theory of mind about the player (see Figure 4.5).

4.8 Information Transmission

Although we believe that the system information transmission described in section 3.7 is best suited to withheld information from the player, for example making the player earn the trust of a character (e.g. by completing some quest), we decided to make the characters always tell something (if they know anything) when the player asks them. The personality vs. relationship plot (see Figure 3.5) will influence the quantity of information that they give. We chose to always say some information because in this scenario withholding information did not make much sense, and because the player might not perceive if the character did not tell them something because s/he did not know it, because s/he did not like us, or because s/he did not like the object of the conversation. Figure 4.7, Figure 4.8, and Figure 4.9 represent the quantity of information that is transmitted between the characters having a conversation based on their personality and the result of the sum of the relationships (explained in section 3.7).

Figure 4.7 - Information transmitted by a Neutral personality.

Figure 4.8 - Information transmitted by a Friendly personality.

Figure 4.9 - Information transmitted by an Unfriendly personality.

For example, with a neutral personality (Figure 4.7) a character would say who has the item, what s/he does, where we could find her/him, and motivate us to go and see her/him, if the result of that character’s relationship with us and with the object was “Likes”. On the other hand, if the result was “Dislikes”, the character would only say who has the item, what s/he does, and then try to demotivate us from talking with her/him. The motivate/demotivate feature was something that we chose to implement to make the relationships more explicit, because users might not understand their relationships based just on the information passed. For example, they might assume that a character does not say where I can find the object because s/he does not know.

4.9 Conclusions

Due to time constrictions we were not able to implement the entire conceptual model, although we believe that this architecture is sufficient to test the hypothesis presented in section 1.3. The two major features that were not fully implemented are related with the conversation (see section 3.4): composed messages and eavesdropping on conversations. However, this concept was not the focus of what we are trying to prove, and although its implementation was not complete, we believe it to be a more interesting approach to conversation than the one present in most of the RPGs, since it is not based on state machines but instead it is based on a character’s objectives and memory.

Chapter 5

Case Study: The Market

5.1 Introduction

This chapter describes our case study that was developed in order to evaluate the effect that social behaviour in NPCs has on the players’ gaming experience. Since we wanted to implement a feature that is not present in today’s RPGs, we chose a scenario that appears in all of them, and, although it is very socially rich environment in the real world, it is usually very poorly portrayed in games – a market. In today’s RPGs the market is a simple scenario; the player has to go there in order to obtain the items s/he needs. However, this is, usually, a “burden” to the player rather than a part of his gaming experience. It is a boring context, which contrasts with the lively ambience experienced in real world markets. We believe the player’s gaming experience can improve by implementing social behaviour, even in simple scenarios like this. There will be two implementations of the scenario. The first version will be an improved version of the market present in today’s RPGs. This means that the NPCs will not only show an inventory when you click on them but they will also have memory from their previous conversations with the player. The second version will have our implementation of social behaviour in the NPCs. This means that the NPCs will have information about the other NPCs and will respond to the player based on their own opinion about each one of them.

5.2 Story

The player takes on the role of a space traveller from planet Zion called Jack, whom while travelling to another planet has an accident and his shuttle malfunctions, causing him to crash in one of Zion’s colonies. After crashing, Jack realizes that there is a city near by, and goes there to find help to repair his shuttle.

At this city there is a market, where the player will do all of his interactions. There are three types of characters in the market: • Merchants – they trade items with the customers; • Customers – they trade items with the merchants; • Mechanic – he repairs shuttles. In order to know what items Jack needs to repair his shuttle, he will have to talk with the mechanic. The mechanic will tell Jack that he needs the following items: • Navigator; • Stabilizer; • Thruster; • Energy Cell; • Antenna. Afterwards Jack will have to go through the merchants in order to get the items. After getting all the items, he will have to hire the mechanic to repair his shuttle.

5.3 Characters

Six characters, excluding the player, populate the market. Like we explained in our conceptual model, every character has a personality. Table 5.1 shows the names of each character and their personalities.

Table 5.1 - Personality of each character.

As we can see, Jade and Patrick have a friendly personality, Jeff and Michael have a neutral personality, and John and Penelope have an unfriendly personality. Jeff, Jade and John are merchants, Patrick and Penelope are customers, and Michael is a mechanic.

Figure 5.1 - Two characters having a conversation.

5.4 First Version

As previously stated, the first version of the scenario will replicate the market presented in today’s RPGs. The merchants will be static figures, which are always there to serve the player, not engaging in conversation with any characters. Because the interaction between these characters and the player is really simple in today’s games – i.e. the player engages in conversation and selected the items he wants to buy -, we decided to add an extra to them: characters will keep a memory of previous interactions. This means that if we go to a merchant and we try to buy an item but s/he does not have it, not only will we remember this, but also next time we talk with her/him, s/he will notify us if s/he already has that item available. To make the market less static than it usually is in RPGs, a type of character, the customer, was added. The customers will enter the market, engage in conversation with the merchants, and trade items with them. After doing their transactions they will exit the market. 51

The merchants will have the following actions: • Buy/Sell items from the customers. The customers will have the following actions: • Engage in conversation with the merchants; • Buy/Sell items from the merchants; • Enter/Leave the market (i.e. enter/exit the scene through exit areas). The mechanic will have the following actions: • Determine the damage of shuttles; • Repair shuttles (after the player collected all the parts needed to repair the shuttle). The player – i.e. Jack – is a customer, although he cannot leave the scene. He will be able to do the following actions: • Engage in conversation with any character; • Buy items from the merchants.

5.5 Second Version

In the second version of the scenario we will explore social behaviour between the NPCs. This social behaviour will be established by the relationships between the characters. The relationship set is presented in Table 5.2.

Table 5.2 - Relationships between the characters.

As we can see in Table 5.2, for example, Jade likes Jeff but she dislikes John.

Like we mentioned in chapter 3, the relationships will influence what characters say about each other. Following the previous example, if we asked Jade if she knew where we could buy an item that both Jeff and John sell, she would encourage (i.e. motivate) us to buy from Jeff (see Figure 4.8). Basically, the differences between the first and the second versions are that in the second the characters have information not only about themselves and the interests of the player, but also about the other characters. If a customer talks with a merchant in the first version, no information will be transferred, but in the second version the customer will know which items that merchant has.

Figure 5.2 - Player having a conversation with an unfriendly merchant.

Chapter 6

User Evaluation

6.1 Introduction

This chapter describes the experiment that we conducted in order to test the hypothesis formulated in the beginning of this document (see section 1.3) – if social relationships could create a richer social behaviour in the NPCs that could improve the player’s gaming experience. In this experiment we used the game described in chapter 5, which provides a scenario common in today’s RPGs – a market – that is usually very poorly represented in games. The following sections will describe how the experiment was conducted.

6.2 Dialog Evaluation

Since one of the ideas that we want to transmit is the individuality of each character, we needed to take into account what the characters said, since it is how the concept of personality was transmitted. To see if the characters’ lines transmitted the three states of personality, friendly, neutral, and unfriendly (see section 4.4), we presented these lines to some users to see if they could identify them. We discovered that if we present the lines grouped – e.g. three lines together and ask which is the friendly, neutral, and unfriendly -, the users had a higher probability of identifying correctly the lines (89,3%). The reason why the percentage of correct answer is not higher is because users often consider the neutral answer to be friendly. For example, one of the users considered that by simply by saying something when we bought an item that the character was being friendly. If the lines are all mixed (i.e. not grouped by the phrase which precedes them), the users classified more neutral answers as friendly. However, in every situation, the unfriendly lines were always correctly identified. This effect is minimized since the dialogs in games are usually grouped.

6.3 Game Evaluation

The game evaluation was done with a total of twenty players, taking the whole process between 20 and 40 minutes. The process was divided into 4 parts: 1. Users were given a version of the scenario to play; 2. Users were given a questionnaire to answer some questions about the version they played; 3. Users were then given the other version of the scenario to play; 4. Finally, users answered the same questionnaire but this time about the second version they played. The version with which the players would start with was evenly distributed, with ten of them starting with the first version, which does not contain the social relationships (see section 5.4), and the other ten starting with the second version, which contains the social relationships (see section 5.5), having already played the game and answered the questionnaire when they had to play for the second time. We saved the logs of their interactions with the NPCs were to be later analysed in conjunction with the questionnaires. Before they started the experiment, the game’s objective was explained to them. There was no time limit for the game; it ended when the player reached the objective. The players were not told which of the versions had the social relationship. The questionnaire had a few questions about the personal information (e.g. gender, age, if they played RPGs), characters’ personalities (if they thought that the characters had different personalities, and what personalities were) and relationships (if they thought that the characters had relationships between them, and what kind of relationship was), and a question for they to rate the experience of each scenario. Finally, they were asked what version they preferred. The questionnaire is available in appendix C.

6.4 Results

This section presents the analysis of the data retrieved from the questionnaire and from the logs of each version of the game that the users played. Basically our main focus is to see if there was a significant difference between version 1 and version 2. The results shows that there is a significant difference regarding the existence of relationships in both versions (Z=-3.464, p=0.001), which we were expecting since it is the differentiating factor between the two version. In the first version 40% of the users answered that the characters had social relationships, although they would generally not remember which characters had them, nor what kind of relationships they were; approximately 93% when asked what kind of relationship was (i.e. likes, neutral or dislikes) answered that they did not know. In the second version 100% of the users answered that the characters had social relationships between them.

There was also a big difference between the number of relationships they identified in the merchants and the mechanic (see Table 6.1), which makes sense given the fact that the users interacted more with the merchants (a mean of 1.59 conversations7 with each) and the mechanic (a mean of 2.25 conversations) than with the customers (a mean of 0.82 conversations with each). NPC Jeff Jade John Michael Rel. with Jade John Michael Jeff John Michael Jade Michael Jeff Jade John Z -2.810 -2.157 -2.658 -2.401 -2.754 -3.402 -1.903 -2.968 -2.913 -3.066 -2.673 p 0.005 0.031 0.008 0.16 0.006 0.001 0.057 0.003 0.004 0.002 0.008

Table 6.1 - Difference between the relationships in version 1 and 2 in the merchants and the mechanic.

It is also worth mentioning that the number of conversations between the user and the merchants is a bit higher in version 2, with 1.65 conversations with each merchant versus 1.53 in version 1. The number of messages exchanged between them is really much higher in version 2 (9.67 with each character, versus 6.33 in version1), as it also was expected, since the users can ask information about the other characters. There was also a big difference between the ratings that was given to each version (Z=3.207, p=0.001), with version 2 receiving a mean rating of 3.45 and version 1 receiving 2.85. Finally, 80% of the users preferred version 2, which contains the social relationships. Of the users that play RPGs, 82% prefer the version with social relationships. One interesting fact is that 100% of the users that played version 1 first preferred version 2 (N=20, r=0.5, p<0.05). We also found some interesting results, like the users remembering better the personalities of characters that were friendly (65%) or unfriendly (77.5%). They also remembered more accurately the social relationships of characters with neutral personalities (between 70 to 90% of accuracy when they tried to answer). This is due to fact that the neutral personality is the one with a biggest variation between the different relationships (see Figure 4.7). As for correlations, we only found one interesting fact in version 1 – users who had more conversations with Jade, the friendly merchant, tended to have more conversations with John (N=20, r=0.683, p<0.01), Michael (N=20, r=0.596, p<0.01), Patrick (N=20, r=0.674, p<0.01), and Penelope (N=20, r=0.545, p<0.05).

7 One conversation is equivalent to one (greet, goodbye) pair, ignoring the number of messages exchanged in between. 57

Chapter 7

Conclusions

7.1 Introduction

Clearly with the maturing of the computer games’ market, game studios start doing more elaborate games. Like with the progression of graphical quality, it is only natural that developers start to focus on areas yet to release their full potential, like in the case of character behaviour. The purpose of this work is to test the hypothesis that if we enrich the social behaviour of the NPCs by implementing social relationships – especially in RPGs -, the players’ gaming experience can improve. By examining some of the latest and most acclaimed RPGs we found some features of NPCs could, in our opinion, be improved. The game reviewers also corroborated some of our opinions, criticizing social behaviour of the NPCs. A survey conducted by us to see if the gamers also had the same opinions determined that gamers believe that the NPCs still need to be improved, and that the enrichment of social behaviour of the NPCs would improve their gaming experience. To test this hypothesis, a conceptual model was developed taking into account the concepts of relationship, personality, conversation, memory, theory of mind and information transmission. This conceptual model was then implemented in an AI architecture, which was integrated into a game engine, and we developed a game (see chapter 5) that was evaluated by users. The users played both versions of the game, one with social relationships and the other one without, and in the end 80% of all players (and 82% of RPG players) preferred the version with social behaviour. However, it is worthy to mention that the players mentioned that they were more aware in the second time they played, and all off them who played the version with social relationships in the second time agreed that they prefer the game with social relationships.

7.2 Future Work

Although we believe that this could be the basis for more believable social behaviour, we also believe that much is still to be done. One of the main features that we believe to be missing is dynamic social behaviour. This is the goal, which we believe to be the next great breakthrough in game AI – characters adapting to their environment. We are currently thinking of adapting the model currently implemented to integrate Heider’s Balance Theory [20] and Festinger’s Theory of Cognitive Dissonance [12]. These theories are based on the principle that the stability of the relationships is important, and when unbalanced, the people will re-evaluate their relationships in order to regain balance. The relationships would be affected based on events that can be performed by either the player or the NPCs, and the impact that these events have would be calculated based on the personality and the current relationships of the NPCs. The addition of such a system would be especially suitable for trust-earning situations, like in the case of the RPGs’ quests, where the user has to do some task to get some item/information. Additionally, it would be great to develop a complete solution for NPC prototyping in academic environment. Today, there is no de-facto framework for AI in the academic community, which implies developing everything from ground up every time we need to develop a project. We believe that a gaming platform for developing agents would suite the academic community, and encourage newcomers, especially in AI courses. Besides developing this platform, we need to develop tools to ease the creation of these environments: populate memory, create relationships, add new personalities, add new performatives, etc.

Bibliography

[1] E. André, M. Klesen, P. Gebhard, S. Allen and T. Rist. Integrating models of personality and emotions into lifelike characters. In Proceedings International Workshop on Affect in Interactions. Towards a New Generation of Interfaces, 1999. [2] Robert Axelrod. The Evolution of Cooperation, Basic Books, Inc., Publishers, New York, NY, USA, 1984. [3] Joseph Bates. The Role of Emotions in Believable Agents. Communications of the ACM, 37(7):122-125, 1994. [4] BioWare Corp. Neverwinter Nights Website http://nwn.bioware.com/about/description.html, online on February 9th 2007. [5] BioWare Corp Forums. http://nwn.bioware.com/forums/, online on February 9th 2007. [6] Blizzard. World of Warcraft – http://www.wow-europe.com/, online on November 30th 2006. [7] Blizzard. World of Warcraft, Europe Forums – http://forums.wow-europe.com/, online on February 9th 2007. [8] Michael E. Bratman et al. Plans and resource-bounded practical reasoning. Computational Intelligence, 1988. [9] Nicolas Ducheneaut and Robert J. Moore. The Social Side of Gaming: A Study of Interaction Patterns in a Massively Multiplayer Online Game, ACM Press, New York, NY, USA, 2004. [10] Nicolas Ducheneaut, Nicholas Yee, Eric Nickell and Robert J. Moore. “Alone Thogether?” Exploring the Social Dynamics of Massively Multiplayer Online Games. In Proceedings of the 2006 ACM SIGCHI Conference on Human Factors in Computing Systems, pages 407-416, Montréal, Québec, Canada, April, 2006. [11] H. J. Eysenck and S. Rachman. The Causes and Cures of Neuroses, London: Routledge and Kegan Paul, 1965. [12] L. Festinger. A Theory of Cognitive Dissonance. Stanford University Press, Stranford, CA, USA, 1957. [13] John Funge. Making Them Behave: Cognitive Models for Computer Animation. Ph.D. Thesis, Department of Computer Science, University of Toronto, Canada, 1998.

[14] GameDev-PT Focums. http://www.gamedev-pt.net/forum.php, online on February 9th 2007. [15] GameSpot. http://www.gamespot.com/, online on October 31st 2006. [16] GameSpot Forums. http://www.gamespot.com/forums/index.html, online on February 9th 2007. [17] GameSpy Forums. http://www.forumplanet.com/gamespy/, online on February 9th 2007. [18] GameTrailers. http://www.gametrailers.com/, on October 31st 2006. [19] Gas Powered Games. Dungeon Siege II – http://www.gaspowered.com/ds2/, online on November 27th 2006. [20] Fritz Heider. The Phsycology of Interpersonal Relations. Wiley, New York, 1958. [21] IGN - http://www.ign.com/, on October 31st 2006. [22] P. J. Lang. The Emotion Probe: Studies of motivation and attention, A study in the Neuroscience of Love and Hate, Hillsdale, NJ: Lawrence Erlbaum Associates, Publishers, 1995. [23] M. Lebowitz. StoryTelling as Planning and Learning. In Poetics. Vol. 14 No. 6, December 1985. [24] Lionhead Studios. Fable – http://www.lionhead.com/fable/, online on December 18th 2006. [25] Lionhead Studios Forums. http://allboards.lionhead.com/, online on February 9th 2007. [26] LucasArts. Knights of the Old Republic II – http://www.lucasarts.com/games/swkotor_sithlords/, online on November 15th 2006. [27] Brian Mac Namee and Pádraig Cunningham. A Proposal for an Agent Architecture for Proactive Persistent Non Player Characters. In Proceedings of the 12th Irish Conference on AI and Cognitive Science, 2001. [28] Brian Mac Namee and Pádraig Cunningham. The Driven Simulation of Socially Interactive Agents, 2002. [29] Abraham H. Maslow. Motivation and Personality, New York, NY, USA: Harper & Row Publishers, Inc., 1954. [30] R. R. McCrae and O. P. John. An introduction to the five-factor model and its applications. Journal of Personality: 60, pages 175–215, 1992. [31] J. Meehan, The Metanovel: Writing Stories by Computer, Research Report #74. Computer Science Department, Yale University. New Haven, CT, 1996. [32] Mega Score Forums. http://megascore.rgb.pt/forum/, online on February 9th 2007. [33] B. Moulin and D. Rousseau. An approach for modelling and simulating conversations. In D. Vanderveken and S. Kubo, editors, Essays in Speech Act Theory. John Benjamins, 2000.

[34] Obsidian Entertainment Forums. http://forums.obsidianent.com/index.php, online on February 9th 2007. [35] A. Ortony, G. L. Clore, A. Collins. The Cognitive Structure of Emotions. Cambridge University Press, 1988. [36] Planet Dungeon Siege Forums. http://www.forumplanet.com/planetdungeonsiege/, online on February 9th 2007. [37] Helmut Prendinger, Mitsuru Ishizuka. Social Role Awareness in Animated Agents. In Proceedings of the fifth international conference on Autonomous agents, pages 270–277, Montréal, Quebec, Canada, 2001. [38] David V. Pynadath, Stacy C. Marsella, Stephen J. Read. PsychSim: Agent-based modeling of social interactions and influence. In Proceedings of the International Conference on Cognitive Modeling, pages 243–248, 2004. [39] David V. Pynadath, Stacy C. Marsella. PsychSim: Modeling Theory of Mind with Decision-Theoretic Agents. In Proceedings of the International Joint Conference on Artificial Intelligence, pages 1181–1186, 2005. [40] Anand S. Rao and Michael P. Georgeff. Modeling rational agents within a {BDI}- architecture. In James Allen, Richard Fikes, and Erik Sandewall, editors, Proceedings of the 2nd International Conference on Principles of Knowledge Representation and Reasoning, pages 473–484. Morgan Kaufmann publishers Inc.: San Mateo, CA, USA, 1991. [41] Anand S. Rao and Michael P. Georgeff. Bdi-agents: from theory to practice. In First International Conference on MultiAgent Systems (ICMAS-95), pages 312– 319, San Francisco, California - USA, 1995. [42] W. Scott Neal Reilly. Believable Social and Emotional Agents, Ph.D. Thesis. Technical Report CMU-CS-96–138, School of Computer Science, Carnegie Mellon University, Pittsburgh, PA, 1996. [43] Abdennour El Rhalibi, Nick Baker and Madjid Merabti. Emotional Agent Model and Architecture for NPCs Group Control and Interaction to Facilitate Leadership Roles in Computer Entertainment. In Proceedings of the 2005 ACM SIGCHI International Conference on Advances in computer entertainment technology, pages 156–163, Valencia, Spain, 2005. [44] Thomas Rist, Markus Schmitt. Avatar Arena: an attempt to apply socio- physiological concepts of cognitive consistency in avatar-avatar negociation scenarios. In Proceeding of AISB’02 Symposium on Animated Expressive Characters for Social Interactions, pages 79–84, London, England, 2002. [45] Fiorella Rosis's Homepage. http://www.di.uniba.it/intint/people/fior.html, online on August 19th 2007. [46] The Elder Scrolls IV: Oblivion. http://www.elderscrolls.com/games/oblivion_overview.htm, online on December 24th 2006.

[47] The Elder Scrolls Forums. http://www.elderscrolls.com/forums/index.php, online on February 9th 2007. [48] Unity. http://www.unity3d.com/, online on November 19th 2007. [49] Wikipedia. Cultural Differences in Computer and Console RPGs - http://en.wikipedia.org/wiki/Cultural_differences_in_computer_and_console_role- playing_games, online on October 31st 2006. [50] Wikipedia, Dungeons & Dragons - http://en.wikipedia.org/wiki/Dungeons_%2526_Dragons, online on October 31st 2006. [51] Wikipedia. Role-Playing Game - http://en.wikipedia.org/wiki/Role-playing_game, online on October 31st 2006. [52] Wikipedia. Abraham Maslow's Hierarchy of Needs Pyramid - http://en.wikipedia.org/wiki/Maslow's_hierarchy_of_needs, online on January 15th 2007. [53] M. Wish, M. Deutsch and S. Kaplan. Perceived Dimensions of Interpersonal Relations. In Journal of Personality and Social Psychology Vol. 33 No. 6, April 1976. [54] Wizards of the Coast. Dungeons & Dragons Official Website – http://www.wizards.com/dnd/, online on October 31st 2006.

Appendix A

RPG Gamer Survey

This appendix presents the RPG gamer survey that was distributed in the Internet in order to get some feedback from the players of RPGs. The results of this survey can be found in section 2.5.

Figure A.1 – RPG Gamer Survey (1/2)

Figure A.2 - RPG Gamer Survey (2/2).

Appendix B

Memory Example

The following XML text represents the memory of the merchant Jeff serialized to a file. 3 John (Yellow) Male DISLIKES John's Stand Merchant Energy Cell Navigator Thruster Energy Cell 1 Navigator 2 69

4 Jade (Pink) Female LIKES Jade's Stand Merchant Navigator Stabilizer Thruster Stabilizer 2 Navigator 2 5 Michael (Red) Male DISLIKES Garage Mechanic

John's Stand 3 Jade's Stand 4 Garage 5 Jeff's Stand Merchant 3 4 Mechanic 5

Antenna Energy Cell 3 Stabilizer 4 Navigator 3 4 Thruster 3

Appendix C

Case Study Questionnaire

This appendix presents the questions used to evaluate the case study presented in chapter 5. It is divided into three parts: the first part consists of personal information questions, the second part consists of questions about the first version tested of the scenario, and, finally, the third part consists of questions about the second version tested of the scenario, with a last question regarding which of the versions the player preferred. The results of this questionnaire can be seen in section 6.4.

C.1 Personal Information Questions

Figure C.1 - Case Study Questionnaire (1/3)

C.2 Questions for the first version evaluated

Figure C.2 - Case Study Questionnaire (2/3).

C.3 Questions for the second version evaluated

Figure C.3 - Case Study Questionnaire (3/3)