Empirical Investigation on Measurement of Game Immersion Using Real World Dissociation Factor

Thesis no: MSCS-2016-13

Empirical Investigation on Measurement of Game Immersion using Real World Dissociation Factor

Gadila Swarajya Haritha Reddy

Faculty of Computing Blekinge Institute of Technology SE–371 79 Karlskrona, Sweden This thesis is submitted to the Faculty of Computing at Blekinge Institute of Technology in partial fulﬁllment of the requirements for the degree of Master of Science in Computer Science. The thesis is equivalent to 20 weeks of full time studies.

Contact Information: Author(s): Gadila Swarajya Haritha Reddy E-mail: [email protected]

University advisor: Prof. Sara Eriksén Department of Creative Technologies

Faculty of Computing Internet : www.bth.se Blekinge Institute of Technology Phone : +46 455 38 50 00 SE–371 79 Karlskrona, Sweden Fax : +46 455 38 50 57 Abstract

Context. Games involve people to a large extent where they relate themselves with the game characters; this is commonly known as game immersion. Generally, some players play games for enjoyment, some for stress relaxation and so on.Game immersion is usually used to describe the degree of involvement with a game. When people play games, they don’t necessarily realize that they have been dissociated with the surrounding world. Real world dissociation (RWD) can be defined as the situation where a player is less aware of the surroundings outside the game than about what is happening in the game itself. The RWD factor has been expected to measure the losing track of time, lack of awareness of surroundings and mental transportation. Objectives. In this thesis, we measure and compare the difference in game immersion between experienced and inexperienced players using RWD factor. In addition, the study involves exploring the significance of game immersion and various approaches used to measure it. Methods. In this study literature review has been carried out to explore the meaning of game immersion and further user studies in the form of an experiment has been conducted to measure game immersion between experienced and inexperienced gamers. The game immersion has been measured using the real world dissociation (RWD) factor. After the experiment has been conducted, a statistical technique has been carried out to measure the difference in game immersion among the two groups. Results. The empirical investigation on the measurement of game immersion has been done using RWD factor. The results state that the significance value is less than 0.05 and hence null hypothesis is rejected for both the games. The measurable difference has been calculated by using Cohen’s d effect size between experienced and inexperienced players. Conclusions. After analyzing the data and calculating the effect size, the overall results state that inexperienced group of players are more immersed than the experienced group of players when measured by RWD factor. Hence it can be concluded that irrespective of the game played, inexperienced players are more dissociated from the real world than the experienced players.

Keywords: Game Immersion, Real world dissociation (RWD), experienced players, inexperienced players. Dota 2, CS:GO.

i Acknowledgement

Immeasurable appreciation and deepest gratitude for the help and support to the people who in one way or the another have contributed in making this study (Master’s Degree) possible.

It is a genuine pleasure to express my deep sense of gratitude and sincerity to my supervisor Prof. Sara Eriksén. I’m grateful to my supervisor, whose expertise, understanding, generous guidance and support made it possible for me to work on a topic that was of a great interest to me. I would also like to forward my thanks to the examiner of Computer Science Department Dr. Martin Boldt, who have supported with valuable guidance and encouragement throughout my study.

Finally, I would like to show my gratitude from the bottom of my heart to my parents Gadila Veerender Reddy and Gadila Hima Bindu, my brother Sa- harsh Reddy for their eternal support and love throughout my study. I would like to forward a special thanks to all my family members and friends who have supported me at every step of my study which has put me forward with conﬁdence and success.

ii List of Figures

2.1 SCI Model ...... 10 2.2 Relationship between factors and models of immersion ...... 11 2.3 Factors aﬀecting RWD ...... 14

3.1 Minimap of the arena in Dota 2 during a battle between two teams 21 3.2 Map representing teams: terrorists and counter-terrorists in CS:GO 22 3.3 Experimental Process ...... 31

4.1 Immersion scores for experienced players of Dota2 ...... 41 4.2 Immersion scores for inexperienced players of Dota2 ...... 42 4.3 Comparison of immersion scores for experienced and inexperienced players (Dota 2) ...... 42 4.4 Immersion scores for experienced players of Counter Strike . . . . 43 4.5 Immersion scores for inexperienced players of Counter Strike . . . 43 4.6 Comparison of immersion scores for experienced and inexperienced players (CS:GO) ...... 44

5.1 Means and Standard deviations for Dota 2 ...... 46 5.2 Values for Independent Sample T-test (Dota 2) ...... 46 5.3 Diﬀerence in means of immersion scores for both the groups (Dota 2)...... 47 5.4 Means and Standard deviations for CS:GO ...... 47 5.5 Values for Independent Sample T-test (CS:GO) ...... 47 5.6 Diﬀerence in means of immersion scores for both the groups (CS:GO) 48 5.7 Interpretation of Cohen’s d ...... 48

B.1 Snapshot of Steam Website for top games ...... 66

iii List of Tables

2.1 Description of ﬁve factors of immersion ...... 9 2.2 Characteristics of RWD with sub-categories ...... 13

3.1 Search results for literature review ...... 19 3.2 Sample calculated for each game individually ...... 25

4.1 Presence Questionnaires used in investigating gaming experience . 34 4.2 Questionnaires used in investigating gaming experience in concept of ﬂow ...... 36 4.3 Questionnaires used in investigating gaming experience that measures speciﬁc aspect of games ...... 38 4.4 Questionnaires used in investigating gaming experience which aim to capture full gaming experience ...... 40

C.1 Immersion scores of experienced players of Dota2 game ...... 69 C.2 Immersion scores of inexperienced players of Dota2 game . . . . . 71

D.1 Immersion scores of experienced players of CS:GO ...... 73 D.2 Immersion scores of inexperienced players of CS:GO ...... 75

iv List of Abbreviations

RWD Real World Dissociation MOBA Multi-player Online Battle Arena FPS First Person Shooter CS:GO Counter Strike:Global Oﬀensive HCI Human Computer Interaction VR Virtual Reality IEQ Immersive Experience Questionnaire GexpQ Game Experience Questionnaire GengQ Game Engagement Questionnaire ITC-SOPI ITC Sense of Presence Inventory TAM Technology Acceptance Model CEGE Core Elements of Gaming Experience SPGQ Social Presence in Gaming Questionnaire CAAS Computer Apathy and Anxiety Scale

v Contents

Abstract i

1 Introduction 1 1.1 Problem Statement ...... 2 1.2 Aim and Objectives ...... 3 1.3 Research Questions ...... 3 1.4 Thesis Outline ...... 4

2 Background and Related Work 5 2.1 Human-Computer Interaction (HCI) ...... 5 2.2 Computer Games ...... 5 2.3 An Overview of Immersion ...... 6 2.3.1 Presence ...... 7 2.3.2 Flow ...... 7 2.4 Definition of Game Immersion ...... 7 2.5 Models of Game Immersion ...... 9 2.5.1 SCI Model of Game Immersion ...... 9 2.5.2 Modified SCI Model of Game Immersion ...... 10 2.5.3 Adams’ Model of Game Immersion ...... 10 2.5.4 Levels of Game Immersion ...... 11 2.6 Real World Dissociation Factor ...... 12 2.7 Research Gap Identification ...... 14 2.8 Contribution ...... 14

3 Method 16 3.1 Literature Review ...... 17 3.2 Experiment ...... 20 3.2.1 Selection of Games ...... 20 3.2.1.1 Description of Dota 2 ...... 21 3.2.1.2 Description of Counter Strike: Global Oﬀensive . 22 3.2.2 Participants and Sampling ...... 23 3.2.2.1 Stratiﬁed Random Sampling ...... 23 3.2.3 Questionnaire Development ...... 25 3.2.3.1 Motivation for Choosing IEQ Questionnaire . . . 25

vi 3.2.4 Procedure ...... 27 3.2.5 Data Analysis ...... 28 3.2.5.1 Description of Independent Sample T-test . . . . 28 3.2.5.1.1 Hypothesis Testing ...... 29 3.2.5.2 Cohen’s D Eﬀect Size ...... 29

4 Results 32 4.1 Literature Review ...... 32 4.1.1 Methods for Measuring Game Immersion ...... 32 4.1.1.1 Objective Measures for Game Immersion . . . . . 32 4.1.1.2 Subjective Measures for Game Immersion . . . . 33 4.2 Experiment ...... 41 4.2.1 Measures of Game Immersion ...... 41 4.2.1.1 Dota 2 ...... 41 4.2.1.2 Counter Strike:Global Oﬀensive ...... 42

5 Analysis 45 5.1 Data Analysis for Results ...... 45 5.1.1 Difference in Game Immersion ...... 45 5.1.1.1 Independent Sample T-test ...... 45 5.1.1.1.1 Hypothesis Testing for Dota 2 ...... 46 5.1.1.1.2 Hypothesis Testing for Counter Strike:Global Offensive ...... 47 5.1.1.2 Effect Size ...... 48 5.1.1.2.1 Cohen’s D for Dota 2 ...... 49 5.1.1.2.2 Cohen’s D for Counter Strike:Global Of- fensive ...... 49

6 Discussions and Validity Threats 50 6.1 Discussions ...... 50 6.2 Threats to Validity ...... 51 6.2.1 Internal Validity ...... 52 6.2.2 External Validity ...... 52 6.2.3 Construct Validity ...... 52 6.2.4 Conclusion Validity ...... 52

7 Conclusions and Future Work 54 7.1 Conclusions ...... 54 7.1.1 Answering Research Questions ...... 54 7.2 Future Work ...... 55

References 56

Appendix A Game Immersion Questionnaire 64

vii Appendix B Snapshot of Steam Website 66

Appendix C Immersion scores of Dota 2 67 C.1 Results of experienced players ...... 67 C.2 Results of inexperienced players ...... 69

Appendix D Immersion Scores of CS:GO 72 D.1 Results of experienced players ...... 72 D.2 Results of inexperienced players ...... 74

viii Chapter 1 Introduction

In recent times, video games have become so engaging to millions of people all around the world and have become an integral part of their lives [1]. Some players play games for enjoyment, some for stress relaxation and so on. People enjoy themselves when engaging in activities that challenge them, where games are one such activity. Video games were once considered to be a negative activity, but recognition of the educational potential of playing video games is now increasing [1]. Video games have a wide range of positive impact as it has become a growing body of research [2]. Successful computer games share the main characteristic that is able to attract and draw people into the game. Most of the games today provide players with more opportunities for social interactions, which can produce feelings of relatedness and belonging [1]. Due to their high accessibility, a variety of games can be played in the same space on the same computer. The fun and cathartic elements of games highly motivate players, and they become so absorbed in the games that their perceptions may become distorted to a certain degree [1].

In the game world, the players can see, hear and manipulate the environment as they do in the real world [3]. Sometimes people find computer games so engaging that they lose themselves in the world of game [4]. When people play games, they necessarily don’t realize that they have been dissociated from the surrounding world [5]. Hence, while playing games, the players not only get involved with the ready-made game world but also actively participate in the field of the game. This experience is referred to as "immersion", a term used by gamers and re- viewers alike [4]. Immersion, the extent to which a person feels immersed in a task or virtual world, is an important goal in computer gaming [6]. According to Brown and Cairns, game immersion is defined as the degree of involvement with the game [7]. Immersion can be experienced in a variety of different contexts, and depending upon the different aspects of experience are emphasized as being important. The concept of immersion has been considered in many contexts but it is most commonly used, for software, when related to virtual reality and games [7].

The experience of immersion is often critical to the game enjoyment and is made or

1 Chapter 1. Introduction 2 destroyed by game characteristics [7]. Game immersion can be measured subjectively or objectively. The subjective measurements are collected and studied with the help of questionnaires and the objective measurements are studied through task completion times and measurements of eye movements [4]. The five factors of game immersion measured by questionnaire are stated as cognitive involvement, real world dissociation (RWD), challenge, emotional involvement and control [5]. Among these identified five factors, this research particularly deals with the RWD factor as it has been found that if a player is present in alternative game world, there may be a measurable effect on their return to the real world [4]. RWD is defined as the situation where a player is less aware about the surroundings outside the game than what is happening in the game itself [8].

1.1 Problem Statement There is a growing interest to understand and measure the physical and social features that constitute gaming experience. Today’s generation spend more time in playing games than in other leisure activities [1]. This is because the highly immersed qualities of gaming have put them beyond the traditional entertainment media such as televisions (TVs) and movies [9]. The separation between the virtual world and the real world can no longer exist, as games involve imaginary worlds that evoke mental images of physical or social situations that does not exist leading to immersion [1]. Hence, virtual reality research has an interesting perspective on immersive experience [7]. With a growing maturity of game science as a research field, more and more studies are devoted to an empirical investigation of different experiences in gaming [10]. Engaging with media such as computer games, TVs, movies and even books have been described as giving rise to the experience of feeling deeply involved with that particular medium. Practical evidence supporting how games are engaging and effective is really needed.This is because there is a lack of research attempting to delve into the question about how players feel and what they experience through playing games [1]. When trying to understand the meaning of immersion, it is very difficult to figure out what exactly is meant by immersion. The lack of clear understanding of immersion becomes acute when moving away from virtual reality and gaming and when considering immersion in other types of software [7]. In software research, many terms have been developed as flow, presence and immersion to describe the experiences of the players in gaming. Within game reviews, the term usually used to describe such an experience is immersion, but it is not clear if this correlates to immersion in other contexts or the extent to which it has been related to other involving experiences [11]. This has led to growing need for measuring game immersion in order to evaluate up to what extent a person can get immersed while getting dissociated from the real world. Chapter 1. Introduction 3

When people play games, due to immersion they will be less aware of the real world [5]. Playing games can have beneﬁts as well as sometimes leads to problems, acting as a distraction from real life activities [12]. It has been recognized that computer game immersion can lead to negative consequences as game addiction, social conﬂicts and guilty feelings about time wastage [12]. Hence, this also led to the need for measuring immersion in computer games when a person is dissociated from the real world. Real world dissociation (RWD) factor mirrors a phenomenon which allows people to explore virtual worlds, which can sometimes be beyond one’s imagination [8]. Therefore, this research is focused on measuring game immersion subjectively, by the player, when dissociated from the real world.

1.2 Aim and Objectives

Aim: The aim of this research is to identify the most commonly used approaches to measure game immersion. To explore more about game immersion, it is compared using real world dissociation factor between experienced and inexperienced players.

Objectives: • Explore the signiﬁcance of game immersion by knowing commonly used approaches to measure it.

• Identify various methods used to measure game immersion that relate to or based on RWD factor.

• Measure game immersion between experienced and inexperienced players using questionnaires based on RWD.

• Evaluation of results based on the experiment conducted.

• Find the measurable diﬀerence in game immersion between experienced and inexperienced players and compare them.

1.3 Research Questions

RQ.1. What are the most commonly used approaches for measuring game immersion? Motivation: Since there are only few studies on measuring game immersion, we ﬁrst need to identify the most commonly used approaches to measure game immersion. The main objective of the ﬁrst research question is to identify the current literature on game immersion. Chapter 1. Introduction 4

RQ.2. What is the diﬀerence in game immersion, measured by RWD, between experienced and novice players? Motivation: The objective of the research question is to explore relation between the player’s subjectively perceived game immersion, as measured by the RWD factor, and the player’s level of experience (in this research categorized as experienced and inexperienced gamer). This is the way to further explore and test the possible challenges when it comes to the measuring of game immersion, using RWD factor.

1.4 Thesis Outline The thesis is structured as follows:

• Chapter 2 covers the background and related work on game immersion. It highlights the methods used to measure game immersion which is deﬁned by various authors.

• Chapter 3 describes the methods used in this research. Literature review and experiment are methods which are well explained how they have been conducted in this research.

• Chapter 4 includes the results obtained during the experiment conducted to measure game immersion.

• Chapter 5 demonstrates the ﬁndings of the research. The results obtained in the experiment are analyzed in this chapter.

• Chapter 6 explains the discussions and the threats to validity that are viewed in this research.

• Chapter 7 summarizes the ﬁndings of the experiment and concludes by considering how this thesis has contributed to the research ﬁeld of computer science and gaming. Chapter 2 Background and Related Work

2.1 Human-Computer Interaction (HCI) Human-computer interaction (HCI) is a cross-disciplinary area (e.g., engineering, psychology, ergonomics, design) that deals with the theory, design, implementation and evaluation of the ways that human use and interact with computer devices [13]. Digital games are psychologically relevant in understanding the evolve- ment of a user-experience in human-computer interaction [9]. Human-computer interaction (HCI) has a key role to play in researching video games, which are an extremely inﬂuential form of computer software.

In recent years, game design has gained a good performance in HCI, but the methods and information exchange between computer and players are still quite limited [14]. Games are always drawing clear distinctions with various referent art forms. The virtual characters and environment created in the computer have become interactive objects for players, in which people are active participants and they make an interactive cycle with the computer, hence HCI is an essential distinction between players and computer games. As participants, players and computer play a motive role in cycle, HCI is not only an important feature of game but also the core of many game elements [14].

2.2 Computer Games There has been a relative increase in games research that focus on definition and ontology of games [15]. Computer games often provide people with a second reality to immerse themselves in, proving another popular form of escapism. Computer games are different from ordinary games because it is now the computer that upholds the rules [8]. "A game is a rule-based system with a variable and quantifiable outcome, where different outcomes are assigned different values, the player exerts effort in order to influence the outcome, the player feels emotion- ally attached to the outcome, and the consequences of the outcome are negotiable" [16]. Games might differ in their properties and visuals, but all the games have one important element in common, they have ability to draw people in [4].

5 Chapter 2. Background and Related Work 6

The design of current video games requires a big firm to gather graphic experts, game designers and story tellers involved in a process of pre/post-productions. Video games from the designer’s point of view, are formed by a three-tier structure: I/O, program and game [16]. The video-game is perceived by two elements: game-play and environment. Game-play defines what the game is about, its rules and scenario. Environment is the way the game is presented to the player, the physical implementation into sounds and graphics [16]. Ermi and Mayra defined the act of playing a game as: firstly rules embedded into game’s structure start operating. Then, the program code starts having effect on cultural and social, as well as artistic and commercial reality [15]. But, the three roles identified for a computer program in a game are [8]:

• Co-ordinating the game process, e.g. evaluating the rules and upholding the game state.

• Depicting the situation, e.g. illustrating a proto-view for the human player and the synthetic view for the synthetic player, including the sensory feedback.

• Participating as a synthetic player, e.g. an opponent, a non-player character, or godly powers that intervene in game events.

Digital games act as a distinctive medium as they allow players to link their perceptions, cognitions and emotions with the ﬁrst-person actions [15]. Grow- ing trends of digitally distributed games- games with no physical components- raise speciﬁc challenges for digital libraries [17]. Games also change over time as popular games for newer systems according to the recommendations and requirements of the players. Among other issues, relationships between players and video games are complex and are not well explicitly represented [17]. Playing games is supposed to produce a positive experience that is usually associated with the term immersion [16].

2.3 An Overview of Immersion Immersion is a complex phenomenon which involves the development and pro- duction of games denoting the degree of involvement and the experience that the player has with the game [18]. It is a term which is difficult to define but is ordinarily called as a subjective measurement [19]. Flow, presence, cognitive absorption are the concepts related to immersion that explain the degree to which the player becomes involved with the game [4][18]. Detailed description about the concepts related to immersion is below and various definitions of game immersion (section 2.4) are given in the following sections. Chapter 2. Background and Related Work 7

2.3.1 Presence "Presence" is the word that aims to assess the immersitivity of a system and hence is used to describe the experiences within virtual reality (VR) [18]. The concept of presence refers to the psychological experience of non-mediation, i.e. sense of being in a world generated by computer instead of just using a computer [15]. Lee defined presence as "a psychological state in which virtual (para-authentic or ar- tificial) objects are experienced as actual objects in either sensory or non-sensory ways" [20]. Ryan et al. defined that the increase in players’ enjoyment and fas- cination for future play in games is strongly influenced by presence and players’ self-determined needs for competence, autonomy and relatedness [21]. Here, competence refers to the need to participate in the activities which allow players to feel capable and successful, autonomy refers to the need to feel the experience of freedom in the chosen activities and relatedness refers to the need to feel the sense of interaction with the other players [21]. Presence builds relationship between the player and the game as a base by discovering the player in the game-world [9]. Tamborini et al. predicted that VR environments increase the levels of presence by developing the features of the gaming interfaces to the technologies [20]. Presence can be measured by investigating the relationship between perception and action of the player, later assessing the degree of involvement between the VR and real world [4]. The difference between presence and immersion is that presence is viewed as a state of mind, while immersion as is an experience in time [4].

2.3.2 Flow Flow is described as the state perceived by the people when they are highly engaged in an activity and are fulfilled with enjoyment [1]. Csikszentmihalyi proposed the idea of flow to describe the positive experience in which individuals perceive the coherence of skills and challenges [1]. There are eight components of flow characterized by Csikszentmihalyi: clear goals, high degree of concentration, loss of feeling of self-consciousness, distorted sense of time, direct and immediate feedback, balance between ability level and challenge, sense of personal control and intrinsic reward [18]. Brown and Cairns have explained that flow is characterized as an optimal and extreme state whereas immersion is defined as a graded experience [7]. Flow overlaps with immersion in terms of time distortion and challenging task for a player, and hence immersion is called as a precondition for flow [4][10]

2.4 Deﬁnition of Game Immersion Immersion is the term that is commonly used to describe the user experience in context of entertainment and exploration [6]. When present in a game, a player Chapter 2. Background and Related Work 8 suppresses the outside world and constructs a mental model of the game world [9]. Several studies were conducted on the use of game engines for perception of virtual environments and the studies reported that, user engagements are said to be positive when using wider ﬁelds of view.[19]. Sometimes a person feels immersed because they are simply exploring the virtual environment [8]. Players feel themselves in the game while playing, immersive experiences can make one lose sense of time [6].

Immersion is a metaphorical term that is derived from the physical experience of being submerged in water [8]. It is a complex notion, often associated with enjoyment and better outcomes in the game. It is important because maintaining players’ interest so that they return to the game over and over again is a major goal in game development [6]. Immersion is often viewed as critical to game enjoyment, immersion being the outcome of good gaming experience [4]. It is a powerful experience of gaming, and has been mentioned by gamers, designers and game researchers alike as an important aspect of interaction [7]. Immersion is conceptualized as psychological feeling of involvement, or the sense that the user is actually in the game [22]. After few researches, ﬁndings indicate that immersion has the following features [4]:

• Lack of awareness of time.

• Loss of awareness of the real world.

• Involvement and a sense of being in the task environment.

There are competitive definitions for the term "immersion" [11]. Coomans and Timmermans describe immersion as "a feeling of being deeply engaged where people enter a make-believe world as if it is real". While Menetta and Blade describe it with respect to the emotional response presented by the virtual environment [11]. Radford has related and defined immersion as the ability to enter the game through its controls, although it is unclear how this relationship affects or gives rise to immersion [7]. Slater et al. defined immersion from a different perspective with respect to the technology rather than the human experience [11]. McMa- han has defined immersion by relating the degrees of immersion to the degree of interactivity between player and game-world while including realism as one of the defining structures of immersion [23]. According to Brown and Cairns, game immersion is usually used to describe the degree of involvement with the game [7]. According to this basis of game immersion, five components of immersion have been identified which allows one to break down the general graded experience into its different components [8]. As defined by C.Jennett, the five components of immersion have been described in the Table 2.1 [8]. Chapter 2. Background and Related Work 9

Immersion Factor Description

Cognitive Involvement (CI) Strong loadings with items expected to measure eﬀort and attention.

Emotional Involvement (EI) Strong loadings for items expected to measure suspense and aﬀect.

Real World Dissociation Strong loadings for items expected to measure (RWD) lack of awareness of surroundings and mental absorption.

Challenge (Ch) Strong loadings for items expected to measure how diﬃcult the player ﬁnds the game.

Control (Con) Strong loadings for items expected to measure the ease of use of the interface of the game.

Table 2.1: Description of ﬁve factors of immersion

2.5 Models of Game Immersion

2.5.1 SCI Model of Game Immersion Ermi and Mayra identified three types of immersion by proposing SCI model: sensory, challenge-based and imaginative [8][24]. Sensory immersion relates to immersion in context of presence, which explains that games act as virtual environments which offer high quality, realistic, audiovisual presentations. Challenge- based immersion is one offered by the game due to the challenges to be faced and the skills needed to overcome those by the player [24]. Immersion is usually expected to arise from the level of challenge experienced by gamers. Challenge is the component that goes into the make-up of immersion and can be isolated as its own factor of immersion when perceived by the player [25]. Imaginative immersion is the type of immersion where the player becomes more absorbed with the stories in the game world and there by feels more emotional with a great involvement towards the characters of the game.[8]. Chapter 2. Background and Related Work 10

Figure 2.1: SCI Model

2.5.2 Modified SCI Model of Game Immersion Arsenault explained immersion by proposing a modified SCI model to better re- flect how immersion arises in games [26]. He proposed two amendments for the SCI model proposed by Erma and Mayra. The first amendment was to remove the notion of challenge and replace it with systemic immersion. Systemic immersion occurs when the player replaces the rules of the real world with those of the game world [24]. The second amendment was to replace imaginative immersion with fictional immersion. Fictional immersion is defined as when the game en- deavours the player to feel that there is more to the fictional world than what the text makes, i.e. it reflects how immersed a player is when they use their imagination[8][24].

2.5.3 Adams’ Model of Game Immersion Another model of game immersion has been described by Adams with a complete different analysis but again with three types: tactical, strategic and narrative [24]. Tactical immersion is defined as the moment-by-moment act of playing the game, which is usually found in fast-action or challenging games. It is physical and immediate state produced by the challenges for the player which are to be solved in a fraction of second [8]. Strategic immersion is related to gameplay where players think carefully over the game, where the game offers enjoyable mental absorption. Finally, narrative immersion is defined as when player begins to care about the game characters and will be interested to know the conclusion of the game, including the emotional involvement of the player [24][8]. Chapter 2. Background and Related Work 11

Figure 2.2: Relationship between factors and models of immersion

2.5.4 Levels of Game Immersion After conducting a grounded investigation on game immersion, Brown and Cairns found three levels of involvement: engagement, engrossment and total immersion [7].

• Engagement is the lowest level of involvement and occurs before any other level. It entails contribution from the player in time, eﬀort and concentration in learning how to play the game and further it makes the player want to keep playing [26][24][8].

• Engrossment is the next level of engagement where the emotions of the player are directly aﬀected by the game [8]. Once the player reaches this state, the game becomes important part and further gamer becomes less aware than before [24][8].

• Total Immersion is another word for presence according to Brown and Cairns. It requires highlest level of attention and occurs rarely when gaming unlike engagement and engrossment, which occur more frequently [8]. Total immersion occurs when the player can associate with the game characters and feel the atmosphere of the game [24]. Chapter 2. Background and Related Work 12

2.6 Real World Dissociation Factor Real world dissociation (RWD) refers to the component of the immersive experience in which a person is less aware of the real world as a result of their involvement in the game world [8]. It has been found that players’ intention to play a game includes factors as perceived enjoyment, escapism, emotional cop- ing, social satisfaction, achievement, challenge, excitement, leisure and the need for power [27]. People play games with the intention of experiencing RWD, i.e., they want to be less aware of the surroundings. RWD factor is one of the main reason for playing games, as this leads to the sense of escapism. This escapism can be categorized as: escaping into the world of fantasy and escaping from the real world [8]. When playing games, the imagination of the fanciful world diﬀer- entiating from the real world is known as escapism into the world of fantasy [8]. Further, players are inspired by opportunities as escapism and ﬂow for enjoyment in the world of fantasy which is not possible in the real world [21]. Escaping from the real world allows the player to manage their psychological states by relieving stress and boredom, reducing loneliness, passing time or providing escape, further leading to RWD [21].

Jennett, Cox and Cairns use the idea of real world dissociation to explain immersion in computer games [23]. The component dubbed "real world dissociation" (RWD) received strong loadings for items expected to measure losing track of time, losing awareness of surroundings and mental absorption [5].Hence, losing track of time, losing awareness of surroundings and mental absorption are considered as the categories of RWD. The categories and sub-categories of RWD are tabulated below [8].

A person while playing a game gets fascinated to escape into a world of fantasy, but this always doesn’t lead to RWD to occur. Playing games provide gamers with a distraction and an alternative reality to participate in [12]. This kind of appealing aspects of leisure activities is called as escapism. Crawford defined such escapism as one of the reason to "slip away" from the real world. C.Jennett conducted a grounded theory on such perceptions and concluded with three aspects of a person interaction with RWD while playing a game [8]. The three aspects associated with RWD are defined as sense of control, game performance and the level of challenge [8]. A player while playing a game experiences a sense of being in control [27]. At the same time a player needs to know the controls of the game to get immersed which influences RWD as it is an interaction between the action on the interface and the screen [8]. During the game-play users enjoy the hallucinatory power or the joy of being able to control the world of fantasy inside the game getting associated with RWD [27]. Concentration is the extent to which a person is associated with a lack of awareness of elapsed time and self-consciousness while playing the game [27]. Hence, game performance Chapter 2. Background and Related Work 13

Categories Sub-categories

Less aware of surroundings 1. Unaware of relevant sounds.

2. Unaware of time passing.

3. Unaware of lightning and ones own sense of movement.

Mental transportation 1. "Being" the character.

2. Having a place in the game world.

Less likely to respond to surroundings 1. Aware of audible hearing sounds. 2. Aware of time passing.

3. Aware of symbolic sounds.

Table 2.2: Characteristics of RWD with sub-categories also influences RWD as a person involves a high degree of concentration and a sense of personal control as proposed by Csikszentmihályi [21]. A player gets dissociated from the real world when there is a better performance in the game, i.e., scoring good points in that particular game and therefore RWD is influenced by the game performance [8]. Challenge is one of the major contributing factor to enjoyment while playing games [3]. Ryan et al. described that the players’ rating of enjoyment and desire to play games were strongly predicted by the need of challenging factor that immerses them in the virtual world [21]. The difficulty level of the game also influences the player’s performance leading to de-motivation for the player. Hence, challenge will have an impact on RWD in terms of game performance, reaching new levels and gaining points [8]. There are three broad categories which influence the RWD factor [8]: 1. Features of the game 2. Features of the environment 3. Features of the person Chapter 2. Background and Related Work 14

Figure 2.3: Factors aﬀecting RWD

2.7 Research Gap Identification As game immersion is a developing field [1][28], there is no standard method yet to measure immersion in games. The idea of relating RWD factor to game immersion has already done in [12],[8], but up to date, no empirical research has been done directly on the effects of immersion when a player is detached from the real world [18]. Further, measuring game immersion between experienced and inexperienced players has also not been well studied [5].Hence, this research study will fill up the gap in the literature on the measurement of immersion when dissociated from the real world between experienced and inexperienced players. Therefore, as a way of exploring further how the real world dissociation factor might be related to the level of game playing experience of the player, the experiment will be carried out using two different groups of players, one classified as experienced game players and one classified as inexperienced players.

2.8 Contribution The purpose of this study is to contribute to developing a base for measuring game immersion. This immersion is measured based on the player’s subjectively perceived sense of immersion, using the RWD factor. An important contribution of this studies is that RWD is not just something that people talk, but it a factor which can be measurable. RWD when measured reveals about a player while playing in game world. In previous studies it has been focused on how players are immersed in an immersive task and non-immersive task. But this research focused on comparing the immersion levels between experienced and inexperienced players, which is considered as a new research to the field of computer science and another contribution from this research. This research has contributed to HCI by giving a number of insights into the nature of RWD, a factor of game Chapter 2. Background and Related Work 15 immerison about which a little is known before. Game developers can have an advantage through this research by knowing how to design games for experienced and inexperienced players to make them more immersive as this research aims to find immersion levels for both experienced and inexperienced players. This also can be one of the contribution to the field of game development. Chapter 3 Method

This section describes the research method that has been used to accomplish the aims and objectives which are described in section 1.2. In the context of a research project, a method refers to an organized approach to problem-solving that includes (1) collecting data, (2) formulating a hypothesis or proposition, (3) testing the hypothesis, (4) interpreting results, and (5) stating conclusions that can later be evaluated independently by others [29].

The main goal of this study is to figure out the difference in game immersion between experienced and inexperienced players when measured by RWD factor. In order to fulfill this goal, literature review and an experiment have been conducted. By conducting a literature review various approaches have been explored which are used to measure game immersion through RWD. Among those approaches a suitable one is chosen to conduct an experiment for measuring and comparing game immersion between experienced and novice gamers.

Motivation of Selected Research Method Literature review will be usually conducted to identify and organize the concepts in the relevant literature [30]. Every research has to be started by knowing the existing knowledge in that subject area, therefore literature review has to be un- dertaken at an early stage of development of research [30]. Without building the state of previous research, it is not possible to establish how a new research advances the previous research [31], hence literature review has been chosen to explore knowledge about the previous researches.

Experiments are usually conducted to compare number of diﬀerent techniques, methods, working procedures, etc [32]. In many studies, user may be a variable while conducting an experiment, such experiments are called user studies in the form of an experiment [33]. Applicable use of humans or users in experiments allows numerous types of rich measurement that give insight into the estimation of computational methods. Researches have been carried out to conduct user studies in the form of an experiment to measure the immersion levels in players at diﬀerent stages [3][4][5][8][12]. Similarly in this research to measure game im-

16 Chapter 3. Method 17 mersion accurately among the players, user studies in the form of an experiment has been conducted.

Surveys are conducted when a technique or a tool already taken place or before it is introduced [32]. They are never conducted to create an understanding of the particular sample but are used in opinion polls and market research. In this study, survey is not chosen as the method, since the participants answer the questionnaire without the gaming controlled environment, therefore the results about their perceived immersion may not be accurate,Further in this research, accurate results for perceived immersion of the user is requires to compare immersion between experienced and inexperienced players, hence survey is not chosen.

A case study is conducted to explore a single entity or phenomenon within a particular time space [32]. They are suitable for industrial evaluation of research methods and tools to avoid scale-up problems. The results obtained by case studies are diﬃcult to generalize and harder to interpret, i.e., it cannot be generalized to every situation [32]. In this research, the diﬀerences when compared between experienced and inexperienced players must be generalizable to all the population, hence the sample is chosen from the population having same characteristics to generalize the results, therefore case study is not selected as the method.

3.1 Literature Review The ﬁrst step of the research was to conduct a literature review to identify and explore various approaches that are used to measure game immersion. This study helped to capture problem domain and improve the background of the research. And it also helped to explore and understand about real world dissociation factor and various approaches used to measure game immersion which are the answers to RQ1.

Literature review is a summary of the subject field that supports the identification of specific research questions. The objective of literature review is to summarize the state of art in that particular field of subject [30]. The purpose of writing a literature review is that it provides a framework for relating new findings to previous findings [31]. According to Justus Randolph, conducting a literature review is to demonstrate the author’s knowledge about a particular field of study, including vocabulary, keywords,theories, its methods and history [31]. According to Rowley and Slack, literature review must be conducted by following fundamental steps [30]. The steps that have been followed to suit the current research are as follows:

Step 1: Identifying required information resources - Chapter 3. Method 18

There exist a range of information sources from where relevant literature can be taken, which might be used to answer the research questions [30]. One of the information resource chosen is the online databases as the articles in scholarly and the research journals form the core of the literature review [30]. The online databases that have been used to obatin the relevant literature include Science Direct, ACM Digital Library, Inspec, Scopus, Google Scholar, Springer and IEEE Explore. The other information resource chosen is books as they provide a summary of current ideas which are regularly updated. Another advantage of books is that they include bibliographies or lists of other references to other useful sources [30].

Step 2: Search the relevant scientific literature from the information resources - The second step in the literature review is to identify and choose the relevant literature from the information resources which have been mentioned in step 1. Various search strings have been formulated as suggested by Rowley and Slack to obtain the relevant literature from the current information resources [30]. The keywords during the search included "game immersion", "real world dissociation", "game immersion" AND "real world dissociation", "real world dissociation" AND "computer games", "game immersion" AND "computer games". Further improv- ing of the search strings have been done based on the results obtained. The results obtained were filtered to contain articles whose full-text was available in the online databases. Second stage of filtering included the titles and abstracts of the obtained articles. If the titles and abstracts were relevant to the literature of the study, the full-text of the article was read to implement the source literature to the current research. Hence, the instructions mentioned in [30] were followed accordingly to obtain the required scientific literature from various sources and further evaluate them to perform the current research .

Step 3: Drawing together the literature review - The ﬁnal step to perform literature review is to gather and draw all the sources together from the selected scientiﬁc literature. The literature obtained are categorized based on the research aim and objectives. The data relevant are then captured and documented accordingly in the Chapter 1 (Introduction), Chapter 2 (Background and Related Work) and Chapter 4 (Results of Literature Review - Section 4.1). Further, the literature review was deepened by following the forward and backward snowballing technique and further looking into the references of the chosen papers [34]. Chapter 3. Method 19

Google Science- Springer Keywords ACM Scopus Scholar Direct

"Game immer- 15 24 1350 37 89 sion"

"Real world 1 1 89 5 7 dissociation"

"Game im- 1 1 72 2 3 mersion" AND "Real world dissociation"

"Real world 1 1 62 5 3 dissociation" AND "Com- puter games"

"Game im- 15 4 765 18 48 mersion" AND "Computer games"

Table 3.1: Search results for literature review Chapter 3. Method 20

3.2 Experiment The next step of the research after literature review was to conduct user studies in the form of an experiment. The experiment conducted helped to identify the diﬀerence in game immersion between experienced and novice players. Ex- periments focus on investigating variables and the way they are aﬀected by the experimental condition. They are typically used to verify or falsify a previously formulated hypothesis [29]. The experiment in this research has been conducted by following the steps described below:

• Selection of Games

• Participants and Sampling

• Questionnaire Development

• Procedure

• Data Analysis

3.2.1 Selection of Games Games provide people with a flow like experience in which individuals are con- stantly engaged and enjoy themselves at the same time [1]. Playing games socially is more immersive than playing alone [35], hence the games chosen in this research are two popular multiplayer online game genres: massively multiplayer online battle arena (MOBA) and first person shooter (FPS) games [36][35]. The game chosen in MOBA is Dota 2, as it is being played by millions of players nowa- days [35]. FPS games as CS:GO is often a subject of public interest, it is one of the most popular and successful games in the world [37][38]. Hence, the game chosen in first person shooter (FPS) games is Counter Strike:Global Offensive.

The other reason for choosing Dota 2 and CS:GO are that they are the most trending games in the current Steam1 website [39]. Steam is a website of Valve corporation which is a video game developer and digital distribution company [40]. Steam is one of the most popular online gaming digital distribution plat- form [41]. More than 4,500 computer games are oﬀered by Steam and it serves over 100 million active users [41]. Valve has built this social network in order to provide social and community functionality for various games [42]. Steam func- tions as a social network and control for the work and trade of games between players. It is virtual storefront through which gamers purchase and play games [40]. Steam accounts save personal ﬁles and settings on the online accounts [42].

1http://store.steampowered.com/ Chapter 3. Method 21

The players with steam accounts can set up private games with friends or join public games to play socially. Among the diﬀerent games Dota 2 and CS:GO have been the top two mostly played games by a large number of players, the snapshot of the top games in Steam website is shown in Figure B.1. Therefore, both the trending games Dota 2 and CS:GO have been chosen in this research as more number of participants can be chosen as gamers from the sample to get generalisable results.

3.2.1.1 Description of Dota 2 Multiplayer online battle arena (MOBA) games are the sub-genre of real time strategy (RTS) games, where Dota 2 (Defense of the Ancients 2) is one of the popular MOBA game [43]. Dota 2 is the game which is viewed from an isometric perspective. There are three main pathways in the Dota 2 map which are referred to as "lanes". These lanes are differentiated as top, middle and bottom. The lanes form an important strategic points of attacks while defensing the opposing team. Further, there are variety of sub-environments in Dota 2 surroundings. It is played by two teams consisting of 5 players each. Each team will be controlling a one of 111 avatars known as "heroes". [44]. Each hero has unique design, different skills and can gain different levels in the game. Additionally, a hero can be armed with extra objects to improve the characters base statistics, alter or add new abilities [44]. Earning of better versions of hero’s abilities or new abilities and increasing the experience of the game characters are done by gaining gold during the game. Extra items required during the game are also earned in similar way by gold.

Another important factor is the communication between team members during the game, as those strategies and tactics are the key components in the game. Players can communicate with each other through voice chat, text chat, by writing on the mini-map (represented in Figure 3.1) or by giving pings (messages on the arena itself) to each other. On an average, a particular match can last up to 40-45 minutes, although there is no time limit. The battle between two teams is competed in a square vir- Figure 3.1: Minimap of the arena in tual arena which is geographically bal- Dota 2 during a battle between two anced, where same arena is used in ev- teams ery match. The arena is divided into two halves by a crossable river and are Chapter 3. Method 22 connected by three paths. Each half of the battle arena is owned by each team at the beginning of the match. The arena in the game consists of variety of fascinating game-related items, example, a base for each team with a central building. The objective of the game is to be the ﬁrst team to destroy the ancient to win the game. This ancients are guarded by towers, where towers are the series of defensive structures with opposing capabilities in Dota 2. Additionally, along with towers there are computer-controlled creatures called "creeps" which spawn throughout the game in groups from two buildings. This creeps rush into opposing team’s towers and players to attack the heroes and buildings in their way.

3.2.1.2 Description of Counter Strike: Global Offensive CS:GO is a first person shooter game having five different game modes. Among them one played professionally is Classic Competitive-Bomb Scenario [45], and hence is chosen for this research. This game mode is played between a team of 5 terrorists and 5 counter-terrorists, designing a terrorist bomb plant scenario [45]. The goal of the terrorist team is to plant a bomb and explode it, and or to kill all the counter-terrorists. The goal of the counter-terrorist team is to defuse the planted bomb, kill all the terrorists and have minimum one player alive in the absence of the bomb plant. The teams shift sides (from terrorists to counter- terrorists, vice-versa) in the halfway of the 15 rounds of the game, and the team that wins 16 rounds first wins the overall game [45].

If the game ends up with a tie (30 rounds, 15 rounds won by each team), then the game extends to overtime to regulate the winner. Each player begins the round having health 100 and zero armor points unless the player purchases the armor in the form of Kevlar vest or helmet. Players loose health and armor points by taking harm from knives, guns, tasers, bomb, grenade explosions, grenade contact and the damage can be dealt from opposing team, one’s own teammate or one itself. When a player looses all of their health, they die and will be un- able to play the game until the next round. Generally, professional players Figure 3.2: Map representing teams: ter- plan such kind of strategies to plant rorists and counter-terrorists in CS:GO the bombs, killing the opposing team Chapter 3. Method 23 players [45].

3.2.2 Participants and Sampling In a statistical study, a population is a set of collection of individuals or items in which investigation is done during a research [46]. Sample is the part of population from which information is collected [46]. The sample in this research are the participants who are gamers, selected from a population by conducting a closed interview with a ﬁxed set of questions [29]. Closed interviews does not allow adding or deleting of questions depending upon the replies of the participants, hence with respect to repeatability, it has an advantage over open interviews. This style of interviews are more suitable for statistical methods to analyze the results [29]. The questions asked to the participants were about their experience in the selected games for the experiment in this research. Based on this interview conducted the categorization of the participants was a binary split into two groups as: experienced and novice gamers. Experienced players are those participants who have played the game either to completion or have atleast one year of playing experience (for each respective game). Inexperienced players are those participants who have never played the game (for each respective game chosen for the experiment) [36].

The research between the groups was done by considering the playing experience of the participant (whether each participant had played the game that he or she was assigned to), and the game genre (which game each participant had been assigned to play). The participants were assigned to one of the four conditions of the research as follows: they were segregated as experienced and inexperienced players based on the interview conducted for the selected two games. If they mentioned that they had never played either of the game, then they were randomly assigned to one of the two inexperienced group of players (either MOBA or FPS). If they mentioned that they have experience (as mentioned previously how the experience has been considered in this research) in both of the games, then they were randomly assigned to one of the two experienced group of players. Finally, if they mentioned they have experience in one of the two games, they were assigned to either of the experienced group of the game they had experience in, or the inexperienced group of the game that they have no experience in [36].

3.2.2.1 Stratified Random Sampling The sampling strategy chosen in this research to select participants for the two groups is stratified random sampling [47]. The stratified random sampling provides an advantage that it is often logistically easier to divide the sample into strata. It allows the representation of each strata (groups) within the population and ensures that these strata are not over-represented [48]. The sample of Chapter 3. Method 24 population chosen was N=140 with age between 15-26years (Mean= 20.37, SD= 2.61). The total number of gamers for the game Dota 2 were 80 and CS:GO were 60. Among the 80 players of the game Dota 2, 46 players were experienced and 34 were inexperienced players. For the game CS:GO, there were 33 experienced players and 27 inexperienced players among the 60 gamers. Further, the stratified random sample was created in the following steps [49]:

• Step 1: Choose the relevant stratification- The main objective of this research is to find the differences in game immersion between experienced and inexperienced players, hence the stratification chosen is the experience (as defined in Section 3.2.2) of the gamer.

• Step 2: List the population according to the chosen stratiﬁcation- Each student was assigned with consecutive numbers from 1 to Nk in each stratum. As a result, it ended up with four lists, i.e.experienced and inexperienced players for each game.

• Step 3: Choose the sample size- The next step was to choose the sample size for both the games to be conducted. The sample size chosen for Dota 2 was 40 and CS:GO was 30. Presence of large sample size makes the research more time intensive, therefore, the number was chosen was narrowed as it reﬂects the limit of resources and time that is required to conduct the experiment.

• Step 4: Calculate the proportionate stratification- The proportionate stratification is calculated for each game to divide the players into strata. The calculation has been done by using the stratified random sample formula (sample size of the strata = size of entire sample / population size * size of each stratum) [50]. After calculating the proportionate stratification for each game, sample has been chosen by conducting simple random sample to choose the required sample.

• Step 5: Use simple random sample to choose the required sample- As the sample chosen for the game Dota 2 was 40, simple random sample has been performed to select those 40 players from the available 80 players. By using the proportionate stratiﬁcation and simple random sample, 22 experienced players were chosen from the available 46 players and 18 inexperienced players from 34 available gamers. Similarly, 30 players have been chosen for the game CS:GO from the available 60 players by conducting simple random sample to obtain the required sample to proceed with the experiment. The players were chosen similar to the participants of Dota 2 by using proportionate stratiﬁcation and simple random sample. 16 experienced and 14 inexperienced players were chosen from the available 33 experienced and 27 inexperienced players respectively for the game CS:GO. Chapter 3. Method 25

The results of the obtained samples for each game have been described in the Table 3.2.

Game Genre Dota 2 CS:GO

Experienced Players 22 16

Inexperienced Players 18 14

Table 3.2: Sample calculated for each game individually

3.2.3 Questionnaire Development The questionnaire that has been used in this research is developed from the findings of the previous studies which is based on Immersive Experience Questionnaire (IEQ) [4]. This IEQ has been created by using Brown and Cairns (2004)’s conceptualization as a basis and it aims to provide a general measure of immersive experiences, spanning a different variety of games [8]. The IEQ consists of questions related to five factors of immersion described by C.Jennett as: cognitive involvement, real world dissociation (RWD), challenge, emotional involvement and control. As this research focuses on RWD, the questions related to this factor have been chosen from the immersive experience questionnaire (IEQ). The IEQ consists of 32 questions, out of which 6 questions were related to RWD factor. For this research those 6 questions were chosen and further 3 questions were developed using positive and negative wordings to control the wording effects. Other questions have been developed based on real world dissociation factor which has been investigated by C.Jennett [5] and from the study of Brown and Cairns that to what extent players felt they were no longer attached to the real world [7]. The questionnaire has a total of 9 questions with 5-point Likert scale ranging from 1 (Strongly Disagree) to 5 (Strongly Agree). Among those 9 questions, 4 questions were negated as defined by C.Jennett in [4][8]. Participants were asked to answer this questionnaire using the 5-point Likert scale by describing their experience in the game. These questions have been analyzed to measure game immersion using RWD factor. Immersion scores have been computed by summing the answers given by the participants in the questionnaire. The questionnaire has been validated and analyzed by conducting a statistical technique [4].

3.2.3.1 Motivation for Choosing IEQ Questionnaire There are various subjective methods to measure game immersion which have been discussed in chapter 4. Each method has been brieﬂy described in Sec- Chapter 3. Method 26 tion 4.1.1.2. This section explains the reasons for choosing Immersive Experience Questionnaire and not the other methods.

The Immersive Experience Questionnaire (IEQ) measures the five components of immersion, which are a mixture of psychological factors (Cognitive Involvement, Emotional Involvement, Real World Dissociation) and game factors (Challenge and Control) [8]. This makes IEQ optimal for measuring different elements of immersion which emerge in different social conditions [18]. The Core Elements of the Gaming Experience Model (CEGE) questionnaire developed by Calvillo- Gámez et al. is not sufficient to measure the full range of immersive experiences (engagement, engrossment and total immersion) which are defined by Brown and Cairns [7][8]. Another method which is Extended Technology Acceptance Model (TAM) questionnaire, cannot predict perceptions such as whether a person is go- ing to have high immersive experience at a certain instance of game-play. Hence, IEQ will be more suitable to measure such an immersive experience at a specific instance of a game-play [8]. The immersion in Game Narrative Questionnaire only measures one type of immersive experience, which occurs in imaginative immersion (defined by Ermi and Mäyrä) [15]. While in contrast the IEQ aims to provide a general measure of immersive experiences, spanning a different variety of games [8]. IEQ varies from EVE-GP questionnaire as it does not aim to provide a complete measure of immersive experiences, rather it just measures those factors which are common to all immersive gaming experiences [8]. IEQ does not aim to be a extensive measure of the range of experiences and feelings that people have while playing the games, while it is generalisable to all the games [8]. Brockmyer claims that there is a limitation for Game Experience Questionnaire (GexpQ), it is associated with only one item related to immersion according to the categorisation of the questionnaire items [51]. Another limitation of GexpQ is that when one inspects the concepts of "presence" and "flow" that are said to be correlated with the questionnaire, the differences between them seem to be confused in the study of Brockmyer [8]. Thus, IEQ is concerned with specific instances of game-play rather than behavioral patterns of people over time [8]. Immersion, as defined by the five components of the IEQ, appears to overlap with the factors of GameFlow, Game Narrative Questionnaire, EVE-GP Questionnaire and Game Experience Questionnaire (GexpQ). Therefore, IEQ includes all the factors measuring different game experiences spanning wide variety of games [8]. The real world dissociation (RWD) factor of the IEQ specifically takes advantage of this experience by evaluating the degree to which player feels isolated from the real world [18]. Hence, IEQ is chosen to measure immersion differences when detached from the real world between experienced and inexperienced players using RWD factor. Chapter 3. Method 27

3.2.4 Procedure All the participants were asked to gather at BTH Karlskrona to participate in the experiment. Each participant was given instructions which did not divulge any information about game world, but only about user interface. Each participant was asked to get their own laptop to play the respective game that they have chosen. The Dota 2 inexperienced players were introduced to the minimap of the game and instructed about the game play by the experiment supervisor. At the same time, the Counter Strike:Global oﬀensive inexperienced players were asked to go through the beginning tutorial of the game, and they were told that if they encountered with any issue, they can take guidance from the experiment supervisor. Inexperienced players of both the games were explained with the basics of the game so that they can participate in the experiment without any interrup- tions.The experienced players were just asked to ﬁnish the game by playing the matches accordingly in both the games.

The total number of players for the game Dota 2 were 40, where 22 players were experienced and 18 were inexperienced. Each game requires 2 teams to play among themselves and further each team required 5 players for the game to be played. Therefore, all the players were randomly divided into 8 teams with 5 players each. After the segregation of the teams players were asked to play a match among themselves. The participants were asked to answer the questionnaire by rating their decision on a 5-point Likert scale after ﬁnishing the game. The answers were analyzed for the two groups (experienced and novice gamers for Dota 2) separately.

CS:GO was the second game chosen to conduct the experiment. There were 30 game players where 16 players were experienced and 14 were inexperienced. The game mode chosen is the classic-competitive bomb scenario which required 5 players as terrorists and 5 players as counter-terrorists for a game to be played. Therefore each game required 10 players for the experiment to be conducted. Hence 30 players were divided as 10 players for each game and were asked to play the game. The questionnaire was given to each player after completing the game and they were asked to answer it by rating their experience on a 5-point Likert scale. The answers of the questionnaire given by CS:GO game players were also analyzed for the two groups separately.

Each game lasted for 45-60minutes approximately for both the games Dota 2 and CS:GO, which has been a long duration for an experiment to be conducted. Therefore, every participant had a chance to play single game due to time con- straint and availability of the participants while conducting the experiment. When the experiment started, further instructions were not given to the players. From then comments were not made by anyone until the game has been ﬁnished. Fur- Chapter 3. Method 28 ther answers were not given to any of the questions about the game, particularly those of the inexperienced players. The experienced players of both the games when asked about their involvement have admitted that they have answered the questionnaire from their experience of the whole game, not just from what they had experience during the study.

3.2.5 Data Analysis The results obtained through the questionnaire are analyzed by using a statistical technique. Statistics is a procedure for collecting, analyzing, interpreting and drawing conclusions from information [46]. It can be called that collection of analytical and computational methods by which characteristics of a population are inferred through observations made in a representative sample from that population [52]. The main objective of statistical methods is to make the process of scientific research more productive,efficient and to gain understanding from data [52]. The branch of statistics which deals with the use of sample data to make inference about a population is called inferential statistics. Inferential statistics consists of methods for drawing and measuring the accuracy of conclusions of the population based on the sample of the same population. It consists of methods as point estimation and hypothesis testing which are all based on probability theory [46]. Therefore, the inferential statistical technique that has been chosen in this research is sample T-test and further the difference is calculated by using Cohen’s d effect size to know the difference between two groups.

3.2.5.1 Description of Independent Sample T-test Independent sample t-test is the statistical technique chosen in this research, as the two samples (i.e. two groups which are experienced gamers and novice gamers) taken are independent samples [53]. The t-test can tell if there is any significant difference between the two groups [53][54]. In this research there are four independent samples, i.e. experienced players and inexperienced players for the two games Dota 2 and Counter Strike. These independent samples are extracted from the same population (as described in Section 3.2.2) which displays a normal distribution and further the difference in the means is computed. Normal distribution is defined as the distribution of a sample which is characterized by its mean (µ) and standard deviation (σ) having a symmetric curve [46].

In this research, game immersion is measured between two samples by using RWD factor. T-test has been conducted for the two games separately, and the final difference is also measured separately between experienced players and inexperienced players of the two games. The test of group differences has been conducted to know whether two populations differ with respect to their mean scores on the factor real world dissociation [55]. The means and standard devia- Chapter 3. Method 29 tions of both the groups were calculated and measured. Further the comparison is done by performing hypothesis testing.

3.2.5.1.1 Hypothesis Testing A hypothesis is a statement about a characteristic of a variable or a collection of variables [56]. There are two hypotheses which are more relevant to most of the statistics. The ﬁrst is called null hypothesis, which is often abbreviated as H0. The second is called alternative hypothesis, which is abbreviated as Ha [56]. Both the hypotheses are deﬁned below:

• Null Hypothesis (H0): There is no signiﬁcant diﬀerence between experienced and inexperienced players with respect to game immersion when measured by RWD.

• Alternative Hypothesis (Ha): There is signiﬁcant diﬀerence between experienced and inexperienced players with respect to game immersion when measured by RWD.

3.2.5.2 Cohen’s D Effect Size Effect sizes are the important outcome of empirical studies and they allow the researchers to compare the magnitude of experimental treatments from one experiment to another [57][58]. Effect size is the difference between two means divided by the standard deviations of two conditions [57]. Cohen’s d can be used to compare effects across the studies, when the values are measured by 5-point, 7-point or 9-point scales [57]. A five-point Likert scale has been used in the questionnaire of this research which has to be analyzed to draw conclusions, therefore, Cohen’s d is the effect size chosen in this research to measure the difference in game immersion between two groups. The effect size signifies which group of players have more immersion than the other group of players. The difference here is calculated between experienced players and inexperienced players for the two games. This difference in game immersion will answer the RQ2. Cohen refers to the standardized mean difference between two groups of independent observations for the sample [57]. Cohen’s d has two advantages over the other effect size measurements [58]. First, it’s popularity is making it standard and further making it applied on a large number of published studies. Second is that Cohen’s suggestion which describes that effect sizes of 0.20 are small, 0.50 are medium and 0.80 are large which enables the researchers to compare experiment’s effect size to known benchmarks [58]. The formula for Cohen’s d is given as follows [58]: x − x d = e i Sp Key to Symbols, Chapter3.Method 30 disCohen’sdeffectsize xismean(averageofexperiencedorinexperiencedconditions)

Sp isStandarddeviation Subscripts: "e"refersexperiencedgroupand"i"referstoinexperiencedgroup

(n 1) s2+( n 1) s2 where, e e i i ; Keytosymbols, ne+ni nisNumberofsubjects sisStandarddeviation Chapter 3. Method 31

Figure 3.3: Experimental Process Chapter 4 Results

4.1 Literature Review

4.1.1 Methods for Measuring Game Immersion Game immersion can be measured subjectively or objectively. The subjective measurements can be collected and studied with the help of questionnaires and the objective measurements can be studied through task completion times and measuring of eye movements [4]. This section will be the answers to the ﬁrst research question.

4.1.1.1 Objective Measures for Game Immersion While playing a game, in transferring from a game task to a real world task, the real world task performance would be weakened in proportion to the level of immersion [4]. C.Jennett et al conducted one of the experiment to relate the experience of immersion to the objective measure of the time taken to complete a task in another action space [4]. The participants were asked to play a control task and a computer game in the experiment conducted. The control task chosen here was a tangram task. The immersion was compared between the two tasks based on the task completion times of both the tasks. Another experiment conducted was to measure eye movements using eye tracking as it has become one of the popular methodology for measuring attention of the people. The experiment included the investigation of how eye movements change over time within immersive and non-immersive games [4]. The eye movements which are considered as objective measures included saccades 1 and fixations 2 [4]. Smith and Graham also conducted an experiment using eye tracking device to measure the experience in immersion. The results in this experiment indicated that the majority of players felt more immersed while using the eye tracker [59]. Hua Qin et al. proposed that with different difficulty of rate change games, players would experience different degrees of immersion [3].

1Fast movements that redirect the eye to a new part of surroundings 2Intervals between saccades in which gaze is held almost stationary

32 Chapter 4. Results 33

4.1.1.2 Subjective Measures for Game Immersion The subjective measurements for game immersion include questionnaires as suggested by C.Jennett et al [4]. L.Nacke et al. conducted an experiment to measure game immersion based on player experience [28]. The different experiences that were focused in this experiment included boredom, immersion and flow. They were measured and analyzed using game experience questionnaire [28]. There are various gaming experiences which are measured subjectively through questionnaires. These different gaming experiences include presence, flow, involvement, enjoyment, arousal and immersion [8][21].

One of the important game experience dimension often linked to technological advancement is presence [60]. Presence is defined as the phenomenon of behav- ing and feeling as if in virtual world created by the computer displays [61]. The presence experience is highly important to video game players’ mediated interaction with a captivating and interactive virtual environment [60]. The presence questionnaires which are used in the studies for investigating gaming experiences are listed in the Table 4.1 [8]. • ITC Sense of Presence Inventory (ITC-SOPI) - Lessiter et al. created a questionnaire which is used to measure presence is the ITC sense of presence inventory (ITC-SOPI) [62]. It is a new state questionnaire measure that focuses on users’ experiences of media, with no reference to objective system parameters [62]. Hence it was created with 44 items to evaluate the presence across a range of media which includes 2D versus 3D, control versus no control, surround sound versus stereo [8]. ITC-SOPI identified four separate factors: (a) Spatial Presence, (b) Engagement, (c) Ecological Validity, (d) Negative Effects [63]. Ravaja et al. examined the emotional response patterns and sense of presence evoked by games with different characteristics (e.g., the view from which the game is played, naturalness of the game, amount of violence) [63]. The study conducted by using the ITC sense of presence inventory (ITC-SOPI) questionnaire resulted that different video games elicit different emotional response patterns and degrees of presence [63].

• The Presence Questionnaire - It included 32 questions and it was created to measure the degree to which the individuals experience presence in virtual world of games [64]. The questionnaire also measures the intensity of the experience contributed by the factors namely control, sensory, distraction and realism [64]. The presence that occurs at the extent of game immersion is more a result of psychological factors (e.g., engagement in game-play) [8].

• The MEC Spatial Presence Questionnaire - It also includes the concept of engagement in the conceptualization of presence [65]. The MEC- Chapter 4. Results 34

SPQ consists of several scales that measure different dimensions of spatial presence [66]. It includes four process factors (Attention Allocation, Spa- tial Situation Model (SSM), Self Location (SPSL) and Possible Actions (SPPA)), two variables relating to states and actions (High Cognitive In- volvement and Suspension of Disbelief (Sod)) and three trait variables (Do- main Specific Interest, Spatial Visual Imagery and Absorption) [66]. The experiment conducted by Laarni et al. resulted in measuring the spatial presence for each individual factors for different games [66].

Questionnaire Factors Measured by:

ITC Sense of • Spatial Pres- Eastin and Presence In- ence Griﬃths [20] ventory [62] • Engagement

• Ecological Validity

• Negative Ef- fects

Presence • Control Ravaja et al. Question- [63] naire [64] • Sensory • Distraction

• Realism

MEC Spatial • Process Fac- Laarni et al. Presence tors [66] Question- naire [65] • Trait Factors

Table 4.1: Presence Questionnaires used in investigating gaming experience

Flow is another concept that bears similarities with immersion [8]. Csikszent- mihalyi describes flow as the "holistic sensation that the people feel when they act Chapter 4. Results 35 with total involvement" [10]. Immersion involves a loss of sense of context, while flow defines a level of complete involvement [10]. There were eight components of flow which were identified: concentration on the task at hand, a challenging activity requiring skill, clear goals, direct/immediate feedback, sense of control over one’s actions, a loss of self-consciousness, an altered sense of time and a merging of action or awareness [8]. Flow theory is one of the prominent formu- lation used to explain the subjective experience while playing games [21]. It is based on the assumption that the elements of enjoyment are universal, providing a general model that summarizes the concepts common to all when experiencing enjoyment (e.g., ability to concentrate on task) [67]. Several researches have been done on measuring flow using various questionnaires which have been described in Table 4.2.

• EGameFlow Questionnaire - It is a transformation of GameFlow questionnaire which is created to assess user-enjoyment of e-learning games [68]. The EGameFlow questionaire consists of 42 items, each with eight dimensions to measure the level of flow. Fu et al. used EGameFlow questionnaire to four e-learning games for scale verification of flow in games [68]. The re- placement of a factor to transform GameFlow questionnaire to EGameFlow questionnaire was of "player skill" replaced with "knowledge improvement" [8].

• GameFlow Questionnaire - A model of game enjoyment was created by Sweetser and Wyeth with eight core elements as mention in Table 4.2 [67]. Based on the game model, the GameFlow questionnaire was created with 35 items to measure enjoyment in games. Using this questionnaire Sweetser and Wyeth compared two real time strategy (RTS) games, where one rated high in professional game reviews and the other as poor [67].

• Video Game Experience Sampling Method - The Experience Sam- pling Method (ESM) created by Csikszentmihalyi was modified by Holt and was termed as Video game Experience Sampling Method (VESM) [69]. The VESM consists of whether the parcticipant was thinking about what they are doing and the nine factors which are described in the Table 4.2 [69]. Holt used VESM in the experiment conducted to reveal the differences in flow over time in one-hour session of game-play [69].

There are various other questionnaires, subjective measure, measuring speciﬁc aspects of gaming experience which are related to game immersion summarized in Table 4.3 [8].

• Immersion in the Narrative Game Questionnaire - The questionnaire for measuring player immersion in computer game narrative was proposed with six elements based on the three stages of immersion deﬁned by Brown Chapter 4. Results 36

Questionnaire Factors Measured by:

EGameFlow • Concentration Fu et al. [68] Questionnaire [68] • Challenge • Knowledge Improvement

• Control

• Goal Clarity

• Feedback

• Immersion

• Social Interaction

GameFlow • Concentration Sweetser and Questionnaire Wyeth [67] [67] • Challenge • Player Skills

• Control

• Clear Goals

• Feedback

• Immersion

• Social Interaction

Video Game Ex- • Hard to Concentrate Holt and Mit- perience Sam- terer [69] pling Method • Skill and Challenge [69] • Control of Actions

• Wish Doing Something Else

• Depth of Consciousness

• Something at Stake in the Activity and Success

Table 4.2: Questionnaires used in investigating gaming experience in concept of ﬂow Chapter 4. Results 37

and Cairns [70]. Initially the questionnaire consisted of 33 items, later after modifying it ended up with 30 questions [70]. The questions about curiosity dimension was developed based on the study of Pace. The questions regarding concentration and control dimensions were written on the basis of Sweetser and Wyeth’s research about game ﬂow criteria [70]. The other dimension challenge questions were constructed based on the combination of the researches of Pace as well as Sweetser and Wyeth [70]. Qin et al. composed the other questions of the two dimensions comprehension and empathy for the Immersion in the Narrative Game Questionnaire [70].

• Extended Technology Acceptance Model Questionnaire - It was extracted from the Technology Acceptance Model (TAM) to incorporate social influences and flow experiences as belief-related constructs [8]. The questionnaire consisted of 19 items to measure the acceptance of the online games [71]. Hsu and Lu used the Extended TAM Questionnaire in their study and justified that social norms, attitude and flow experience accounted for 80 percent of game-playing [71]. When Extended TAM Ques- tionnaire is considered, it can be argued that one’s perceptions of games is a good predictor of how often a person plays games [8].

• Core Elements of the Gaming Experience (CEGE) Model Ques- tionnaire - The CEGE Questionnaire was developed to measure the ob- servable variables in order to understand the behaviour of latent constructs [16]. It was developed using an iterative process following the usual psy- chometric guidance [16]. The questionnaire consisted of 38 items with 10 scales. The scales included enjoyment, CEGE, puppetry, frustration, videogame, control, facilitators, ownership, game-play and environment [16]. The ﬁrst two scales were included as a reference to see the relationships between CEGE, enjoyment and frustration. The remaining scales were the latent variables obtained from the theory [16].

• Social Presence in Gaming Questionnaire (SPGQ) - It consists of 25 items amd was created to measure the gamers’ awareness of and involvement with their co-players [72]. Dimensionality analysis of the social presence gaming questionnaire (SPGQ) resulted in three sub-scales as empathy, negative feelings and behavioral engagement [72].

• Computer Apathy and Anxiety Scale (CAAS) - It was primitively created to diﬀerentiate behavioral addiction in computing from high engagement of addiction [73]. There were two versions of CAAS which were actually created to measure concepts in gaming. Among the two, Charlton and Danforth used the Addiction-Engagement part of CAAS which consisted of 19 items [74]. CAAS is concerned with the degree of computer usage, a highly engaged person being someone that plays games a lot [8]. Chapter 4. Results 38

Questionnaire Factors Measured by:

Immersion in the • Curiosity Qin et al. [70] Narrative Game Questionnaire [70] • Concentration • Control

• Challenge

• Comprehension

• Empathy

Extended Tech- • Social Norms Hsu and Lu nology Acceptance [71] Model Question- • Perceived Critical Mass naire [71] • Perceived Ease of Use

• Perceived Usefulness

• Flow Experience

• Attitude towards Online Games

• Behavioral Intentions to Play Online Games

Core Elements of • Puppetry Factors (Control, Calvillo- the Gaming Expe- Ownership, Facilitators) Gámez et al. rience Model Ques- [16] tionnaire [16] • Video Game Factors (Environ- ment, Game-play)

Social Presence in • Empathy De Kort et al. Gaming Question- [72] naire [72] • Negative Feelings • Behavioral Engagement

Computer Apathy • High Engagement Charlton and and Anxiety Scale Danforth [74] [73] • Addiction • Comfort

Table 4.3: Questionnaires used in investigating gaming experience that measures speciﬁc aspect of games Chapter 4. Results 39

In the concept of gaming, there are distinct questionnaires which aim to capture the full experience of gaming [8]. These methods of measuring game immersion have been outlined in Table 4.4.

• SCI Model Questionnaire - The SCI Model for game immersion has already been described in Section 2.3.1 which has been developed by Ermi and Mäyrä. The SCI Model Questionnaire was constructed based on that model consisting of 18 items [15]. It was created to measure the three components of immersion namely sensory immersion, challenge-based immersion and imaginative immersion [15].

• Game Experience Questionnaire (GexpQ) - The Game Experience Questionnaire created by IJsselsteijn et al. consists of 33 items that are scored to obtain measures on seven different components described in Ta- ble 4.4 [75]. It can assess experiential constructs of immersion, tension, competence, flow, negative affect, positive affect and challenge with good reliability [76]. It has been used to compare the participants’ experiences of the game levels designed to differ in game-play (strength of opponents, choice of weapons and pacing of the game) [76].

• Game Engagement Questionnaire (GengQ) - It was developed using both Classical Test Theory and the Rasch Rating scale model [51]. The GengQ provides a psychometrically strong measure of levels of engagement particularly evoked while playing video games [51]. The GengQ was a 19 item version, where the items were developed that reflected commonly reported game-playing experiences that were consistent with descriptions of distinct levels of engagement in game-playing [51]. The motivation for the development of GengQ was for the measurement of engagement in playing video games that should be useful in determining the impact of playing video games, specifically violent games [75]. The primary difference between GexpQ and GengQ was that GexpQ captures a broad range of player experience while GengQ was more concerned with developing a one-dimensional scale [75].

• EVE-GP Questionnaire - It was developed to understand multidimen- sional user experiences in games [8]. The questionnaire consisted of 180 items which is based on Presence-Involvement-Flow-Framework (PIFF) [77]. The PIFF consolidates various concepts that are commonly referred to as gaming (e.g. presence, ﬂow) [77]. The main aim to develop the questionnaire was to grasp the complex and multivariate nature of the play experience and make it measurable [77]. Chapter 4. Results 40

Questionnaire Factors Measured by:

SCI Model Ques- • Sensory Immersion Ermi and tionnaire [15] Mäyrä [15] • Challenge-based Immersion

• Imaginative Immersion

Game Experience • Immersion Lindley et al. Questionnaire [76] [76] • Tension

• Competence

• Flow

• Negative/Positive Aﬀect

• Challenge

Game Engagement • Immersion Brockmyer et Questionnaire [51] al. [51] • Presence

• Flow

• Absorption

EVE-GP Question- • Role Engagement Takatalo et naire [77] al. [77] • Attention and Valence

• Interest and Importance

• Co-presence

• Arousal and Interaction

• Physical Presence

• Impressiveness

• Competence

• Challenge and Control

• Enjoyment and Playfulness

Table 4.4: Questionnaires used in investigating gaming experience which aim to capture full gaming experience Chapter 4. Results 41

4.2 Experiment After conducting the experiment, Immersive Experience Questionnaire (IEQ) was answered by all the participants who have participated in the experiment playing both the games Dota 2 and CS:GO. All statistical analyses were performed using SPSS3 24.0 . The immersion scores were calculated by summing the values answered by the participants based on the Likert scale as 1 for strongly disagree for immersion question and 5 for strogly agree. This was adjusted accordingly for the positive and negative questions. The negated questions were calculated by reverse scoring which means that the numerical scale was measured in the opposite direction [78]. The immersion values for each individual describe the extent to which a player is dissociated from the real world as measured by RWD. The measures of game immersion for both the games have been described in the below sections.

4.2.1 Measures of Game Immersion 4.2.1.1 Dota 2 The immersion scores for the game Dota 2 were calculated for both the groups experienced and inexperienced players individually. The values of immersion scores for experienced players and inexperienced players of Dota 2 are shown in section C.1 and section C.2 respectively. These immersion scores are represented graphically for each individual of the two groups below.

Figure 4.1: Immersion scores for experienced players of Dota2

The graphs were plotted for 22 experienced players and 18 inexperienced players respectively according to the immersion scores measured. The participants

3Statistical Package for the Social Science - It is a statistical package developed by IBM which can perform highly complex data manipulation and analysis with simple instructions (http://www-03.ibm.com/software/products/en/spss-statistics) Chapter 4. Results 42

6, 17 and 20 showed relatively lower immersion levels when compared to the remaining experienced players as shown in the Figure 4.1. Inexperienced players had relatively higher immersion levels on an average as shown in Figure 4.2 when compared to experienced players. The comparison between experienced and inex-

Figure 4.2: Immersion scores for inexperienced players of Dota2 perienced players for the game Dota 2 has been plotted in the graph (Figure 4.3) where the immersion scores are measured by RWD.

Figure 4.3: Comparison of immersion scores for experienced and inexperienced players (Dota 2)

4.2.1.2 Counter Strike:Global Oﬀensive CS:GO was the second game for which immersion scores were calculated for both the groups separately. The values of immersion scores of experienced and inexperienced players of CS:GO are shown in section D.1 and section D.2 respectively. Chapter 4. Results 43

The results for each individual were obtained and hence represented graphically. The graphs were plotted for 16 experienced players and 14 inexperienced players respectively according to the immersion scores measured. The immersion scores obtained between two groups individually are shown in the graphs below.

The immersion levels of both experienced and inexperienced players were plotted in a graph for a clear pictorial understanding.

Figure 4.4: Immersion scores for experienced players of Counter Strike

Figure 4.5: Immersion scores for inexperienced players of Counter Strike

All the players of two groups of CS:GO showed similar levels of game immersion. As in Dota 2 experienced players didn’t show unequal levels of game immersion, but the immersion levels for all the experienced players were almost the same. Comparatively, participants 5 and 11 showed relatively less immersion scores than other experienced players. In inexperienced players graph, it is clear that almost all the players showed high immersion scores, but while com- Chapter 4. Results 44 paring participants 2,3,5 and 6 showed relatively higher immersion scores than the remaining inexperienced players.

Figure 4.6: Comparison of immersion scores for experienced and inexperienced players (CS:GO)

The comparison between the immersion scores of experienced players and inexperienced players has been plotted in a graph and shown below for a pictorial representation. The immersion levels of the experienced players was relatively low when compared to the inexperienced players as shown in Figure 4.6. The graph represents the difference in game immersion for both the groups, further the significant difference is calculated in the chapter 5. Chapter 5 Analysis

5.1 Data Analysis for Results Data analysis refers to the evaluation of the data systematically against the objectives of the research [29]. The data collected for this research are represented in Appendix C and Appendix D as immersion scores which are the results. These results infer the information pertaining to this research. Therefore, this sections gives the answers to the research questions.

5.1.1 Difference in Game Immersion The statistical method of analysis refers to collecting, arranging and drawing general regularity from the data [79]. Sample T-test is the statistical technique chosen to find if there is any significant difference between the two groups for both the games.

5.1.1.1 Independent Sample T-test Independent sample T-test is used to test if there is any significant difference between the two groups (experienced and inexperienced players) [80]. All the statistical analyses were performed using SPSS 24.0. The parametric statistical procedure has been chosen in this research as it confides on assumptions of the shape of the distribution in the elemental population and the parameters [81]. In this research, the assumption is a normal distribution for the population and the parameters considered are the means and standard deviations for the assumed distribution. Hypothesis test was conducted for both the groups of the two games individually to find if there is any significance difference for game immersion between the experienced and inexperienced players. There are two types of inferential tests when using inferential procedures [55]:

1. Tests of group diﬀerences

45 Chapter 5. Analysis 46

2. Tests of association In this research, tests of group differences has been chosen as the inferential procedure to know whether the two populations (experienced and inexperienced players) differ with respect to their mean scores on RWD factor. Hypothesis test was conducted for both the games by assuming 95 per cent (0.95) confidence interval and significance level as 0.05. The confidence interval is considered as 0.95 as the statistical analyses are conducted by SPSS 24.0 and significance level as 0.05 as it is the most accepted cut-off value [55].

5.1.1.1.1 Hypothesis Testing for Dota 2 The means and standard deviations of the two groups experienced and inexperienced players were compared by conducting independent sample T-test by SPSS 24.0. The values are in the ﬁgure below.

Figure 5.1: Means and Standard deviations for Dota 2

After conducting the T-test for Dota 2 game, the mean and standard deviation of the inexperienced and experienced groups were 30.33 and 26.72 respectively; corresponding standard deviations were 3.89 and 5.65 respectively. The T-test was conducted with 95 percent conﬁdence interval of the diﬀerence and the values are shown in the Figure 5.2.

Figure 5.2: Values for Independent Sample T-test (Dota 2)

The p-value calculated by SPSS was 0.027 for the groups of Dota 2 game which is less than the level of significance (i.e 0.027 < 0.05), which concludes to reject null hypothesis [46]. The null hypothesis (H0) which says that there is no difference in game immersion between experienced and inexperienced groups has been rejected. Hence, there is significant difference for game immersion between Chapter 5. Analysis 47 the two groups for the game Dota 2. The difference in means of experienced and inexperienced players has been represented in Figure 5.3. Further the significant difference between the two groups has been calculated using the effect size Cohen’s d that has been mentioned in the Section 5.1.1.2.

Figure 5.3: Diﬀerence in means of immersion scores for both the groups (Dota 2)

5.1.1.1.2 Hypothesis Testing for Counter Strike:Global Oﬀensive The means and standard deviations of experienced and inexperienced players of CS:GO were also compared by conducting independent sample T-test by SPSS. The values are shown in the Figure 5.4.

Figure 5.4: Means and Standard deviations for CS:GO

After conducting the T-test, 30.07 and 27.25 were the means of inexperienced and experienced players respectively; further corresponding standard deviations were 3.73 and 2.95 respectively.

Figure 5.5: Values for Independent Sample T-test (CS:GO) Chapter 5. Analysis 48

The T-test was conducted with 95 percent conﬁdence interval of the diﬀerence and the values are shown in Figure 5.5.

The p-value obtained was 0.028 for the two groups of CS:GO which is less than the level of significance (i.e 0.028 < 0.05), which means to reject the null hypothesis [46]. Therefore, there is significant difference in game immersion between experienced and inexperienced players. The difference in means of experienced and inexperienced players has been represented in the Figure 5.6. The significant difference between the two groups has been calculated and mentioned in the Section 5.1.1.2.

Figure 5.6: Diﬀerence in means of immersion scores for both the groups (CS:GO)

5.1.1.2 Effect Size Statistical tests of significance tell if the experimental results differ from chance expectations, but effect-size measurements tell the relative magnitude of the experimental treatment [58]. They tell the size of the experimental effect [58], i.e., the difference in game immersion between experienced and inexperienced players in this research. The difference in game immersion has been calculated for the two games Dota 2 and CS:GO using Cohen’s d that has been described in the below sections.

Based on the benchmarks as suggested by Cohen, the eﬀect sizes can be interpreted as small, medium and large [57]. A commonly used interpretation for Cohen’s d is referred to as small (d=0.20), medium (d=0.50) and large (d=0.80) [46][57].

Figure 5.7: Interpretation of Cohen’s d Chapter 5. Analysis 49

5.1.1.2.1 Cohen’s D for Dota 2 Cohen’s d for the diﬀerence in game immersion between experienced and inexperienced players of Dota 2 has been calculated. Cohen’s d has been calculated by using the formula which has been mentioned in the Section 3.2.5.2 that includes the means and standard deviations of both the groups. When calculated, the Cohen’s d eﬀect size was 0.7423 which is almost equivalent to 0.8 and can be interpreted as large as shown in Figure 5.7.

5.1.1.2.2 Cohen’s D for Counter Strike:Global Offensive The significant difference in game immersion between experienced and inexperienced players of the game CS:GO has been calculated using Cohen’s d effect size. Cohen’s d was calculated similar to Dota 2 as mentioned in the formula in Section 3.2.5.2. The Cohen’s d effect size obtained after calculations was 0.8383, which is also almost equivalent to 0.8. As similar to the game Dota 2, the effec size for the game CS:GO between experienced and inexperienced players can be interpreted as large as shown in Figure 5.7. Chapter 6 Discussions and Validity Threats

6.1 Discussions This section describes the findings of the previous sections by summarizing them. The main aim of the study was to investigate measurement of game immersion when measured by RWD between experienced and inexperienced players. This study illustrates momentary nature of game immerison and RWD. This research also contributes to the field of HCI by giving an observation of RWD about which a very less is known when compared between experienced and inexperienced players. It has been defined in literature that if a player is present in alternative game world (i.e., while playing game), there may be measurable effect on their return to real world, it is termed as RWD factor [4]. The RWD factor has been measured and calculated for experienced and inexperieced players who are segregated based on their level of playing experience. As mentioned earlier, Immersive Experience Questionnaire (IEQ) was used to measure the extent to which the participant was dissociated with the real world [4], which can be referred to as game immersion. The questionnaire was successful in obtaining immersion scores for both the conditions, experienced and inexperienced gamers. The IEQ was suitable to measure immersion perceived by the player while dissociated from the real world as it can be effective measurement to all the games [8].

In chapter 5, it was found that the null hypothesis was rejected and further the hypothesis that there is diﬀerence in game immersion between experienced and inexperienced players was supported for both the games Dota 2 and CS:GO. Hence, the diﬀerence in game immersion between the two groups for both the games was calculated in the previous section (Section 5.1.1.2).

The immersion scores obtained for inexperienced players was significantly higher compared to that of the experienced players for both the games Dota 2 and CS:GO. The Cohen’s d effect size for Dota 2 and CS:GO were 0.7423 and 0.8383 respectively, which are both almost equivalent to 0.8. As suggested by Cohen, if the effect size is almost equivalent to 0.8 (d ≈ 0.8), then it can be interpreted as effect size is large [57]. Therefore, the significant difference in game immersion

50 Chapter 6. Discussions and Validity Threats 51 between experienced players and inexperienced players can be considered as large.

The results show that RWD factor was comparatively high in inexperienced players than the experienced players. This research shows a considerable support for the study of C.Jennett for the findings of RWD, i.e., a player will be dissociated from the real world when they find the game more appealing and interesting [8]. The experienced players found the game simple and interesting due to their more knowledge and greater experience during the experiment. At the same time inexperienced players showed more interest for learning and playing the game. Maglio et al. have described that experienced players use the game world more effectively than the novice players [12], but the findings in this study reveal that novice gamers are more dissociated from the real world when compared to the experienced players.

There are indeed diﬀerences between the two measured categories. Inexperi- enced players report higher levels of immersion and being dissociated from the real world than experienced players for both the games. Both the games were found engaging for the players while playing, hence the diﬀerence between the games cannot be generalized.

This study infers that in real world when people play games, they are more immersed if they ﬁnd the games interesting and have the ability to learn engaging games. The level of immersion will be more when a player has more interaction with the virtual characters in the game environment [14]. Further, inexperienced players ﬁnd the game more interesting and they try to learn and interact more with the virtual characters in the game environment. Hence, inexperienced players are more dissociated from the real world than the experienced players. A more general result that can be extracted when considering the analysis is that irrespective of the game played, inexperienced players were more interested to learn and play the game than the experienced players.

6.2 Threats to Validity Validity within empirical studies is defined as a particular conclusion or inference that represents a good approximation to the true conclusion, i.e., whether the methods of research and the subsequent observations provide an acceptable reflection of the truth [82]. The threats to validity in this research are identified and listed below. Chapter 6. Discussions and Validity Threats 52

6.2.1 Internal Validity "Internal validity is the degree to which a study establishes the cause-and-effect relationship between the treatment and the observed outcome" [83]. The threats to internal validity include instrumentation and selection [83]. Instrumentation refers to the measurement device that has been used in the study. Immersive Experience Questionnaire (IEQ) is the instrument used in this study which was properly validated and reviewed before answered by the participants. Selection refers to the manner in which the participants are chosen in the study and are assigned to groups. On the off chance that there are differences in the groups prior to the study taking place, these differences proceed all through the study and may appear as a change in statistical analysis. To mitigate this threat random assignment has to be used for the selection of groups [83], therefore stratified random sampling has been chosen in this study.

6.2.2 External Validity External validity is deﬁned as "The extent to which the results can be generalised and thus applied to other populations" [84]. In this study, it is possible to gener- alise the results to the entire population from which the sample has been chosen, as the population has the same characteristics of the sample. The sample was chosen from the population by random assignment and once selected it was as- sured that the participants participated in the study. Hence, the results were generalisable to the entire population from the sample.

6.2.3 Construct Validity "Construct validity focuses on the relation between the theory behind the experiment and the observations" [85]. The questionnaire was the measurement procedure used to measure immersion which is the construct in this study. The IEQ was chosen from the literature which was ensured valid to be used for measuring immersion to mitigate the threat. Another threat to construct validity can be level of experience, as it is evaluated in years, it cannot be ensured that the same results might be obtained if more precise measurements are taken. Therefore, the experience level chosen was 1year to segregate the experienced players to mitigate the threat.

6.2.4 Conclusion Validity Conclusion validity refers to the extent to which the treatment used in the experimental study can be related to the actual outcome observed [85]. It is typically concerned if there is a statistically significant effect on the outcome [85]. The assumptions made in hypothesis testing about the data when violated might lead Chapter 6. Discussions and Validity Threats 53 to incorrect inferences of the cause-effect relationship. To mitigate this problem, statistical tests are done by analysing the results obtained from the experiment using independent sample T-test and Cohen’s d effect size. Chapter 7 Conclusions and Future Work

7.1 Conclusions In this study, diﬀerence in game immersion between experienced and inexperienced players has been measured using real world dissociation (RWD) factor. The research has been conducted for the two games, MOBA and FPS games, which are Dota2 and Counter Strike:Global Oﬀensive respectively. The players have been divided into two groups based on their experience of game play. The immersion has been measured subjectively by using an IEQ. The questionnaire has been evaluated and validated to compute immersion scores for both the groups.

Independent sample t-test has been conducted to find the significance between the two groups. Further, Cohen’s d has been calculated to find the difference between two groups for both the games. The results show that there is a large difference in game immersion between experienced and inexperienced players for the two games. The difference illustrates that the game immersion when dissociated from the real world is higher in the inexperienced players than the experienced players for both Dota 2 and CS:GO.

An overall conclusion that can be derived from the research is that the immersion levels are relatively low for the experienced players than the inexperienced players while getting dissociated from the real world. A generalized conclusion that can be drawn through the experiment conducted is that regardless of the game played, inexperienced players are more immersed than the experienced players. The conclusion that can be inferred from this research is that the enthusiasm and ability to learn and play new games makes the player more immersive into the game.

7.1.1 Answering Research Questions RQ.1. What are the most commonly used approaches for measuring game immersion? Answer: From the literature review conducted, commonly used approaches to

54 Chapter 7. Conclusions and Future Work 55 measure game immersion have been explored. It also helped to map the existing methods to measure game immersion with RWD factor. The commonly used methods to measure game immersion include both subjective measures and the objective measures. The objective measures include task completion times, eye movements measurements, etc and subjective measures include various questionnaires. As this research focused on game immersion perceived by the player when dissociated from the real world, subjective measure has been chosen to conduct an experiment. Hence the questionnaire chosen that related to real world dissociation factor is Immersive Experience Questionnaire (IEQ). This questionnaire has been further used in the experiment conducted in this research which is answered by the participants to measure the immersion levels perceived by each player.

RQ.2. What is the difference in game immersion, measured by RWD, between experienced and novice players? Answer: Experiment was chosen as the method to find the difference in game immersion between experienced and inexperienced players when measured by RWD. After conducting the experiment, first independent sample T-test was conducted to know if there is significant difference between experienced and inexperienced groups for the two games. The results of hypothesis testing was that there is significant difference between the groups. Further this difference in game immersion between the two groups has been calculated by using Cohen’s d effect size. Therefore the results indicate that the difference in game immersion was large between the two groups for both the games.

7.2 Future Work Game immersion has been one of the developing research field which has been explored through the literature study. Therefore, the findings of this research reveal that the immersion is higher in inexperienced players than the experienced players when measured by RWD factor. In this study only one factor, i.e., RWD factor has been considered, hence in the future work it would be interesting to measure the game immersion using all the immersion factors and compare the level of immersion among those factors for the experienced and inexperienced groups of players. The immersion levels can also be calculated by considering the time as one of the factor, i.e how the time spent on playing games affects the immersion levels of experienced and inexperienced players.

Further in future it would be also interesting to ﬁnd the diﬀerence in game immersion between the two groups objectively by using galvanic measurements or by using eye tracking devices that track the eye movements to measure immersion. References

[1] M.-T. Cheng, H.-C. She, and L. A. Annetta, “Game immersion experience: its hierarchical structure and impact on game-based science learning,” Jour- nal of Computer Assisted Learning, vol. 31, no. 3, pp. 232–253, 2015. [2] D. Johnson, J. Gardner, and P. Sweetser, “Motivations for videogame play: Predictors of time spent playing,” Computers in Human Behavior, vol. 63, pp. 805–812, 2016.

[3] H. Qin, P.-L. P. Rau, and G. Salvendy, “Effects of different scenarios of game difficulty on player immersion,” Interacting with Computers, vol. 22, no. 3, pp. 230–239, 2010.

[4] C. Jennett, A. L. Cox, P. Cairns, S. Dhoparee, A. Epps, T. Tijs, and A. Wal- ton, “Measuring and deﬁning the experience of immersion in games,” Inter- national journal of human-computer studies, vol. 66, no. 9, pp. 641–661, 2008.

[5] C. Jennett, A. L. Cox, and P. Cairns, “Investigating computer game immersion and the component real world dissociation,” in CHI’09 Extended Abstracts on Human Factors in Computing Systems, pp. 3407–3412, ACM, 2009.

[6] J. Huhtala, P. Isokoski, and S. Ovaska, “The usefulness of an immersion questionnaire in game development,” in CHI’12 Extended Abstracts on Human Factors in Computing Systems, pp. 1859–1864, ACM, 2012. [7] E. Brown and P. Cairns, “A grounded investigation of game immersion,” in CHI’04 extended abstracts on Human factors in computing systems, pp. 1297–1300, ACM, 2004.

[8] C. I. Jennett, Is game immersion just another form of selective attention? An empirical investigation of real world dissociation in computer game immersion. PhD thesis, UCL (University College London), 2010. [9] J. Takatalo, J. Häkkinen, J. Komulainen, H. Särkelä, and G. Nyman, “In- volvement and presence in digital gaming,” in Proceedings of the 4th Nordic

56 References 57

conference on Human-computer interaction: changing roles, pp. 393–396, ACM, 2006.

[10] L. Nacke and C. A. Lindley, “Flow and immersion in ﬁrst-person shooters: measuring the player’s gameplay experience,” in Proceedings of the 2008 Conference on Future Play: Research, Play, Share, pp. 81–88, ACM, 2008. [11] P. Cairns, A. Cox, N. Berthouze, C. Jennett, and S. Dhoparee, “Quantifying the experience of immersion in games.,” in CogSci 2006 Workshop: Cognitive Science of Games and Gameplay, 2006. [12] C. I. Jennett, “Investigating real world dissociation in computer game immersion: Manipulating sense of progression and measuring awareness of dis- tracters,” 2009.

[13] J. W. Kim, “Human computer interaction,” Ahn graphics, 2012.

[14] X. Cai, “Principles of human-computer interaction in game design,” in Com- putational Intelligence and Design, 2009. ISCID’09. Second International Symposium on, vol. 2, pp. 92–95, IEEE, 2009. [15] L. Ermi and F. Mäyrä, “Fundamental components of the gameplay experience: Analysing immersion,” Worlds in play: International perspectives on digital games research, vol. 37, p. 2, 2005. [16] E. H. Calvillo-Gámez, P. Cairns, and A. L. Cox, “Assessing the core elements of the gaming experience,” in Game User Experience Evaluation, pp. 37–62, Springer, 2015.

[17] R. I. Clarke, J. H. Lee, J. Jett, and S. Sacchi, “Exploring relationships among video games,” in Proceedings of the 14th ACM/IEEE-CS Joint Conference on Digital Libraries, pp. 481–482, IEEE Press, 2014. [18] H. Martin, “How social context aﬀects levels of immersion: Does physical presence matter?,” Unpublished Master of Science Dissertation, University College London, 2010. [19] P. Barr, J. Noble, and R. Biddle, “Video game values: Human–computer interaction and games,” Interacting with Computers, vol. 19, no. 2, pp. 180– 195, 2007.

[20] M. S. Eastin and R. P. Griﬃths, “Beyond the shooter game examining presence and hostile outcomes among male game players,” Communication Re- search, vol. 33, no. 6, pp. 448–466, 2006. References 58

[21] E. A. Boyle, T. M. Connolly, T. Hainey, and J. M. Boyle, “Engagement in digital entertainment games: A systematic review,” Computers in Human Behavior, vol. 28, no. 3, pp. 771–780, 2012. [22] R. McGloin, K. M. Farrar, M. Krcmar, S. Park, and J. Fishlock, “Mod- eling outcomes of violent video game play: Applying mental models and model matching to explain the relationship between user diﬀerences, game characteristics, enjoyment, and aggressive intentions,” Computers in Human Behavior, vol. 62, pp. 442–451, 2016. [23] M. Grimshaw, J. Charlton, and R. Jagger, “First-person shooters: Immersion and attention,” Eludamos. Journal for Computer Game Culture, vol. 5, no. 1, pp. 29–44, 2011.

[24] P. Cairns, A. Cox, and A. I. Nordin, “Immersion in digital games: review of gaming experience research,” Handbook of digital games, pp. 339–361, 2014. [25] A. Cox, P. Cairns, P. Shah, and M. Carroll, “Not doing but thinking: the role of challenge in the gaming experience,” in Proceedings of the SIGCHI Con- ference on Human Factors in Computing Systems, pp. 79–88, ACM, 2012. [26] D. Arsenault, “Dark waters: Spotlight on immersion,” 2005.

[27] D.-M. Koo, “The moderating role of locus of control on the links between experiential motives and intention to play online games,” Computers in Human Behavior, vol. 25, no. 2, pp. 466–474, 2009. [28] L. Nacke and C. Lindley, “Boredom, immersion, ﬂow: A pilot study investigating player experience,” in IADIS International Conference Gaming 2008: Design for engaging experience and social interaction, IADIS Press, 2008.

[29] M. Berndtsson, J. Hansson, B. Olsson, and B. Lundell, Thesis projects: a guide for students in computer science and information systems. Springer Science & Business Media, 2007.

[30] J. Rowley and F. Slack, “Conducting a literature review,” Management Re- search News, vol. 27, no. 6, pp. 31–39, 2004.

[31] J. J. Randolph, “A guide to writing the dissertation literature review,” Prac- tical Assessment, Research & Evaluation, vol. 14, no. 13, pp. 1–13, 2009. [32] C. Wohlin, M. Höst, and K. Henningsson, “Empirical research methods in software engineering,” in Empirical methods and studies in software engineering, pp. 7–23, Springer, 2003.

[33] J. Zobel, Writing for computer science, vol. 8. Springer, 2004. References 59

[34] C. Wohlin, “Guidelines for snowballing in systematic literature studies and a replication in software engineering,” in Proceedings of the 18th International Conference on Evaluation and Assessment in Software Engineering, p. 38, ACM, 2014.

[35] M. Hudson and P. Cairns, “The eﬀects of winning and losing on social presence in team-based digital games,” Computers in Human Behavior, vol. 60, pp. 1–12, 2016.

[36] G. Christou, “A comparison between experienced and inexperienced video game players’ perceptions,” Human-centric computing and information sciences, vol. 3, no. 1, p. 1, 2013. [37] J. Jansz and M. Tanis, “Appeal of playing online ﬁrst person shooter games,” CyberPsychology & Behavior, vol. 10, no. 1, pp. 133–136, 2007. [38] J. Rambusch, P. Jakobsson, and D. Pargman, “Exploring e-sports: A case study of game play in counter-strike,” pp. 157–164, DiGRA, 2007.

[39] “Steam:Game and Player Statistics.” http://store.steampowered.com/ stats/. [Online; accessed 02-October-2016]. [40] D. Shen, “The people’s history of steam,” 2015.

[41] T. W. Windleharth, J. Jett, M. Schmalz, and J. H. Lee, “Full steam ahead: A conceptual analysis of user-supplied tags on steam,” Cataloging & Classi- fication Quarterly, pp. 1–24, 2016. [42] N. Pobiedina, J. Neidhardt, M. d. C. Calatrava Moreno, and H. Werthner, “Ranking factors of team success,” in Proceedings of the 22nd International Conference on World Wide Web, pp. 1185–1194, ACM, 2013. [43] P. Yang, B. Harrison, and D. L. Roberts, “Identifying patterns in combat that are predictive of success in moba games,” Proceedings of Foundations of Digital Games, 2014. [44] A. Drachen, M. Yancey, J. Maguire, D. Chu, I. Y. Wang, T. Mahlmann, M. Schubert, and D. Klabajan, “Skill-based differences in spatio-temporal team behaviour in defence of the ancients 2 (dota 2),” in Games media entertainment (GEM), 2014 IEEE, pp. 1–8, IEEE, 2014. [45] E. G. Olshefski, “Game-changing event definition and detection in an esports corpus,” in Proceedings of the 3rd Workshop on EVENTS at the NAACL- HLT, pp. 77–81, 2015.

[46] J. Isotalo, “Basics of statistics,” Finland: University of Tampere, 2001. References 60

[47] A. Saifuddin, “Methods in survey sampling biostat 140.640–,” John Hopkins University, Bloomberg, 2009.

[48] N. Salkind, Encyclopedia of Measurement and Statistics. 2455 Teller Road,Â Thousand OaksÂ CaliforniaÂ 91320Â United States of America: Sage Publications, Inc., 2007.

[49] “Stratiﬁed random sampling | LÃŠrd Dissertation.” http://dissertation. laerd.com/stratified-random-sampling.php. [Online; accessed 25-July- 2016].

[50] “How to Get a Stratiﬁed Random Sample in Statistics.” http://www. statisticshowto.com/stratified-random-sample/. [Online; accessed 31-July-2016].

[51] J. H. Brockmyer, C. M. Fox, K. A. Curtiss, E. McBroom, K. M. Burkhart, and J. N. Pidruzny, “The development of the game engagement questionnaire: A measure of engagement in video game-playing,” Journal of Experimental Social Psychology, vol. 45, no. 4, pp. 624–634, 2009. [52] A. Delorme, “San diego california, ca92093-0961, la jolla, usa. email: arno@ salk. edu.,”

[53] T. K. Kim, “T test as a parametric statistic,” Korean Journal of Anesthesi- ology, vol. 68, no. 6, p. 540, 2015. [54] E. Hedberg and S. Ayers, “The power of a paired t-test with a covariate,” Social science research, vol. 50, pp. 277–291, 2015.

[55] A. Lehman, JMP for basic univariate and multivariate statistics: a step-by- step guide. SAS Institute, 2005.

[56] A. Agresti and B. Finlay, “Introduction to multivariate relationships,” Sta- tistical methods for the social sciences, Ed, vol. 3, pp. 356–372, 1997. [57] D. Lakens, “Calculating and reporting eﬀect sizes to facilitate cumulative science: a practical primer for t-tests and anovas,” Frontiers in psychology, vol. 4, p. 863, 2013.

[58] W. Thalheimer and S. Cook, “How to calculate eﬀect sizes from published research: A simpliﬁed methodology,” Work-Learning Research, pp. 1–9, 2002. [59] J. D. Smith and T. Graham, “Use of eye movements for video game control,” in Proceedings of the 2006 ACM SIGCHI international conference on Advances in computer entertainment technology, p. 20, ACM, 2006. References 61

[60] J. D. Ivory and S. Kalyanaraman, “The eﬀects of technological advancement and violent content in video games on players’ feelings of presence, involvement, physiological arousal, and aggression,” Journal of Communication, vol. 57, no. 3, pp. 532–555, 2007.

[61] M. V. Sanchez-Vives and M. Slater, “From presence to consciousness through virtual reality,” Nature Reviews Neuroscience, vol. 6, no. 4, pp. 332–339, 2005.

[62] J. Lessiter, J. Freeman, E. Keogh, and J. Davidoﬀ, “A cross-media presence questionnaire: The itc-sense of presence inventory,” Presence, vol. 10, no. 3, pp. 282–297, 2001.

[63] N. Ravaja, M. Salminen, J. Holopainen, T. Saari, J. Laarni, and A. Järvinen, “Emotional response patterns and sense of presence during video games: Po- tential criterion variables for game design,” in Proceedings of the third Nordic conference on Human-computer interaction, pp. 339–347, ACM, 2004. [64] B. G. Witmer and M. J. Singer, “Measuring presence in virtual environments: A presence questionnaire,” Presence: Teleoperators and virtual environments, vol. 7, no. 3, pp. 225–240, 1998. [65] P. Vorderer, W. Wirth, T. Saari, F. Gouveia, F. Biocca, L. Jäncke, S. Böck- ing, H. Schramm, A. Gysbers, T. Hartmann, et al., “Development of the mec spatial presence questionnaire (mec-spq),” Report to the European Commu- nity, Project Presence, MEC, 2004. [66] A. Sacau, J. Laarni, N. Ravaja, and T. Hartmann, “The impact of person- ality factors on the experience of spatial presence,” in The 8th International Workshop on Presence (Presence 2005), pp. 143–151, 2005. [67] P. Sweetser and P. Wyeth, “Gameﬂow: a model for evaluating player enjoyment in games,” Computers in Entertainment (CIE), vol. 3, no. 3, pp. 3–3, 2005.

[68] F.-L. Fu, R.-C. Su, and S.-C. Yu, “Egameﬂow: A scale to measure learners’ enjoyment of e-learning games,” Computers & Education, vol. 52, no. 1, pp. 101–112, 2009.

[69] R. Holt and J. Mitterer, “Examining video game immersion as a ﬂow state,” 108th Annual Psychological Association, Washington, DC, 2000. [70] H. Qin, P.-L. Patrick Rau, and G. Salvendy, “Measuring player immersion in the computer game narrative,” Intl. Journal of Human–Computer Inter- action, vol. 25, no. 2, pp. 107–133, 2009. References 62

[71] C.-L. Hsu and H.-P. Lu, “Why do people play on-line games? an extended tam with social inﬂuences and ﬂow experience,” Information & management, vol. 41, no. 7, pp. 853–868, 2004.

[72] Y. A. De Kort, W. A. IJsselsteijn, and K. Poels, “Digital games as social presence technology: Development of the social presence in gaming questionnaire (spgq),” Proceedings of PRESENCE, vol. 195203, 2007. [73] J. P. Charlton, “A factor-analytic investigation of computer ‘addiction’and engagement,” British journal of psychology, vol. 93, no. 3, pp. 329–344, 2002. [74] J. P. Charlton and I. D. Danforth, “Distinguishing addiction and high engagement in the context of online game playing,” Computers in Human Behavior, vol. 23, no. 3, pp. 1531–1548, 2007.

[75] K. L. Norman, “Geq (game engagement/experience questionnaire): a review of two papers,” Interacting with Computers, vol. 25, no. 4, pp. 278–283, 2013. [76] C. Lindley, L. Nacke, and C. Sennersten, “Dissecting play–investigating the cognitive and emotional motivations and aﬀects of computer gameplay,” in 13th International Conference on Computer Games (CGames 2008), UNIV WOLVERHAMPTON, 2008.

[77] J. Takatalo, J. Häkkinen, J. Kaistinen, and G. Nyman, “Measuring user experience in digital gaming: Theoretical and methodological issues,” in Elec- tronic Imaging 2007, pp. 649402–649402, International Society for Optics and Photonics, 2007.

[78] “How to reverse score questions.” http://www2.yorksj.ac.uk/pdf/ Reverse.pdf. [Online; accessed 28-August-2016]. [79] K. H. Yim, F. S. Nahm, K. A. Han, and S. Y. Park, “Analysis of statistical methods and errors in the articles published in the korean journal of pain,” The Korean journal of pain, vol. 23, no. 1, pp. 35–41, 2010.

[80] D. W. Cunningham and C. Wallraven, Experimental design: From user studies to psychophysics. CRC Press, 2011.

[81] H. Tanya, “Parametric and nonparametric: Demystifying the terms,” Mayo Clinic Department of Health Science, Tutorial. [82] B. E. Roe and D. R. Just, “Internal and external validity in economics research: Tradeoffs between experiments, field experiments, natural experiments, and field data,” American Journal of Agricultural Economics, vol. 91, no. 5, pp. 1266–1271, 2009. References 63

[83] M. K. Slack and J. R. Draugalis, “Establishing the internal and external validity of experimental studies,” American Journal of Health System Phar- macy, vol. 58, no. 22, pp. 2173–2184, 2001. [84] G. Winter, “A comparative discussion of the notion of’validity’in qualitative and quantitative research,” The qualitative report, vol. 4, no. 3, pp. 1–14, 2000.

[85] R. Feldt and A. Magazinius, “Validity threats in empirical software engineering research-an initial survey.,” in SEKE, pp. 374–379, 2010. Appendix A Game Immersion Questionnaire

64 Appendix A. Game Immersion Questionnaire 65 Appendix B Snapshot of Steam Website

Figure B.1: Snapshot of Steam Website for top games

66 Appendix C Immersion scores of Dota 2

C.1 Results of experienced players

No. Q1 Q2 Q3 Q4 Q5 Q6 Q7 Q8 Q9 Sum

1. 4 1 2 4 4 4 2 4 5 30

2. 3 4 3 3 3 3 3 3 4 29

3. 1 5 3 1 1 5 5 5 5 31

4. 1 5 2 3 3 5 3 5 5 32

5. 2 5 3 4 1 5 5 5 2 32

6. 1 1 1 1 1 1 1 1 5 13

7. 2 3 3 2 4 5 2 2 2 25

67 Appendix C. Immersion scores of Dota 2 68

8. 1 2 3 2 2 1 3 3 4 21

9. 3 5 2 4 5 2 2 4 5 32

10. 2 2 2 2 2 5 3 4 5 27

11. 4 4 3 3 3 4 2 3 3 29

12. 3 5 1 3 1 5 2 3 5 28

13. 1 3 1 3 5 5 4 3 4 29

14. 5 4 1 5 4 4 2 3 4 32

15. 5 3 1 4 4 5 2 3 4 31

16. 1 5 1 4 5 5 5 5 1 32

17. 1 1 1 1 1 5 1 3 5 19

18. 3 5 1 1 2 5 5 1 5 28 Appendix C. Immersion scores of Dota 2 69

19. 2 4 4 2 1 4 5 4 4 30

20. 1 1 1 1 1 5 1 1 5 17

21. 3 2 2 2 2 2 2 3 2 20

22. 3 1 3 1 2 3 1 2 5 21

Table C.1: Immersion scores of experienced players of Dota2 game

C.2 Results of inexperienced players

No. Q1 Q2 Q3 Q4 Q5 Q6 Q7 Q8 Q9 Sum

1. 4 4 2 4 4 3 5 3 4 33

2. 3 1 2 2 2 5 2 3 4 24

3. 3 5 3 3 3 3 3 3 3 29 Appendix C. Immersion scores of Dota 2 70

4. 2 3 4 1 2 5 3 3 4 27

5. 4 4 2 3 4 4 4 4 5 34

6. 1 4 2 1 4 5 5 4 3 29

7. 3 3 4 3 4 5 4 4 4 34

8. 3 2 3 4 3 5 4 4 4 32

9. 4 4 2 3 4 5 5 5 5 37

10. 4 4 2 3 4 5 5 5 5 37

11. 3 4 2 5 2 3 4 5 2 30

12. 2 1 2 3 2 3 1 4 5 23

13. 4 4 2 2 5 5 1 3 4 30

14. 2 5 4 2 3 5 4 5 3 33 Appendix C. Immersion scores of Dota 2 71

15. 2 5 3 5 2 2 5 3 2 29

16. 3 4 2 3 3 4 4 2 4 29

17. 1 5 1 4 3 2 4 3 4 27

18. 5 4 4 2 3 1 5 3 2 29

Table C.2: Immersion scores of inexperienced players of Dota2 game Appendix D Immersion Scores of CS:GO

D.1 Results of experienced players

No. Q1 Q2 Q3 Q4 Q5 Q6 Q7 Q8 Q9 Sum

1. 4 3 4 2 2 4 3 2 2 26

2. 2 3 2 2 3 3 3 4 4 26

3. 4 4 4 2 2 5 3 2 5 31

4. 2 5 3 3 3 2 2 2 4 26

5. 2 5 3 1 1 3 1 3 4 23

6. 3 3 4 1 2 5 3 4 5 30

7. 2 4 5 4 4 5 1 1 4 30

72 Appendix D. Immersion Scores of CS:GO 73

8. 2 2 3 3 2 3 2 3 5 25

9. 1 3 2 3 4 4 5 4 3 29

10. 3 3 2 3 4 4 3 2 3 27

11. 3 2 3 3 2 2 1 4 2 22

12. 4 4 2 2 4 4 4 4 5 33

13. 2 1 5 1 3 5 2 1 5 25

14. 3 3 3 3 3 4 3 3 3 28

15. 4 3 3 2 2 5 3 3 4 29

16. 3 2 3 4 3 2 3 3 3 26

Table D.1: Immersion scores of experienced players of CS:GO Appendix D. Immersion Scores of CS:GO 74

D.2 Results of inexperienced players

No. Q1 Q2 Q3 Q4 Q5 Q6 Q7 Q8 Q9 Sum

1. 2 3 4 3 2 5 4 4 4 31

2. 4 4 2 4 4 4 4 4 4 34

3. 4 4 2 4 4 5 5 4 4 36

4. 2 5 2 1 2 5 5 3 4 29

5. 4 3 1 4 4 5 4 4 5 34

6. 4 4 1 4 4 5 5 4 5 36

7. 4 3 4 3 3 5 3 2 3 30

8. 4 1 4 1 3 4 4 2 5 28 Appendix D. Immersion Scores of CS:GO 75

9. 4 3 4 2 4 2 2 3 4 28

10. 3 4 3 4 2 2 5 4 1 28

11. 1 3 3 1 2 5 3 4 5 27

12. 3 4 1 4 4 4 3 4 2 29

13. 2 3 3 2 1 2 3 3 4 23

14. 5 1 2 4 3 4 4 3 3 29

Table D.2: Immersion scores of inexperienced players of CS:GO