Daniel M. Shafer* Controller Required? The Impact Corey P. Carbonara Baylor University of Natural Mapping on Department of Communication Interactivity, Realism, Presence, Waco, TX 76798 and Enjoyment in Motion-Based Lucy Popova Video Games University of California at San Francisco San Francisco, CA

Abstract

In three experiments with U.S. undergraduates, effects of three levels of naturally mapped control interfaces were compared on a player’s sense of presence, interactiv- ity, realism, and enjoyment in video games. The three levels of naturally mapped con- trol interfaces were: kinesic natural mapping (using the player’s body as a game con- troller), incomplete tangible mapping (using a controller in a way similar to a real object), and realistic tangible mapping (using a controller or an object that directly relates to the real-life activity the game simulates). The results show that levels of interactivity, realism, spatial presence, and enjoyment were consistent across all conditions. However, when performing activities such as table tennis or lightsaber dueling with objects in-hand (incomplete tangible or realistic tangible conditions), perceived reality was a more important predictor of spatial presence. When perform- ing the same activities with empty hands, interactivity emerged as the more important direct predictor of spatial presence. Control interface, therefore, matters greatly to the route by which cognitive processing of games takes place and how enjoyment is produced.

1 Introduction

Over the past four to five years, academic study of controller naturalness and motion-based video games has become increasingly rich. Grounded in and expanding upon traditional entertainment theories, studies in this vein offer increasingly detailed explanations of the processes behind enjoyment of games played with the newest gaming technology. The present study seeks to add to the scholarly conversation on the impact of naturally mapped controllers on players’ perceptions of interactivity, reality, spa- tial presence, and, above all, enjoyment. This study grows from the psychologi- cal theory of play, which Vorderer (2001) first applied to the experience of entertaining media. It also builds upon previous empirical work that utilized early motion control devices (e.g., Wii) investigating the impact of natural con- troller mapping on perceived interactivity, perceived reality, spatial presence, and enjoyment in the context of the latest motion-based control systems, such Presence, Vol. 23, No. 3, Summer 2014, 267–286 doi:10.1162/PRES_a_00193 ª 2014 by the Massachusetts Institute of Technology *Correspondence to [email protected].

Shafer et al. 267

Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/PRES_a_00193 by guest on 23 September 2021 268 PRESENCE: VOLUME 23, NUMBER 3

as the Move for the PlayStation 3 (PS3) and the Kinect several studies, e.g., Klimmt & Vorderer; Steuer, 1992; add-on for the Xbox 360. Wirth et al.). By extending the arguments of Vorderer’s applica- 2 Theoretical Background and Literature tion of the psychological theory of play, it becomes Review clear that perceptions of realism (i.e., perceived reality) also impact players’ feelings of spatial presence. Playing video games has been shown to be an Busselle and Bilandzic (2008) explain the concepts entertaining and enjoyable activity according to much of internal and external realism. Internal realism is the academic research and documented success of the video idea that the fictional world has a consistent narrative game industry. Part of the success of video games as an structure that is not in violation of its own laws. Exter- entertainment medium can no doubt be attributed to nal realism simply means that the fictional world has a the fact that they create alternate realities for their users degree of similarity to and consistency with the real to explore (Vorderer, 2001; Klimmt & Vorderer, 2003). world; so that when obviously unrealistic elements are Klimmt and Vorderer relate the outcomes experienced introduced, they can be accepted without dislodging by users to the psychological theory of play, which the viewer from the experience. With enough internally explains many of the criteria needed for the production and externally realistic elements, it is not necessary for of enjoyment via video games. the viewer (or, in the case of the present study, the First, play is interactive; the player is in control and has player) to make active judgments about perceived real- power to influence the outcome of the game and the vir- ity(Busselle&Bilandzic). Therefore, if there are tual environment (Grodal, 2000; Klimmt, Hartmann, & enough realistic elements in the game, and interaction Frey, 2007). Furthermore, through action, the player is on the part of players is believable and effective, players impacting an alternate reality. To the extent that the can become immersed more fully into the virtual player feels effective in this activity (i.e., how interactive world, fostering a sense of involvement (see McMahan, the game is perceived to be), the more realistic the alter- 2003). nate reality—or, in the present study’s case, the virtual In addition to arguing that interactive media enhance world—will seem. In addition to the relationship spatial presence and perceptions of realism, Klimmt and between perceived interactivity and perceived reality, Vorderer (2003) suggest that spatial presence impacts Vorderer’s application of the psychological theory of play enjoyment of the mediated experience. The authors note points out another crucial relationship. The more inter- that past theoretical work has argued, and empirical active a player perceives the game, the more likely the studies have demonstrated, that affective reactions such player is to become deeply immersed in the game world as delight, joy, and fascination are ‘‘closely connected to, because he or she wants the experience to continue or even identical with the experience of presence’’ (Klimmt & Vorderer, 2003). This process leads to a (Klimmt & Vorderer, p. 356). Therefore, theory sug- greater sense of presence. Presence, or more specifically gests spatial presence should positively impact enjoy- spatial presence, is the sense of being located within a ment. All of the theoretical arguments offered here have virtual world (Wirth et al., 2007). The player’s sense of been confirmed in several empirical studies (Klimmt self is enveloped by the game world such that it, not the et al., 2007; Shafer, Carbonara, & Popova, 2011; Skal- real world, takes up the primary position in conscious- ski, Tamborini, Shelton, Buncher, & Lindmark, 2011; ness. This process results in the game world becoming Wirth et al., 2007). Other theoretical perspectives that what Wirth and colleagues call the Primary Egocentric lend insight to the processes surrounding experiential Reference Frame (PERF; Wirth et al.). Video games, reactions to video games are the mental models approach because of their interactive nature, are especially effective (Brewer, 1987) and the common coding framework at creating or promoting a sense of spatial presence, de- (Chandrasekharan, Mazalek, Nitsche, Chen, & Ranjan, spite the lack of full-body tactile feedback (as noted in 2010).

Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/PRES_a_00193 by guest on 23 September 2021 Shafer et al. 269

2.1 Mental Models and Common Coding physiologically to perform the action. Essentially, the mind simulates the observed action. When this happens, The mental models approach provides insight as to overt execution is inhibited until the appropriate time how individuals process and interpret the information for action is apparent (Chandrasekharan et al.). they receive from video games. Mental models or sche- Central to the argument for common coding is the mas are constructed cognitive representations of real or notion that people are considered to be action-based sys- mediated people, objects, events, situations, and so on. tems. Both sensory and cognitive systems occupy a com- When judging the reality of a media message or a video mon coding neural network. This network encodes both game, people compare the object or the experience to a action and perception; either can automatically activate preexisting mental model for the object or experience, the other in an associative priming state where the activa- that is, to their expectations of what this object or expe- tion of one element (e.g., observation) triggers the other rience should be. The closer the match between the two elements (e.g., imagination, action; Chandrase- expectations (mental model) and the object or experi- kharan et al., 2010). The synergy between elements is ence, the greater the perceived reality. disrupted when one is observing a noncongruent action Along similar lines, cognitive psychologists (Chandra- (Chandrasekharan et al.). In terms of video games, diffi- sekharan, Mazalek, Nitsche, Chen, & Ranjan, 2010) culty would arise when on-screen actions do not closely have argued that common coding exists between an match the actions the player is expected to take (e.g., action and a cognitive representation of an action. The pressing a key on a keyboard to turn a steering wheel in a concept of common coding is linked to the ‘‘ideomotor driving game). Therefore, motion-based control sys- principle’’ (James, 1890; Chandrasekharan et al.). The tems, or Natural User Interfaces (NUIs), as argued ideomotor principle is described by James (1890): ‘‘Ev- below, should offer greater synergy thanks to common ery representation of a movement awakens in some coding than standard control systems. Furthermore, degree the actual movement which is its object...’’ (p. common coding arguments suggest that the more natu- 526). Essentially, what James is arguing is that watching ral or representative of reality the NUI is, the more or imagining a movement triggers the same responses in interactive and the more like reality participants should the brain as if the movement were actually being per- find it. formed. The ideomotor principle serves as a model of McGloin, Farrar, and Krcmar (2011) argued that cognition that explains how perception, execution, and more interactive motion capturing controllers with imagination of movements are all interpreted by the NUIs allow the player to call upon the well-developed brain in relatively the same way. Because of this, doing real-world schemas (or common codes). For many video an activity, watching an activity, and imagining an activ- games, such as sports or driving, players already have ity share a common coding pattern in the brain. Simply highly developed schemas and it is easier to interact with imagining a movement implicitly activates a person’s the virtual environment using existing real-world sche- motor functions, guiding how we perceive and imagine mas. For example, in sports video games, by using NUIs, other movements—even those of an avatar in a video players can use the same physical movements they use in game. the real world. If the natural controllers allow for a closer Therefore, the ideomotor principle can be explained match to these schemas, people will feel that this experi- by a common coding in the brain that connects a per- ence (playing a video game with a natural controller) is son’s movement, observation of movements as percep- more real than playing a video game with a traditional tual thoughts, and imagining movements themselves, as button and joystick controller. Thus, more interactive an implicit function of both motor and perceptual repre- NUIs result in higher perceived reality. sentations (Chandrasekharan et al., 2010). Upon observ- The theoretical explanation of the link between inter- ing an action, the human brain behaves as if the action activity and perceived reality based on mental models can were being replicated in the body, readying the actor also be applied to the link between interactivity and spa-

Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/PRES_a_00193 by guest on 23 September 2021 270 PRESENCE: VOLUME 23, NUMBER 3

identities merge with those of the characters. All this allows a researcher to argue that interactivity is an explaining factor in what makes video games entertaining. In the case of video games, interactivity affects players’ perceptions and resultant states (such as presence or Figure 1. Hypothesized model. transportation), which are inherently enjoyable for the player. Thus, players’ perceptions mediate the relation- ship between interactivity and enjoyment. As already tial presence (see Skalski et al., 2011). Greater potential hypothesized, differences should be evident between dif- for interactivity, that is, fewer barriers between the player ferently mapped controller types; however, player per- and the virtual world, leads to greater spatial presence. ception of interactivity is the measurable variable we wish An object (controller) in the hands feels less natural and to investigate. is harder to ignore than empty hands. However, this is Many researchers place interactivity in relation to the not the case when mental models demand an object in users’ perspective (e.g., Heeter, 2000; Downes & the hands, such as when a person wields a sword or uses McMillian, 2000; Vorderer, 2000; Chen & Raney, a paddle to strike a ball. In this case, a greater perception 2009). In that context, interactivity can impact a number of nonmediation and perceived reality would be achieved of player perceptions. For example, almost all the user with an object instead of empty hands. However, in prerequisites for enjoyment listed in Vorderer, Klimmt, comparison between empty hands or wand devices and Ritterfeld (2004)—suspension of disbelief, emo- (NUIs) and button-and-joystick style controllers, more tional connection with character, interaction with char- interactive natural-mapping controllers present fewer acters, and users’ sense of being there—can be affected barriers to the feeling of being there than traditional by interactivity. However, most of these, as well as many controllers. of the determinants of entertainment listed above, can The relationships described above are graphically pre- be described as part of one of the two concepts central sented in the hypothesized model (Figure 1) and sup- to video game experience—presence and perceived real- ported by the empirical evidence described in the next ity. Presence has been defined as the perception of virtual section. objects and environments ‘‘as actual objects in either sensory or non-sensory ways’’ (Lee, 2004, p. 44). Pres- ence is a multidimensional construct (Tamborini & Skal- 2.2 Recent Empirical Findings ski, 2006), composed of dimensions of social presence, Within a burgeoning field of video game research, spatial presence, and self-presence. In our research, we many factors have been named as determinants of video focus on spatial presence, defined as the sense of being game enjoyment, such as competition (Vorderer, Hart- there, the sense of physical immersion in the virtual envi- mann, & Klimmt, 2003), effectance and control ronment. (Klimmt et al., 2007), transportation into the narrative Perceived reality (sometimes referred as ‘‘perceived re- world (Green, Brock, & Kaufman, 2004), and identifica- alism’’) is the perception of the degree of correspon- tion with the player character (Hefner, Klimmt, & dence between the media representation and the real- Vorderer, 2007), among others. For all of these factors, world content (Hall, 2003). Like presence, perceived interactivity is either a precursor or an integral part; for reality is also a multidimensional construct, composed of example, a player cannot compete or control the game dimensions of magic window, typicality, identity, utility, without interactivity afforded by technology, and inter- perceptual fidelity, and virtual experience (see Popova, activity has been shown to be a strong predictor of trans- 2010; Shafer et al., 2011). Magic window, as the belief portation and identification, to the point where players’ in the literal reality of the media content, is a dimension

Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/PRES_a_00193 by guest on 23 September 2021 Shafer et al. 271

peripheral to video game research because, presumably, els of presence. Shafer and colleagues (2011) demon- most players have a firm belief that the game does not strated a significant positive relationship between provide a ‘‘magic window’’ into a real world in the way interactivity and spatial presence in two experimental news does, for example. The rest of the dimensions of studies. It is expected, then, that perceived interactivity perceived reality are highly relevant to video game expe- should have a general positive impact on spatial rience. Typicality is the perceived match between gaming presence. experience or specific content details and characters and players’ mental models for such experience or content. 2.2.3 Perceived Reality and Spatial Identity is the feeling of closeness to the characters in Presence. In the model of narrative comprehension the game. Utility refers to how applicable information and engagement, Busselle and Bilandzic (2008) argue and skills learned in a video game are to real life. Percep- that high perceived reality enables viewers or players to tual fidelity is sensory realism, that is, how real the visual, ‘‘shift the center of their experience from the actual audio, and tactile stimuli feel. Finally, virtual experience world into the fictional world’’ and experience the world is the perception of interaction and sense of control in of the game ‘‘from the inside’’ (p. 272). Thus, higher the game world. Recent study findings demonstrate dif- perceived reality of video games, defined as a close match ferential roles of various dimensions in video game between the game content and players’ expectations, research (Shafer et al.). leads to a greater sense of being there. Empirical evi- dence demonstrates the effects of various dimensions of 2.2.1 Interactivity and Perceived perceived reality on spatial presence. The perceptual fi- Reality. Experimental studies that compared various delity dimension (high-definition images and quality levels of interactivity in the form of different controllers sound) has been shown to induce greater presence using samples of casual players found a causal link (Bracken & Skalski, 2009; Ivory & Kalyanaraman, 2007; between interactivity and perceived reality (McGloin Skalski & Whitbred, 2010; Jeong et al., 2011). Shafer et al., 2011; Shafer et al., 2011; Krcmar, Farrar, & et al. (2011) found that perceived reality was a strong McGloin, 2011; Jeong, Bohil, & Biocca, 2011). Other predictor of spatial presence and the dimensions of util- studies found that interactivity of Artificial Intelligence ity, perceptual fidelity, and identity were the significant (AI) and virtual reality systems was positively related to predictors of spatial presence. Given this evidence, it is perceived reality (Laird & van Lent, 2001; Drettakis, likely that perceived reality will generally have a positive Roussou, Reche, & Tsingos, 2007). Therefore, we effect on spatial presence. expect that perceived interactivity should generally have a positive impact upon perceptions of realism (perceived 2.2.4 Spatial Presence and Enjoyment. One reality). can argue that spatial presence, unlike perceived reality, which is not inherently pleasurable, is an essentially 2.2.2 Interactivity and Spatial enjoyable state in and of itself (Green et al., 2004). Presence. Several studies have found that greater inter- Thriving movie, book, television, and video game indus- activity leads to greater spatial presence. McGloin and tries—all of which provide customers an opportunity to colleagues (2011) manipulated interactivity in the form escape to alternative worlds—are a testimony to that. of controller type much as the present study intends to Video game players routinely maximize presence by do. They found that more naturally mapped controllers removing distracting obstacles in their environments and (i.e., those with more interactivity potential; see Vor- updating gaming hardware components (Nunez & derer, 2000) induced a greater sense of presence. Persky Blake, 2006). Experimental studies have shown that and Blaskovich (2008) demonstrated that playing on a male players report higher enjoyment when using soft- more interactive system (an Immersive Virtual Environ- ware with spatial presence cues (Horvath & Lombard, ment Technology System, IVET) resulted in higher lev- 2010). McGloin and colleagues (2011) found that play-

Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/PRES_a_00193 by guest on 23 September 2021 272 PRESENCE: VOLUME 23, NUMBER 3

ers of tennis games who experienced higher levels of spa- with Kinect. They found that higher technological inter- tial presence also reported higher enjoyment. Shafer and activity (more points of interaction with the player) was colleagues (2011) found that spatial presence could related to greater perceived reality, spatial presence, and explain nearly 50% of the variance in enjoyment both in enjoyment of video games. Perceived reality predicted a study that compared traditional controllers to more spatial presence, which, in turn, positively predicted naturally mapped motion controllers, and in a second video game enjoyment (Shafer et al.). In that study, the study that compared the three motion-based control sys- Kinect system induced higher spatial presence, perceived tems (Wii, Move, and Kinect) to one another. It is reality, and enjoyment than Wii or Move, indicating that expected, then, that spatial presence will significantly the full body scanning technology and the absence of a predict enjoyment in all studies, regardless of controller controller was more immersive and fun for players, and, type. ostensibly, felt more natural. However, controller natu- ralness or perceived controller naturalness were not ex- plicitly measured. 2.3 Varying Effects of Naturally There are, conceivably, some activities for which Mapped Control Interfaces holding a controller or some other object while playing Several studies have demonstrated a link between could be perceived as more natural than playing empty- control or interface method (e.g., natural mapping) and handed. For instance, in the sport of table tennis, real- enjoyment. A study using a Madden NFL game from life players hold paddles in their hands, and swing them 2007–2008 found that PlayStation 2 players reported attheball.WhileplayingboththeWiiandMovever- more control and enjoyment than Wii players, despite sions of table tennis, players hold a controller. Holding predicting the opposite relationship (Limperos, Schmier- the controller is reminiscent of holding the grip of a ta- bach, Kegerise, & Dardis, 2011). Part of the explanation ble tennis paddle. This notion is reinforced by the for this unexpected finding was that the Wii was perhaps image of an avatar on screen holding a paddle, and too new, and players were unaccustomed to the control responding as the player swings the controller. This scheme it offered; whereas the traditional PS2 controls, form of mapping, when a player holds a controller and while not intuitively or naturally mapped, were more fa- uses or moves it in a way that is similar to real life, is miliar to players and therefore more enjoyable. The known as incomplete tangible mapping (Skalski et al., approach scholars have taken since that study, however, 2011). Systems such as the Wii and Sony’s Move has been to test more modern motion control devices employ this mapping approach. Systems like Kinect, that offer more points of interaction (i.e., Kinect) or however, employ what is known as kinesic natural map- more precise control (i.e., Move; Shafer et al., 2011); or ping (Skalski et al.), using the player’s body as a con- to use stimulus material that more closely simulates an troller, without holding anything. Skalski and col- activity for which there is one definite activity that can be leagues posit that kinesic natural mapping is less natural naturally mapped to a control device such as swinging a than incomplete tangible mapping. Higher in the natu- club or racket, or driving (McGloin et al., 2011; Shafer ralness hierarchy is realistic tangible natural mapping, et al., 2011; Skalski et al., 2011). Skalski and colleagues which incorporates a control device or object that demonstrated that controller naturalness does have posi- directly relates to the real-life activity the game simu- tive impact on both spatial presence and enjoyment of lates. Examples would be using a steering wheel con- the game. troller for a racing game, or using a gun-shaped con- Shafer and colleagues (2011) reported similar find- troller for a shooter game. ings. In two experiments they had college undergradu- The present article, in three studies, will test the three ates play golf, driving, or fighting video games with tra- levels of natural mapping in order to investigate possible ditional controllers (PlayStation 3 or Xbox 360), the differences between them on the experience-related vari- Nintendo Wii, the PlayStation Move, or the Xbox 360 ables of perceived interactivity, perceived reality, spatial

Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/PRES_a_00193 by guest on 23 September 2021 Shafer et al. 273

presence, and enjoyment. Based on the evidence given they play, it is likely that there will be differences not so far, it is predicted that: only in the magnitude of the variables themselves, but also in the strength of the links between the variables. H1: Each model will follow a predictable pattern of To explore that possibility, the following prediction is effects such that perceived interactivity will have a made: positive impact on both perceived reality and spatial presence; perceived reality will have a positive H3: There will be significant differences between impact on spatial presence, and spatial presence will models of kinesic, incomplete tangible, and realistic have a positive impact on enjoyment. tangible mapping conditions in the relationships between perceived interactivity, perceived reality, These expected relationships are shown in the spatial presence, and enjoyment. hypothesized model (see Figure 1). According to the natural mapping typology defined by Skalski and colleagues (2011), there should be signifi- 3 Study 1 Method cant differences in perception of the game between play- ers who use various levels of natural mapping. For 3.1 Participants instance, the incomplete tangible mapping condition Data were collected from 46 student participants should be the least natural, and should therefore feel recruited from film, speech, and communication courses generally less interactive and perhaps less realistic. How- at a midsized research university in the southern-central ever, it can certainly be argued that for some games, United States. Students were offered extra credit for par- holding an object such as a controller that resembles the ticipation, along with other equally valuable extra credit object used in the real-life playing of that game may be activities. The average age in the sample was 21 years more interactive and more realistic than playing with (SD ¼ 2.14 years). Women (n ¼ 25, 54%) slightly out- empty hands. Therefore, it is predicted that: numbered men (n ¼ 21, 46%). H2: Players will generally judge the most natural form of playing a game highest according to whether the game is played with or without an object in real life. 3.2 Stimulus Materials H2a: In study 1, which utilizes boxing games, players Two games were used as stimulus material. The will judge the kinesic condition generally higher in games were matched as closely as possible on the key interactivity, realism, presence, and enjoyment as components that were salient to the present study. Play- opposed to the incomplete tangible condition. ers were randomly assigned to play either Fighters Un- H2b: In study 2, which utilizes a lightsaber fighting caged for Kinect (kinesic condition, n ¼ 23), or The simulation from Kinect Star Wars, players will judge Fight: Lights Out for the Move (incomplete tangible con- the realistic tangible condition generally higher in dition, n ¼ 23). interactivity, realism, presence, and enjoyment as Fighters Uncaged is a third-person street-fighting opposed to the kinesic condition. game for Kinect that requires the use of punches, kicks, H2c: In study 3, which utilizes table tennis games, dodges, and blocks to defeat an opponent. Matches are players will judge the realistic tangible condition single-player fights against a computer-controlled oppo- generally highest, the incomplete tangible condition nent that scale in difficulty. in the midrange, and the kinesic condition least in The Fight: Lights Out is a third-person street-fighting interactivity, realism, spatial presence, and enjoy- game for the Move system. Since movement is inter- ment. preted by tracking the two Move controllers, only Also, in keeping with the notion that players will react punches and blocks are recognized, as opposed to kicks differently based on the naturalness condition in which and dodging that can also be done with the Kinect.

Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/PRES_a_00193 by guest on 23 September 2021 274 PRESENCE: VOLUME 23, NUMBER 3

Matches are single player fights against computer-con- Table 1. Cronbach’s a Studies 1, 2, and 3 trolled opponents that scale in difficulty. Study Study Study Scale 1 a 2 a 3 a 3.3 Experimental Procedure Perceived interactivity .84 .81 .77 Players stood approximately 6 ft from the game/ Perceived reality .87 .87 .88 computer station to play, and were seated at the com- Spatial presence .94 .94 .93 puter station to complete the questionnaire administered Enjoyment .95 .91 .91 online via surveymonkey.com. Players first answered the demographic portion of the questionnaire, then played the randomly assigned game for 15 min. Gameplay peri- 2001). It was intended as a cross-media measure of spa- ods were timed by a research assistant who received 3 hr tial presence experiences. The subscale includes 19 items, of training on the experimental procedure and details of such as ‘‘I felt I was visiting the places in the video game the gameplay. Participants then completed the game ex- environment.’’ Each item was answered on a 5-point perience portion of the questionnaire, which included scale ranging from 1 (strongly disagree) to 5 (strongly questions of spatial presence, perceived interactivity, per- agree). The scale has been shown to be reliable and valid ceived reality, and enjoyment. Each session lasted a total (Lessiter et al., 2001; Shafer et al., 2011). of 30–40 min. Enjoyment was assessed using 12 items taken from pre- vious studies (Raney & Bryant, 2002; Raney, 2002) and 3.4 Measures modified to apply to video game enjoyment as in past Perceived interactivity was measured using an research (e.g., Shafer, 2012). Some sample items are: eight-item scale adapted from a measure used by Wu ‘‘The game made me feel good’’ and ‘‘I enjoyed the (2006, 2005) that was originally intended to measure game.’’ Each item was rated on an 11-point scale, rang- perceived interactivity of websites (e.g., ‘‘I was in total ing from 0 (not at all) to 10 (extremely). control over the pace of my visit to this website’’). Cronbach’s alpha reliability coefficients for each scale Because of the present study’s focus on video games, in each study are shown in Table 1. some modifications were necessary (e.g., ‘‘I was in total control over the pace of my experience with this game’’). 4 Study 1 Results The items were measured on a 5-point scale ranging 4.1 Analysis of Variance Between from 1 (strongly disagree) to 5 (strongly agree). Conditions Perceived reality was measured using Popova’s (2010) perceived reality measure for video games. Six dimen- The first set of analyses investigated possible differ- sions of perceived reality are measured with the 29-item ences in the variables of interest between the kinesic and scale: magic window (a ¼ .76), typicality (a ¼ .67), incomplete tangible conditions. An analysis of variance identity (a ¼ .76), utility (a ¼ .77), perceptual fidelity (ANOVA) was performed, and the results indicated that (a ¼ .74), and virtual experience (a ¼ .71). Due to the there were no statistically significant differences between necessity to limit the number of parameters estimated in conditions on any of the variables tested. Table 2 shows the path model, the subscales were combined to yield a the descriptive statistics and the results of the ANOVA. single aggregate measure of perceived reality. The items These results indicate that more natural mapping (i.e., were measured on a 5-point scale ranging from 1 kinesic natural mapping) did not result in higher levels (strongly disagree) to 5 (strongly agree). of perceived interactivity, perceived reality, spatial pres- Spatial presence was measured using the spatial pres- ence, or enjoyment for these boxing games, although ence subscale of the ITC-Sense of Presence Inventory the difference for enjoyment did approach significance. (ITC-SOPI: Lessiter, Freeman, Keogh, & Davidoff, However, differences are still possible in the patterns of

Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/PRES_a_00193 by guest on 23 September 2021 Shafer et al. 275

Table 2. Study 1 Descriptive Statistics and ANOVA Results

Variable Condition M (SD) dF(1, 44) p value

Perceived interactivity Kinesic 2.82 (0.77) .16 0.31 .581 Incomplete tangible 2.70 (0.69) Perceived reality Kinesic 2.51 (0.46) .18 0.44 .508 Incomplete tangible 2.60 (0.53) Spatial presence Kinesic 2.63 (0.74) .42 2.05 .159 Incomplete tangible 2.97 (0.87) Enjoyment Kinesic 4.65 (2.34) .52 3.09 .086 Incomplete tangible 5.87 (2.34)

Table 3. Fit Indices for Each Condition Model Study 1

Model APC pAPC ARS pARS AVIF Fit

Incomplete tangible 0.616 <.001 0.647 <.001 2.106 Good Kinesic 0.579 <.001 0.583 <.001 2.489 Good

effects between the two conditions. To investigate this (Kock, 2012; Rosenthal & Rosnow, 1991). The recom- possibility, two path models were analyzed. mended p value for both the APC and the ARS is <.05, which indicates good model fit. The APC and ARS counterbalance each other: typically, the introduction of 4.2 Path Analysis additional latent variable increases the ARS but Path analyses were performed using the WarpPLS decreases the APC, and only the latent variables that 3.0 software package. WarpPLS is designed for both increase the overall explanatory quality of the model nonlinear and linear path analysis. The software is able increase both APC and ARS. The AVIF is an indicator to provide significance testing for each path coefficient. of multicollinearity in the model. When an additional It also produces effect size indicators for each endoge- latent variable overlaps in meaning with an existing nous variable. The numbers WarpPLS produces are latent variable in the model, then that increases multi- more suited for the theoretically derived hypothesis test- collinearity and subsequently increases AVIF. The AVIF ing being conducted in the present study, rather than needs to be lower than 5, and the lower the better analysis that concentrates on model fit. Nevertheless, (Kock). Table 3 shows the fit indices for all models from model fit indices are needed to ensure that the data do each condition in the analysis. As the table shows, both not deviate significantly from the model we are inter- models demonstrated good fit for the data according to ested in applying. To that end, WarpPLS produces three the criteria explained by Kock. model fit indices: the average path coefficient (APC), av- The path analysis was conducted to investigate possi- erage R-squared (ARS), and average variance inflation ble differences between conditions in the patterns of factor (AVIF). The APC and the ARS are calculated as effects apparent between the two controller types. Fig- averages of the absolute values of the path coefficients in ures 2 and 3 show the incomplete tangible and kinesic the model and the R-squared values in the model, conditions, respectively. respectively. The p values for APC and ARS are calcu- In order to determine whether differences exist in the lated via resampling using Bonferroni-like corrections pattern of effects between the models, t-tests were per-

Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/PRES_a_00193 by guest on 23 September 2021 276 PRESENCE: VOLUME 23, NUMBER 3

Figure 2. Incomplete tangible condition path model. Note: Figure 3. Kinesic condition path model. Note: *p < .05; **p < .01. **p < .01.

Table 4. Kinesic and Incomplete Tangible Corresponding Path Coefficient Differences, Study 1

Variable pair Kinesic b Incomplete tangible b tP

Perceived interactivity, perceived reality .82 .75 2.81 <.01 Perceived interactivity, spatial presence .49 .28 3.17 <.01 Perceived reality, spatial presence .42 .64 3.15 <.01 Spatial presence, enjoyment .59 .79 6.69 <.001

formed to compare each path coefficient with its coun- The results of the analysis of the corresponding path terpart. Four t-tests were performed, one for each path coefficients (b) appear in Table 4. As the table shows, and its counterpart in the model. The formula found in significant differences between corresponding paths Equation 1 was used to calculate the t value. This for- existed for all variable pairs. mula was used previously by Keil et al. (2000) to com- Specifically, the impact of perceived interactivity on pare corresponding paths in two path models. In order perceived reality was significantly stronger in the kinesic to use the formula, sample sizes used to compute each condition than in the incomplete tangible condition. path model and standard errors from each path coeffi- Both paths were significant, but for Kinect players, their cient to be compared are entered into the formula to perceptions of realism were more strongly influenced by

produce Spooled (SEs are obtained from WarpPLS out- how interactive they felt the system was. The same was

put). Once Spooled is obtained, that value may be entered true of the impact of perceived interactivity on spatial into the formula for t along with the path coefficients presence; in fact, in the incomplete tangible condition, being compared and the sample sizes from each path the relationship was not significant. For Kinect players, model. Therefore, the t-test formula to compare coun- the level of interactivity they experienced strongly terpart path coefficients is given as: impacted their sense of spatial presence, while the sense of presence was not significantly influenced by perceived sffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi 2 interactivity in the incomplete tangible condition. ½ðÞN1 1 =ðÞN1 þ N2 2 SE1 Spooled ¼ 2 The impact of perceived reality on spatial presence, þ ½ðÞN2 1 =ðÞN1 þ N2 2 SE2 hiqffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi ð1Þ however, was found to work opposite of the previous t ¼ ðÞPC PC = S 1 þ 1 1 2 pooled N1 N2 two relationships. Perceived reality did not significantly impact spatial presence in the kinesic condition, but did

where Spooled ¼ pooled estimator for the variance; t ¼ t significantly impact spatial presence for players in the

statistic with N1þN22 degrees of freedom; Ni ¼ sam- incomplete tangible condition. This finding points to a

ple size of data set according to condition; SEi ¼ stand- major difference between the conditions: namely, feel-

ard error of path i in model i;PCi ¼ path coefficient of ings of spatial presence were driven much more by per- path i in model i. ceived interactivity in the kinesic condition, while it was

Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/PRES_a_00193 by guest on 23 September 2021 Shafer et al. 277

far more dependent on perceived reality in the incom- 6 Study 2 Method plete tangible condition. 6.1 Participants The final comparison was the impact of spatial pres- ence on enjoyment. Spatial presence was a significantly Data were collected from 100 student participants stronger predictor of enjoyment in the incomplete tangi- recruited from film, speech, and communication courses ble condition than in the kinesic condition. In fact, at a midsized research university in the southern-central about 62% of the variance in enjoyment could be United States. Students were offered extra credit for par- explained by spatial presence for the incomplete tangible ticipation, along with other equally valuable extra credit condition. In the kinesic condition, however, only 32% activities. The average age in the sample was 20.6 years of the variance in enjoyment was explained by spatial (SD ¼ 1.6 years). Men (n ¼ 53, 53%) outnumbered presence. women (n ¼ 47, 47%) by a slight margin.

6.2 Stimulus Material 5 Study 1 Discussion One game was used as stimulus material. Kinect Star Wars offers players a one- or two-player storyline as In the first study, participants played street fight- well as a series of minigames. All in-game activities ing games either holding wand controllers (incomplete require the full-body motion inherent to the Kinect sys- tangible condition) or without any controllers in their tem. In the present study, players played the Duels of hands (kinesic condition). The levels of perceived inter- Fate minigame, which pitted their Jedi Knight avatar activity, spatial presence, perceived reality, and enjoy- against a series of progressively more difficult enemies. ment did not differ significantly between the two Players were required to swing a virtual lightsaber (or, in conditions, although the differences in enjoyment the realistic tangible mapping condition, a real lightsaber approached significance. The model, in which perceived hilt) to attack and defend. They were also able to push interactivity, perceived reality, and perceived spatial and kick the enemy at different points in the battle. presence predicted enjoyment, was a good fit for the Type of controller mapping was manipulated to pro- data and explained 62% of variance in enjoyment in the duce two distinct conditions. In the kinesic condition, incomplete tangible condition model and 34% of var- players (n ¼ 48) played Kinect Star Wars as it was origi- iance in the kinesic model. The relationships between nally intended. In the realistic tangible condition, players the variables in the model, however, were different (n ¼ 52) played the same game while holding a realistic between the two conditions. When players held a Move lightsaber hilt1—an object with weight that looks similar wand in their hands, perceived interactivity predicted to the weapon the game’s avatar holds. perceived reality, which, in turn, predicted spatial pres- ence, and spatial presence predicted enjoyment. In con- 6.3 Experimental Procedure trast, when participants used their whole body as the controller (Kinect), perceived interactivity predicted The experimental procedure was identical to that both perceived reality and spatial presence, but the path used in Study 1. between perceived reality and spatial presence was not significant. 6.4 Measures It seems that when players use their bodies as controls The measures used for Study 2 were identical to (kinesic condition), perceived interactivity is an impor- the instruments used in Study 1. Table 1 (under Study tant predictor of spatial presence and perceived reality; 1) shows Cronbach’s a levels for each scale. however, these variables explain only 34% of variance in

enjoyment, indicating that there might be other variables 1. The blade was removed from the Force FX lightsabers because of that account for the rest of the variance in enjoyment. space constraints in the lab.

Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/PRES_a_00193 by guest on 23 September 2021 278 PRESENCE: VOLUME 23, NUMBER 3

Table 5. Study 2 Descriptive Statistics and ANOVA Results

Variable Condition M (SD) dF(1, 99) p value

Perceived interactivity Kinesic 2.42 (0.84) .05 0.09 .770 Realistic tangible 2.46 (0.68) Perceived reality Kinesic 2.40 (0.48) .02 0.04 .849 Realistic tangible 2.41 (0.49) Spatial presence Kinesic 2.62 (0.82) .26 1.55 .217 Realistic tangible 2.84 (0.86) Enjoyment Kinesic 5.14 (2.02) .04 0.03 .876 Realistic tangible 5.07 (1.86)

Table 6. Fit Indices for Each Condition Model, Study 2

Model APC pAPC ARS pARS AVIF Fit

Realistic tangible 0.462 <.001 0.373 <.001 1.592 Good Kinesic 0.559 <.001 0.524 <.001 1.749 Good

7 Study 2 Results

7.1 Analysis of Variance between Conditions

As in Study 1, possible differences between condi- tions on each of the variables were examined. The results of the ANOVA indicated that there were no statistically Figure 4. Kinesic condition path model. Note: *p < .05; **p < .01. significant differences between the kinesic and the realis- tic tangible conditions on any of the variables tested, as shown in Table 5. These results are consistent with the results of Study 1. More natural mapping, in this case, realistic tangible natural mapping, did not result in higher levels of per- ceived interactivity, perceived reality, spatial presence, or enjoyment for Kinect Star Wars. The path analysis that Figure 5. Realistic tangible condition path model. Note: **p < .01. follows was designed to investigate the patterns of effects between the two conditions. Figures 4 and 5 show the incomplete tangible and kinesic condition results for the purpose of comparing 7.2 Path Analysis corresponding paths. Path analyses were performed using the WarpPLS As in Study 1, t-tests were performed to compare each software package as in Study 1. Models for the realistic path coefficient with its counterpart. Four t-tests were tangible and kinesic models demonstrated good fit for performed, one for each path and its counterpart in the the data as shown in Table 6. model. Significant differences between corresponding

Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/PRES_a_00193 by guest on 23 September 2021 Shafer et al. 279

Table 7. Kinesic and Realistic Tangible Corresponding Path Coefficient Differences, Study 2

Variable pair Kinesic b Realistic tangible b tP

Perceived interactivity, perceived reality .67 .61 5.05 <.001 Perceived interactivity, spatial presence .57 .08 19.61 <.001 Perceived reality, spatial presence .33 .64 –11.44 <.001 Spatial presence, enjoyment .67 .52 9.87 <.001

paths coefficients (b) existed for all variable pairs, as condition). Similar to Study 1, the levels of perceived shown in Table 7. interactivity, spatial presence, perceived reality, and Consistent with the results from Study 1, the impact enjoyment did not differ significantly between the two of perceived interactivity on perceived reality was signifi- conditions. The model, in which perceived interactivity, cantly stronger in the kinesic condition than in the realis- perceived reality, and perceived spatial presence pre- tic tangible condition. Both paths were significant, but dicted enjoyment was a good fit for the data and Kinect players based more of their perceptions of realism explained 27% of the variance in enjoyment in the realis- on how responsive the system was to them or how in tic tangible condition model and 45% of the variance in control of the game they felt they were (i.e., interactiv- the kinesic model. The relationships between the varia- ity). The impact of perceived interactivity on spatial pres- bles in the model were different between the two condi- ence mirrored that of Study 1. Perceived interactivity tions. When players held a toy lightsaber hilt in their had no significant impact on spatial presence in the real- hands, the path from perceived interactivity to enjoy- istic tangible condition, while the effect was significant ment went first through perceived reality, which, in turn, and strong in the kinesic condition. predicted spatial presence, and spatial presence predicted The impact of perceived reality on spatial presence was enjoyment. In contrast, when participants used their found to be significantly stronger in the realistic tangible whole body as the controller (kinesic condition), per- condition than in the kinesic condition. Perceptions of ceived reality was not a necessary step in the path because realism again drove spatial presence in the condition in perceived interactivity predicted both perceived reality which players were able to hold an object in their hands. and spatial presence, and spatial presence predicted In the kinesic condition, spatial presence was driven by enjoyment. Thus, in the kinesic condition, the path of perceived interactivity, although perceived reality did effects between perceived interactivity and enjoyment contribute to feelings of spatial presence in Study 2 was shorter and did not include perceived reality as one where it did not in Study 1. of its necessary steps. The effect of spatial presence on enjoyment was found to be significantly stronger in the kinesic condition than in the realistic tangible condition; 45% of the variance in 9 Study 3 Method enjoyment was explained by spatial presence, versus only 9.1 Participants 27% explained in the realistic tangible condition. Data were collected from 157 student participants 8 Study 2 Discussion recruited from film, speech, journalism, computer sci- ence and communication courses at a midsized research In Study 2, participants played Kinect Star Wars university in the southern-central United States. Stu- and battled enemies with a lightsaber by either holding a dents were offered extra credit for participation, along replica lightsaber hilt (realistic tangible condition) or by with other equally valuable extra credit activities. The av- pretending to hold a lightsaber in their hands (kinesic erage participant was 21 years of age (SD ¼ 2.4 years).

Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/PRES_a_00193 by guest on 23 September 2021 280 PRESENCE: VOLUME 23, NUMBER 3

Table 8. Study 3 Descriptive Statistics and ANOVA Results

Variable Condition M (SD) dF(2,155) p value

Perceived interactivity Realistic tangible 2.91 (0.63) .23 0.81 .448 Kinesic 3.03 (0.71) Incomplete tangible 2.88 (0.65) Perceived reality Realistic tangible 2.86 (0.48) .06 0.04 .906 Kinesic 2.85 (0.51) Incomplete tangible 2.83 (0.57) Spatial presence Realistic tangible 2.94 (0.77) .35 1.76 .177 Kinesic 3.18 (0.76) Incomplete tangible 2.91 (0.78) Enjoyment Realistic tangible 6.55 (1.86) .40 2.32 .102 Kinesic 6.44 (1.63) Incomplete tangible 5.78 (2.25)

Men (n ¼ 89, 57%) outnumbered women (n ¼ 68, 43%) 9.3 Experimental Procedure by a relatively small margin in the sample. The experimental procedure was identical to that used in Studies 1 and 2. 9.2 Stimulus Material

Two games were employed as stimulus material: 9.4 Measures Kinect Sports Table Tennis (Kinect system for the Xbox 360) and Move Sports Champions Table Tennis (Move The measures used for Study 3 were identical to system for the PS3). Players were randomly assigned to the instruments used in Studies 1 and 2. Table 1 (under one of the following conditions: realistic tangible (n ¼ Study 1) shows Cronbach’s a for each scale. 53, Kinect played with a real paddle); kinesic (n ¼ 53, Kinect played as intended with empty hands), and incomplete tangible (n ¼ 51, Move played with wand 10 Study 3 Results controllers). 10.1 Analysis of Variance between Kinect Sports Table Tennis pits your avatar against a Conditions computer-controlled opponent for a game of ping pong. Using a hand as the paddle, players hit, spin, or smash Possible differences between conditions on the var- the ball over the net. The Kinect device senses the force iables themselves were examined using ANOVA. Just as of the strike swinging as well as the angle of the player’s in Studies 1 and 2, the results indicated that there were hand, and adjusts the ball’s trajectory accordingly. The no statistically significant differences between conditions player must also move left and right to return tough on any of the variables, as shown in Table 8. shots. Just as in Study 1 and Study 2, more naturally mapped Move Sports Champions Table Tennis is quite similar to control interfaces did not result in significant differences the Kinect version of ping pong. Players control a styl- in perceived interactivity, perceived reality, spatial pres- ized avatar against a CPU-controlled opponent in a ence, or enjoyment. The path analysis to follow exam- game of ping pong. There is a slight difference in the ined and compared three models to determine whether, mechanics of gameplay, since the Move emphasizes a as in the other two studies, differences would be found more precise approach to ball placement than Kinect. in the pattern of effects between the variables.

Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/PRES_a_00193 by guest on 23 September 2021 Shafer et al. 281

Table 9. Fit Indices for Each Condition Model

Model APC pAPC ARS pARS AVIF Fit

Incomplete tangible 0.535 <.001 0.494 <.001 2.383 Good Kinesic 0.492 <.001 0.395 <.001 1.467 Good Realistic tangible 0.483 <.001 0.389 <.001 1.416 Good

Figure 6. Incomplete tangible condition path model. Note: *p < .05; Figure 8. Realistic tangible condition path model. Note: *p < .05; **p < .01. **p < .01.

tion than in the kinesic condition or the realistic tangible condition. The interactivity experienced in the incom- plete tangible condition impacted perceived reality more strongly than in either of the other two conditions. However, perceived interactivity was most effective in predicting spatial presence in the kinesic condition; the effect was significantly stronger than in either tangible Figure 7. Kinesic condition path model. Note: **p < .01. condition. For the effect of perceived reality on spatial presence, the impact was significantly stronger for both 10.2 Path Analysis tangible conditions. In conditions where players held Path analyses were performed just as they were in objects, whether real objects or controllers, their percep- Studies 1 and 2, but a third model was added due to the tions of reality drove their sense of spatial presence. For presence of three conditions in Study 3. Table 9 shows the kinesic condition across all studies, the results show the fit indices for all models from each condition in the that perceived interactivity has a much stronger influence analysis. As the table shows, each model fits the data well. on spatial presence than in the tangible conditions. Figures 6, 7, and 8 show the path models for the In terms of enjoyment, the results indicate that the incomplete tangible, kinesic, and realistic tangible condi- incomplete tangible condition was most effective, but tions. this effect was not significantly different from the predic- As in Studies 1 and 2, t-tests were performed to com- tive relationship found in the kinesic condition or the re- pare each path coefficient with its counterpart in each alistic tangible condition. Both paths were significant, other condition. Twelve t-tests were performed, one for but Kinect players based more of their perceptions of re- each path and its counterpart in the other two models. alism on how responsive the system was to them, or how The results of the analysis of the corresponding path in control of the game they felt they were (i.e., interac- coefficients (b) appear in Tables 10, 11, and 12. tivity). The impact of perceived interactivity on spatial Contrary to the results of Study 1 and Study 2, the presence mirrored Study 1. Perceived interactivity had impact of perceived interactivity on perceived reality was no significant impact on spatial presence in the realistic significantly stronger in the incomplete tangible condi- tangible condition, while the effect was significant and

Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/PRES_a_00193 by guest on 23 September 2021 282 PRESENCE: VOLUME 23, NUMBER 3

Table 10. Kinesic and Incomplete Tangible Condition Path Coefficient Differences, Study 3

Variable pair Kinesic b Incomplete tangible b tP

Perceived interactivity, perceived reality .56 .76 –13.07 <.001 Perceived interactivity, spatial presence .43 .28 5.56 <.001 Perceived reality, spatial presence .39 .50 –3.86 <.001 Spatial presence, enjoyment .58 .60 –1.34 Ns

Table 11. Kinesic and Realistic Tangible Condition Path Coefficient Differences, Study 3

Variable pair Kinesic b Realistic tangible b tP

Perceived interactivity, perceived reality .56 .54 1.23 Ns Perceived interactivity, spatial presence .43 .25 7.33 <.001 Perceived reality, spatial presence .39 .59 –7.41 <.001 Spatial presence, enjoyment .58 .55 2.01 <.05

Table 12. Incomplete Tangible and Realistic Tangible Condition Path Coefficient Differences, Study 3

Realistic Incomplete Variable pair tangible b tangible b tP

Perceived interactivity, perceived reality .54 .76 –9.57 <.001 Perceived interactivity, spatial presence .25 .28 –1.20 Ns Perceived reality, spatial presence .59 .50 3.91 <.001 Spatial presence, enjoyment .55 .60 –2.17 <.05

strong in the kinesic condition. There was a marginally levels of perceived interactivity, perceived reality, spatial significant difference between the incomplete tangible presence, and enjoyment. The pattern of path coeffi- and the realistic tangible conditions, and no significant cients indicate that, similar to Studies 1 and 2, when difference between kinesic and realistic tangible condi- players hold something in their hands, the path from tions. perceived interactivity to enjoyment goes through per- ceived reality first, which predicts spatial presence, which, in turn, predicts enjoyment. When players are not 11 Study 3 Discussion holding anything in their hands, the path from perceived interactivity to enjoyment is shortened and goes directly In the third study, participants played a simulated from perceived interactivity through spatial presence to game of table tennis and all three conditions (kinesic, enjoyment. incomplete tangible, and realistic tangible) were com- pared simultaneously. However, because the same game could not be used for all conditions, the comparison 12 General Discussion between kinesic and realistic tangible and incomplete tangible is imperfect. As in Studies 1 and 2, there were In three experimental studies, we examined the no significant differences among the conditions in the impact of different natural controllers on the video game

Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/PRES_a_00193 by guest on 23 September 2021 Shafer et al. 283

experience. Specifically, we compared the effects of three mon coding, both sensory and cognitive systems exist in different types of NUI controllers: handheld wands a common neural network which encodes both action (with Move for the PS3) in the incomplete tangible con- and perception. Either can automatically activate the dition, empty hands (with Kinect for the Xbox 360) in other in an associative priming state. ‘‘Common coding the kinesic condition, and holding a real object to repre- implies that there are interactions between execution, sent the virtual object (while using Kinect) in the realistic perception, and imagination of movement’’ (Chandrase- tangible condition. We compared the effects of the con- kharan et al., p. 317). There is evidence that even just ditions on perceptions of interactivity, reality of the the perception of an action fires mirror neurons which game, spatial presence, and, ultimately, enjoyment. We are associated with the muscles responsible for carrying also proposed and tested a theoretical model of the pro- out that action (Oztop, Kawato, & Arbib, 2006; Fadiga, cess of effects between these variables, arguing that per- Fogassi, Pavesi, & Rizzolatti, 1995; Chandrasekharan ceived interactivity influences perceptions of reality and et al.). When observation is done by those who have spatial presence. Perceived reality has an effect on spatial much experience in the observed action (such as profes- presence, and spatial presence, being inherently pleasura- sional athletes observing their sport), there is strong ac- ble, directly affects enjoyment (McGloin et al., 2011; tivity in the premotor, parietal, and posterior cortical Shafer et al., 2011; Skalski et al., 2011). regions of the brain (Chandrasekharan et al.). Comparing perceptions of interactivity, reality, spatial Perceived reality seems to be incredibly important to presence, and enjoyment among the players who used feelings of spatial presence when holding objects, different controllers to play street fighting, lightsaber whether those objects are the actual device one would fighting, or table tennis games, we found no differences use for the activity shown on screen or whether they are between the conditions on our key variables. In all three controllers. Evidence from experiments in neuroscience studies, participants in every condition rated perceived with primates indicates that when objects are grasped, interactivity, perceived reality, the level of spatial pres- canonical neurons, in addition to the neighboring mirror ence they experienced, and the enjoyment they felt at neurons, are activated (Oztop et al., 2006; Rajmohan & about the same level. However, the pattern of effects var- Mohandas, 2007). ied across the conditions. What this evidence suggests is that a player’s perception It seems that the general difference in path coefficients of reality is tied to the grasping of an object that is repre- between the conditions was that in empty-handed or sentative of the virtual object being used by the avatar. kinesic conditions, perceived interactivity was a strong The acts of (1) observing a graspable object (a virtual predictor for both perceived reality and spatial presence, object would not be considered graspable), and (2) sub- and perceived reality (weakly) influenced perceptions of sequently grasping that object, fire both canonical neu- spatial presence. In contrast, when participants held a rons and mirror neurons. In situations where there are no controller or an actual object in their hands, perceived graspable objects involved (i.e., the kinesic conditions), interactivity was a strong predictor of perceived reality, only mirror neurons would fire (Oztop et al., 2006; Raj- which predicted spatial presence. Spatial presence then, mohan & Mohandas, 2007), which seems to have in turn, predicted enjoyment. The relative importance of resulted in perceived reality having a diminished role in the link between perceived reality and spatial presence in the production of spatial presence in kinesic conditions. the process of effects between perceptions of interactivity and enjoyment is interesting. 12.1 Limitations Our results suggest that when the body is used as the controller, the importance of perceived reality as a medi- There are a few minor limitations to the present ator between interactivity and enjoyment is diminished. study. First, data were collected only from a student sam- The common coding framework (Chandrasekharan ple; therefore, the results cannot be generalized to all et al., 2010) can explain these results. According to com- gamers.

Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/PRES_a_00193 by guest on 23 September 2021 284 PRESENCE: VOLUME 23, NUMBER 3

Also, the reader may have noted that values for each of build theoretical models that can widen and deepen our the variables tested (e.g., spatial presence, interactivity, understanding of human cognition, interaction, and perceived reality) were not particularly high. Means were presence in video games. neither high nor low on a scale from 1–5, but were merely average. This would indicate that there are games/controller types that could potentially invoke References higher levels of the studied dimensions. Rather than demonstrating a weakness in the study, however, this Bracken, C. C., & Skalski, P. (2009). Presence and video indicates that we do not have a ceiling effect or a floor games: The impact of image quality. PsychNology Journal, 7, effect, which is positive. Keep in mind that many of the 101–112. Brewer, W. F. (1987). Schemas versus mental models in human participants were not gamers per se, and therefore may memory. In P. Morris (Ed.), Modelling cognition (pp. 187– have had a steep learning curve or unfamiliarity with the 197). New York: Wiley. games. This adds ecological validity to the study. Busselle, R., & Bilandzic, H. (2008). Fictionality and A third limitation is the inability to directly compare perceived realism in experiencing stories: A model of narra- the same title across both platforms used (Move and tive comprehension and engagement. Communication Kinect). This is unavoidable, since very few identical Theory, 18(2), 255–280. doi:10.1111/j.1468- titles are produced for both systems, and represents only 2885.2008.00322.x a small limitation in the study. Chandrasekharan, S., Mazalek, A., Nitsche, M., Chen, Y., & Ranjan, A. (2010). Ideomotor design: Using common cod- ing theory to derive novel video game interactions. Prag- 12.2 Future Research matics & Cognition, 18(2), 313–339. This explanation of our findings rests on argu- Chen, Y., & Raney, A. A. (2009). Mood management and ments that integrate neuroscience with player percep- highly interactive video games: An experimental examination of Wii playing on mood change and enjoyment. Paper pre- tions of video game interactions. Hard empirical evi- sented at the International Communication Association dence is needed, however. Future research will integrate Conference, Chicago, IL. these areas by analyzing neurological data of players play- Downes, E. J., & McMillian, S. J. (2000). Defining interactiv- ing games with various NUIs. Such data may be gath- ity: A qualitative identification of key dimensions. New ered via functional magnetic resonance imaging (fMRI) Media & Society, 2(2), 157–179. doi:10.1177/ or electroencephalography (EEG). Specifically, we argue 14614440022225751 that neurological differences should be seen between Drettakis, G., Roussou, M., Reche, A., & Tsingos, N. (2007). players using tangibly mapped systems and players using Design and evaluation of a real-world virtual environment kinesic systems. for architecture and urban planning. Presence: Teleoperators In addition, recent developments in virtual reality and Virtual Environments, 16(3), 318–332. doi:10.1162/ (VR) technology suggest links to the common coding pres.16.3.318 framework. Research indicates that video game play can Fadiga, L., Fogassi, L., Pavesi, G., & Rizzolatti, G. (1995). result in improved mental rotations (Chandrasekharan Motor facilitation during action observation: A magnetic stimulation study. Journal of Neurophysiology, 73(6), 2608– et al., 2010). Shifting camera views in a game (which 2611. take on a new twist in immersive VR) can engage a player Green, M. C., Brock, T. C., & Kaufman, G. F. (2004). Under- by requiring more mental rotations to reorient the ava- standing media enjoyment: The role of transportation into tar. There is a growing body of work showing connec- narrative worlds. Communication Theory, 14(4), 311–327. tions between attention and motor systems (Welsh, doi:10.1111/j.1468-2885.2004.tb00317.x Weeks, Chua, & Goodman, 2007; Hommel, Mu¨sseler, Grodal, T. (2000). Video games and the pleasures of control. Aschersleben, & Prinz, 2001), and further investigation In D. Zillmann & P. Vorderer (Eds.), Media entertainment is needed. The goal of such studies suggested here is to (pp. 197–213). Mahwah, NJ: Erlbaum.

Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/PRES_a_00193 by guest on 23 September 2021 Shafer et al. 285

Hall, A. (2003). Reading realism: Audiences’ perceptions of 22(2), 15. Retrieved from http://www.aaai.org/ojs the realism of mass media texts. Journal of Communication, /index.php/aimagazine/article/viewArticle/1558 53(4), 624–641. doi:10.1093/joc/53.4.624 Lee, K. M. (2004). Presence, explicated. Communication Heeter, C. (2000). Interactivity in the context of designed Theory, 14, 27–50. doi:10.1093/ct/14.1.27 experiences. Journal of Interactive Advertising, 1(1), 75–89. Lessiter, J., Freeman, J., Keogh, E., & Davidoff, J. (2001). Hefner, D., Klimmt, C., & Vorderer, P. (2007). Identification A cross-media presence questionnaire: The ITC-Sense of with the player character as determinant of video game Presence Inventory. Presence: Teleoperators and Virtual enjoyment. Proceedings of Entertainment Computing, ICEC Environments, 10(3), 282–297. doi:10.1162/ 2007, 39–48. 105474601300343612 Hommel, B., Mu¨sseler, J., Aschersleben, G., & Prinz, W. Limperos, A. M., Schmierbach, M. G., Kegerise, A. D., & (2001). The theory of event coding (TEC): A framework for Dardis, F. E. (2011). Gaming across different consoles: perception and action planning. Behavioral and Brain Scien- Exploring the influence of control scheme on game-player ces, 24(5), 849–878. doi:10.1017/S0140525X01000103 enjoyment. Cyberpsychology, Behavior, and Social Networking, Horvath, K., & Lombard, M. (2010). Social and spatial pres- 14(6), 345–350. ence: An application to optimize human–computer interac- McGloin, R., Farrar, K. M., & Krcmar, M. (2011). The impact tion. PsychNology Journal, 8(1), 85–114. of controller naturalness on spatial presence, gamer enjoy- Ivory, D. J., & Kalyanaraman, S. (2007). The effects of ment, and perceived realism in a tennis simulation video technological advancement and violent content in video game. Presence: Teleoperators and Virtual Environments, games on players’ feelings of presence, involvement, 20(4), 309–324. physiological arousal, and aggression. Journal of Communi- McMahan, A. (2003). Immersion, engagement and presence. cation, 57(3), 532–555. doi:10.1111/j.1460- In M. J. P. Wolfe & B. Perron (Eds.), The video game theory 2466.2007.00356.x reader (pp. 67–86). New York: Routledge. James, W. (1890). Principles of psychology. New York: Holt. Nunez, D., & Blake, E. (2006). Learning, experience, and cog- Jeong, E. J., Bohil, C. J., & Biocca, F. A. (2011). Brand logo nitive factors in the presence experiences of gamers: An ex- placements in violent games. Journal of Advertising, 40(3), ploratory relational study. Presence: Teleoperators and Virtual 59–72. doi:10.2753/JOA0091-3367400305 Environments, 15(4), 373–380. Keil, M., Tan, B. C., Wei, K. K., Saarinen, T., Tuunainen, V., Oztop, E., Kawato, M., & Arbib, M. (2006). Mirror neurons & Wassenaar, A. (2000). A cross-cultural study on escalation and imitation: A computationally guided review. Neural of commitment behavior in software projects. Management Networks, 19(3), 254–271. doi:10.1016/ Information Systems Quarterly, 24(2), 299–326. j.neunet.2006.02.002 Klimmt, C., Hartmann, T., & Frey, A. (2007). Effectance and Persky, S., & Blascovich, J. (2008). Immersive virtual video control as determinants of video game enjoyment. CyberPsy- game play and presence: Influences on aggressive feelings chology & Behavior, 10(6), 845–848. doi:10.1089 and behavior. Presence: Teleoperators and Virtual Environ- /cpb.2007.9942 ments, 17(1), 57–72. doi:10.1162/pres.17.1.57 Klimmt, C., & Vorderer, P. (2003). Media psychology ‘‘is not Popova, L. (2010). Perceived reality in media messages: Concept yet there’’: Introducing theories on media entertainment to explication and testing. Dissertation Abstract International, the presence debate. Presence: Teleoperators and Virtual 72(3), 853A. Retrieved from http://proquest.umi.com Environments, 12(4), 346–359. /pqdweb?did=2265987221&sid=1&Fmt=2&clientId Kock, N. (2012). WarpPLS 3.0 user manual [online docu- =1569&RQT=309&VName=PQD ment]. Retrieved from http://www.scriptwarp.com Rajmohan, V., & Mohandas, E. (2007). Mirror neuron system. /warppls/UserManual_WarpPLS_V3.pdf Indian Journal of Psychiatry, 49(1), 66. doi:10.4103/0019- Krcmar, M., Farrar, K., & McGloin, R. (2011). The effects of 5545.31522 video game realism on attention, retention and aggressive Raney, A. A. (2002). Moral judgment as a predictor of enjoy- outcomes. Computers in Human Behavior, 27(1), 432–439. ment of crime drama. Media Psychology, 4(4), 307–324. doi:10.1016/j.chb.2010.09.005 doi:10.1207/S1532785XMEP0404_01 Laird, J., & van Lent, M. (2001). Human-level AI’s killer Raney, A. A., & Bryant, J. (2002). Moral judgment and crime application: Interactive computer games. AI Magazine, drama: An integrated theory of enjoyment. Journal of Com-

Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/PRES_a_00193 by guest on 23 September 2021 286 PRESENCE: VOLUME 23, NUMBER 3

munication, 52(2), 402–415. doi:10.1111/j.1460- Vorderer, P. (2000). Interactive entertainment and beyond. In 2466.2002.tb02552.x D. Zillmann & P. Vorderer (Eds.), Media entertainment: Rosenthal, R., & Rosnow, R. L. (1991). Essentials of The psychology of its appeal (pp. 21–36). Mahwah, NJ: Erl- behavioral research: Methods and data analysis. New York: baum. McGraw-Hill. Vorderer, P. (2001). It’s all entertainment—sure. But what Shafer, D. M. (2012). Causes of state hostility and enjoyment exactly is entertainment? Communication research, media in player vs. player and player vs. environment video games. psychology, and the explanation of entertainment experien- Journal of Communication, 62(4), 719–737. doi:10.1111/ ces. Poetics, 29(4), 247–261. j.1460-2466.2012.01654.x Vorderer, P., Hartmann, T., & Klimmt, C. (2003). Explaining Shafer, D. M., Carbonara, C. P., & Popova, L. (2011). Spatial the enjoyment of playing video games: The role of competition. presence and perceived reality as predictors of motion-based Paper presented at the Second International Conference on video game enjoyment. Presence: Teleoperators and Virtual Entertainment Computing, Pittsburgh, PA. Environments, 20(6), 591–619. Vorderer, P., Klimmt, C., & Ritterfeld, U. (2004). Enjoyment: Sherry, J. L. (2004). Flow and media enjoyment. Communica- At the heart of media entertainment. Communication Theory, tion Theory, 14(4), 328–347. doi:10.1111/j.1468- 14(4), 388–408. 2885.2004.tb00318.x Welsh, T. N., Weeks, D. J., Chua, R., & Goodman, D. (2007). Skalski, P., Tamborini, R., Shelton, A., Buncher, M., & Lind- Perceptual-motor interaction: Some implications for HCI. mark, P. (2011). Mapping the road to fun: Natural video In A. Sears & J. A. Jacko (Eds.), The human–computer inter- game controllers, presence, and game enjoyment. New action handbook: Fundamentals, evolving technologies and Media and Society, 13(2), 224–242. doi:10.1177/ emerging applications (2nd ed., pp. 27–41). Boca Raton, FL: 1461444810370949 CRC Press. Skalski, P., & Whitbred, R. (2010). Image versus sound: A Wirth, W., Hartmann, T., Bo¨cking, S., Vorderer, P., Klimmt, comparison of formal feature effects on presence and video C., Schramm, H., ...Ja¨ncke, P. (2007). A process model of game enjoyment. PsychNology Journal, 8, 67–84. the formation of spatial presence experiences. Media Psychol- Steuer, J. (1992). Defining virtual reality: Dimensions deter- ogy, 9(3), 493–525. mining telepresence. Journal of Communication, 42(4), 73– Wu, G. (2006). Conceptualizing and measuring the perceived 93. doi:10.1111/j.1460-2466.1992.tb00812.x interactivity of websites. Journal of Current Issues and Tamborini, R., & Skalski, P. (2006). The role of presence in Research in Advertising, 28(1), 87–104. the experience of electronic games. In P. Vorderer & J. Bry- Wu, G. (2005). The mediating role of perceived interactivity in ant (Eds.), Playing video games: Motives, responses, and conse- the effect of actual interactivity on attitude toward the web- quences (pp. 225–240). Mahwah, NJ: Erlbaum. site. Journal of Interactive Advertising, 5(2), 29–39.

Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/PRES_a_00193 by guest on 23 September 2021