Narrative Games for Learning: Testing the Discovery and Narrative Hypotheses

Deanne M. Adams, Richard E. Mayer, Alan Koenig and Richard Wainess and Andrew MacNamara University of California, Los Angeles University of California, Santa Barbara

Strong claims are made for the potential educational effectiveness of narrative-based adventure games, but evidence about how to construct effective educational games is needed (Clark, Yates, Early, & Moulton, 2010; O’Neil & Perez, 2008). College students played a computer-based narrative discovery learning game called Crystal Island (Spires et al., 2010), in which they learned about pathogens (in Experiment 1), or one called Cache 17 (Koenig, 2008), in which they learned how electromechanical devices work (in Experiment 2). In

broadly. media comparison tests, participants who learned by playing the game performed worse than students who learned from a matched slideshow presentation on retention (d ϭ 1.37), transfer (d ϭ 0.57), and difficulty publishers. rating (d ϭ 0.93) in Experiment 1 and on posttest score (d ϭ 0.31) and learning time (d ϭ 2.89) in Experiment 2. In value-added tests, taking away the narrative theme concerning a detective story in the Cache 17 game allied did not significantly affect students’ posttest score (d ϭϪ0.16) or learning time (d ϭϪ0.22) in Experiment disseminated its 2. Overall, these results provide no evidence that computer-based narrative games offer a superior venue for be of

to academic learning under short time spans of under 2 hr. Findings contradict the discovery hypothesis that one students learn better when they do hands-on activities in engaging scenarios during learning and the narrative not or

is hypothesis that students learn better when games have a strong narrative theme, although there is no evidence concerning longer periods of game play. and

user Keywords: educational games, discovery, narrative, computer-based learning, game design Association

individual There is strong evidence that computer games are hugely pop- In the present set of experiments, we take a step toward provid- the ular. For example, as of 2002, more money was spent on computer

Psychological ing evidence-based guidance for game designers by examining the of games in the United States—$6.9 billion—than on box-office instructional effectiveness of the game genre of computer-based use movies, and approximately 145 million Americans (or about 60% narrative discovery learning games, which has been proposed to of the population over age 6) regularly played computer games improve academic learning. In computer-based narrative discovery American

personal (Lee, Park, & Jin, 2006). Advocates of educational gaming have learning games, players are faced with a challenge within a sim- the

the proposed that educators should harness the appeal of computer by ulated environment in which they can control the actions of one or

for games as a vehicle for fostering student learning, but reviews of more characters. We define narrative games as those having a clear the research literature have not yielded strong support for the story line in which characters’ actions are directed toward a goal in

solely instructional effectiveness of computer games (Clark, Yates, Early, the story. We define discovery games as those in which players copyrighted & Moulton, 2010; O’Neil & Perez, 2008; O’Neil, Wainess, & learn academic content in the context of solving problems or is Baker, 2005; Tobias & Fletcher, 2011). In a recent review, for exploring within a game rather than by being directly presented intended example, O’Neil and Perez (2008) lamented that “there is almost with the academic content. is no guidance for game designers and developers on how to design Rather than trying to develop games on our own, we received document games that facilitate learning” (p. ix). permission to use two narrative discovery games that had been article

This built by game design experts and field-tested over several years. This We sought high-quality narrative discovery games that were ex- This article was published Online First September 26, 2011. Deanne M. Adams, Richard E. Mayer, and Andrew MacNamara, De- pertly designed to be fun and to serve a clear educational purpose, partment of Psychology, University of California, Santa Barbara; Alan and we used two different, well-designed games to give our media Koenig and Richard Wainess, National Center for Research on Evaluation, comparison studies a fair testing. We measured learning outcomes Standards, and Student Testing (CRESST), University of California, Los through tests designed to tap transfer (i.e., ability to use the Angeles. material to generate solutions to new problems that were not part This research was supported by a grant from the Office of Naval of the game) and retention (i.e., ability to remember the material), Research. The game used in Experiment 1 was developed for use by Spires, consistent with calls to include measures of transfer in assessments Turner, Rowe, Mott, and Lester (2010). The game used in Experiment 2 of learning with games (Hung & van Eck, 2010; Mayer, 2011; was developed by Koenig (2008). Experiment 1 was based on a senior Reese, 2010). honors thesis by Andrew MacNamara. Correspondence concerning this article should be addressed to Deanne First, Crystal Island was developed and revised by James Lester M. Adams or Richard E. Mayer, Department of Psychology, University of at North Carolina State University, who is a well-known expert in California, Santa Barbara, CA 93106. E-mail: [email protected] or narrative game design. In Crystal Island (Spires, Turner, Rowe, [email protected] Mott, & Lester, 2010), the player is a visitor to a research team on

235 236 ADAMS ET AL.

a remote island whose stay is disrupted by the spread of an “Even though the current chapter provides strong theoretical rea- unknown disease (shown in Figure 1). The player’s task is to sons why narratives are important, future studies should empiri- discover the source of the disease through interacting with other cally test the effects of narratives on various psychological vari- characters and using lab microscopes to run tests. The game is ables” (Lee et al., 2006, p. 271). We take up this challenge in the intended to help the player learn about how pathogens work within present set of experiments. the context of playing the game. Computer-based narrative discovery learning games such as Second, Cache 17 was developed by Alan Koenig at Arizona State Crystal Island and Cache 17 are based on two interrelated hypoth- University, using game design principles with a focus on narrative eses concerning educational game design, the discovery hypothesis theme (Koenig, 2008). Development required approximately 6 and the narrative hypothesis. The discovery hypothesis is that months of programming time, and the game has undergone several students learn academic content more deeply when they must cycles of revision based on field testing. In Cache 17, the player views interact within a realistic learning context than when they are a brief introductory cinematic that lays out the story line about a simply told the academic content. The narrative hypothesis is that long-lost painting that may be found in an old bunker system dating students learn better from computer-based games that have strong back to World War II. The player’s job is to make his or her way narrative themes than from games that do not have strong narrative through the bunker system to solve the detective story about the themes. The present set of studies provides experimental tests

broadly. whereabouts of the painting, along the way constructing electrome- concerning the efficacy of the discovery hypothesis and the nar- chanical devices to help open doors (shown in Figure 2). The instruc- rative hypothesis for guiding the design of educational games. publishers. tional objective of the game is to help the players understand how various electromechanical devices work. allied The Discovery Hypothesis for Designing Educational disseminated its Lee et al. (2006) pointed to four reasons for incorporating Computer Games be of narrative themes such as these in computer games: They provide to

one surrogate experiences, reduce cognitive load, motivate game play-

not The Case for Discovery in Educational Computer Games or ing, and allow for a strong sense of presence. Yet, there is not a is sufficient body of empirical evidence for evaluating the educa- According to the discovery learning view, students learn better and tional effectiveness of narrative as a feature in computer games: when they can actively explore a learning environment than when user Association individual the Psychological of use American personal the the by for solely copyrighted is intended is document article This This

Figure 1. Screen shot from Crystal Island game displaying a problem-solving exercise aimed to guide the learner to the solution. NARRATIVE GAMES FOR LEARNING 237 broadly. publishers. allied disseminated its be of to one not or is and user Association Figure 2. Screen shot from Cache 17 game displaying the wet-cell battery construction and the PDA entry on wet-cell batteries. individual the Psychological of

use they are simply told what they need to know (Bruner, 1961; shown to have the potential to mimic real-world environments and Mayer, 2010; Papert, 1980), but research on the instructional thus the potential to create the benefits of anchored instruction.

American effectiveness of hands-on discovery methods in in-game environ- Teaching academic content within an authentic real-world context, personal the ments does not offer strong supporting evidence (Kirschner, a principle of anchored instruction, has shown promise in provid- the by Sweller, & Clark, 2006; Mayer, 2004). Supporters of discovery ing a deeper and more substantial education for students (Barab,

for learning note that most studies that do not support the effectiveness Hay, & Duffy, 1998; Brown, Collins, & Duguid, 1989; Hung, of discovery methods involve short learning spans and do not test 2002; The Cognition and Technology Group at Vanderbilt, 1990). solely for self-correction and self-regulation skills. An important new copyrighted is venue proposed for discovery learning involves narrative computer The Case Against Discovery in Educational games, because embedding a game within a story line is intended intended Computer Games

is to engage learners more deeply in game activity (Dickey, 2006; Gee, 2007; Lee et al., 2006; Neitzel, 2005; Shaffer, 2006). The In contrast, the distraction hypothesis holds that game material document

article discovery learning view predicts that students will learn better and and activities that are not directly related to the instructional This more easily by playing a narrative computer game than by receiv- objective can distract the learner from focusing on the key instruc- This ing direct instruction from a multimedia presentation devoid of a tional material in a lesson. Consistent with the cognitive theory of story line and opportunities for exploration activity. multimedia learning (Mayer, 2009) and cognitive load theory On the basis of past research, game proponents have proposed (Sweller, 1999), learners are able to process only a limited amount that ideal learning conditions for discovery learning include some of material in working memory at any one time. When a player guidance and a realistic context for learning (Gee, 2007; The uses too much processing capacity on understanding an interesting Cognition and Technology Group at Vanderbilt, 1990). Proponents story line and learning how to engage in activity in the game, further argue that computer-based adventure games have the po- insufficient processing capacity may be available to support the tential to encourage active and critical learning within exploratory player’s construction of a mental representation of the key instruc- conditions (Gee, 2007). Computer-based adventure game environ- tional material. In addition, when minimal guidance is provided ments tend to consist of easy beginning stages, which provide concerning the learning objectives, learners may not be able to generalizations that will be useful for future difficult stages, and figure out what they should attend to in the game. According to the thus the in-game limitations can become a form of guidance for cognitive theory of multimedia learning, the game environment in future hypothesizing and decision making (Gee, 2007). Finally, general may require the learner to engage in too much extraneous virtual environments (including computer games) have been processing, which is processing that does not support the instruc- 238 ADAMS ET AL.

tional objective (Mayer, 2009). If managing the virtual environ- game play, which in turn result in better learning. These motiva- ment causes too much demand for extraneous processing, over- tional features of games reflect the definition of motivation whelming the learner’s working memory, there will not be enough (Mayer, 2010) and are distinguishable from affective features of cognitive resources left for essential processing and generative games, such as being fun and enjoyable. For example, motivation processing. Essential processing is cognitive processing required for game playing is reflected in a learner’s willingness to continue to represent the essential material, whereas generative processing playing the game when given the option to do so or to persist in is deeper processing used to understand the material (Mayer, trying to overcome a challenge in the game, whereas affect for 2009). In some cases, cognitive processing that initially appears to game playing is reflected in a learner’s expressing a feeling of fun be extraneous could be useful in helping the learner develop and enjoyment while playing the game. The causal chain proposed effective strategies for meaningful learning. by the narrative hypothesis highlights player motivation as the mechanism by which adding a narrative theme to a computer- Using the Media Comparison Approach to Test based adventure game has its positive effects on learning. Accord- Predictions of the Discovery Hypothesis ing to the narrative hypothesis, students who play an educational game with a strong narrative theme will learn more than students The discovery hypothesis was tested with what Mayer and who play the same game without a strong narrative theme.

broadly. colleagues (Johnson & Mayer, 2010; Mayer, 2011; Mayer & The justification for including narrative theme in educational Johnson, 2010) call a media comparison methodology, in which computer games comes from reviewing mainstream adventure publishers. learning with one medium (e.g., a game) is compared to learning games and proposing that the same motivational techniques be with another medium (e.g., a slideshow). In Experiment 1, the applied to educational games (Dickey, 2006). Dickey proposed allied

disseminated media comparison approach involved comparing the learning out- its that engaging narratives induce high levels of participation from be of comes of students who learned about how pathogens work by players and provide scaffolding for problem solving during the to playing the Crystal Island game (narrative group) or by viewing a one game. From her review of adventure games, Dickey developed a not or matched slideshow that contained the same text and images used in design heuristic for integrating game design narrative into instruc- is the game to explain pathogens (slideshow group). In Experiment 2, tional games. Game design must include providing an initial chal- and the media comparison approach involved comparing the learning lenge; smaller obstacles, puzzles, minor challenge, and resources

user outcomes of students who learned how electromechanical devices that lead up to solving the main challenge; having established Association work by playing the Cache 17 game (narrative group) or by roles, such as the learner as the hero; dimensions for the game viewing a matched slideshow that contained the same text and environment that include physical, temporal, environmental, emo-

individual images used in the game’s resources to explain the devices (slide- tional, and ethical elements; a backstory that outlines the dimen- show group). According to the discovery hypothesis, the narrative the sions of the narrative’s environment; and cut scenes that support Psychological of group should perform better than the slideshow group on measures the narrative story line. Although Dickey postulated that these

use of learning outcome. According to the distraction hypothesis, the elements can produce an engaging interactive learning environ- slideshow group should perform better than the narrative group. ment, research is needed to determine whether these kinds of

American These predictions were tested in Experiments 1 and 2 (and Exper- elements will actually help foster learning. Although Crystal Island personal the iment 2 included a comparison between a nonnarrative game group and Cache 17 were developed and field tested by game design the by and a slideshow group). experts to be high-quality narrative discovery games, they may not

for be representative of all narrative discovery games; nor is Dickey’s The Narrative Hypothesis for Designing Educational model of game design the only approach to game design. solely Computer Games copyrighted is The Case Against Adding Narrative to Educational

intended The Case for Narrative Theme in Educational Computer Games is Computer Games In contrast, as described previously, the distraction view is document

article Research on learning from text investigates the role of narrative based on the idea that learners have a limited cognitive capacity so This theme. Graesser, Hauft-Smith, Cohen, and Pyles (1980) proposed they are able to engage in only a limited amount of cognitive This that narrative text is better retained in memory because the events processing at any one time during learning (Kirschner et al., 2006; and actions are more concrete and are organized in more cohesive Mayer, 2009; Sweller, 1999). According to the distraction hypoth- causal relationships, whereas in contrast, expository texts are usu- esis, a compelling story line can distract the learner from focusing ally more abstract and loosely organized. Correspondingly, one on the key instructional material in a lesson or can at least be argument for adding narrative to educational games is that narra- something that the learner has to ignore. For example, within the tive could help organize the educational material, making it easier research literature on learning from multimedia presentations, for the learner to remember. Two other proposed possible reasons there is some evidence that adding interesting but irrelevant nar- that narratives could increase learning are because they are usually rative (sometimes called seductive details) can reduce student easier to understand and they increase interest (Cunningham & learning of the core material (Harp & Mayer, 1997, 1998; Mayer, Gall, 1990). Griffith, Naftaly, & Rothman, 2008). According to the distraction Narrative is often considered an essential motivating element in hypothesis, adding a strong narrative theme will reduce student computer games (Lee et al., 2006; Marsh, 2010). Games with learning, because players use too much of their limited cognitive compelling story lines are intended to motivate the learner to capacity during learning on game content and activity that is not initiate game play, persist in game play, and exhibit intensity in relevant to the academic content. NARRATIVE GAMES FOR LEARNING 239

Using the Value-Added Approach to Test the was spreading through a research camp’s staff, what object was Predictions of the Narrative Hypothesis acting as the carrier of the disease, and whether anyone was responsible for the infections. Participants also completed three The narrative hypothesis was tested using what Mayer and quizzes given by the in-game characters on information being colleagues (Johnson & Mayer, 2010; Mayer, 2011; Mayer & taught within the game. Johnson, 2010) have called a value-added methodology, in which The learning environment for the slideshow was a Microsoft one instructional method is compared to another that is identical PowerPoint presentation containing information that was pre- except for one feature. In Experiment 2, the value-added approach sented within the game, excluding information about the game’s involved comparing the learning outcomes of students who learned story line (e.g., the mystery of the disease carrier, who was about electromechanical devices within the context of a version of responsible for the infections, names of the staff). All text and the Cache 17 game that had a strong narrative theme about a poster images relevant to learning about human diseases were detective case that had to be solved (narrative group) versus an taken directly from the game and presented on static slides. All otherwise identical version of Cache 17 that lacked a narrative three quizzes given in the game were also given during slideshow, theme (nonnarrative group). According to the narrative hypothesis, with each quiz being given after the relevant slideshow informa- the narrative group should perform better than the nonnarrative tion was presented. The slideshow was self-paced and included 28

broadly. group on measures of learning outcome. According to the distrac- slides of information. tion hypothesis, the nonnarrative group should perform better (or The participant questionnaire solicited demographic information publishers. no worse) than the narrative group. These predictions were tested including the participant’s age and gender. The questionnaire also in Experiment 2. solicited information concerning the participant’s previous knowl- allied disseminated

its edge of human biology, viruses, and diseases on a 5-point scale be of (from very high to very low) in response to the questions “Please

to Experiment 1

one rate your knowledge of human biology” and “Please rate your not or Experiment 1 featured a computer-based narrative discovery knowledge of viruses and infections.” Further information on is learning game called Crystal Island (Spires et al., 2010) in which previous knowledge was gathered using a biology background and players seek to discover the cause of a disease outbreak on a checklist in which participants asked to “place a check mark next

user remote island and are expected to learn how pathogens work in the to the items which are true about you” in response to the items: “I Association process of their game playing. We tested the discovery hypothesis have taken an upper level biology class (e.g., BIO 2) during high by taking a media comparison approach in which we compared the school,” “I have taken or are in the process of taking a class on

individual learning outcomes of students who learned by playing the game biology in college,” “I read/have read webMD to help understand

the (narrative group) and the learning outcomes of students who illnesses better,” “I enjoy watching shows on medical research,” Psychological of viewed a slideshow presentation containing identical information and “I know the difference between viruses and bacteria.”

use (slideshow group). The rationale for creating a slideshow with The questionnaire also solicited information concerning the images and words identical to those used in the game is to avoid participant’s perceived difficulty of the lesson and level of effort

American a confound in which the instructional content (i.e., words and during learning. Perceived difficulty of the lesson was solicited personal the pictures about the to-be-learned material) for the slideshow con- using a 7-point scale (from extremely difficult to extremely easy)in the by tained more or better information than that for the game or vice response to the question “How difficult was the educational expe-

for versa. As the presentation of content was designed specifically for rience you just had?” Effort put into learning the lesson was the game, retaining the same images and words for the slideshow solicited by the participant’s answer to the question “What level of solely should not advantage the slideshow condition. effort did you put into this educational experience?” on a 1 copyrighted

is (extremely high)to7(extremely low) scale. The four-question transfer test assessed the participant’s ability intended Method

is to solve novel problems (Mayer, 2010). Each question was pre- Participants and design. The participants were 42 college sented on its own on a computer screen with a space below for the document

article students from the University of California, Santa Barbara (UCSB), student to type an answer. The amount of time left to complete a This with 21 serving in the narrative group and 21 serving in the question was displayed at the bottom of the screen. Before each of This slideshow group. The mean age was 19.4 years, 60% were men, the transfer test questions was given, participants were informed and the mean score on biological knowledge was low. Participants on the computer screen how much time they would have to were recruited through the UCSB Psychology Subject Pool, and complete the question. Three minutes were allotted for the first they either were paid $20 or received course credit for their three questions, and 6 minutes were given for the final question. participation. Examples of transfer questions are “Imagine that you have two Materials and apparatus. The learning environment for the microbes that are different sizes. Please explain how you could narrative group consisted of an educational computer game called identify each microbe if you know that one is a virus and one is a Crystal Island (Spires et al., 2010) in which participants explored bacterium” and “Your school is on lock-down because an illness is a 3-D environment via a first-person perspective in which they spreading rapidly throughout the student body. You have a micro- could talk to the in-game characters, interact with one of the scope in your classroom and your teacher has isolated a sample of in-game computers, pick up objects, and read posters on the wall. the pathogen. What do you need to know about the pathogen and A screen shot of one of the posters can be found in Figure 3. All microscope, in order to determine if you can look at the pathogen in-game verbal communication was through text printed on the through your microscope?” These transfer questions were de- screen. In this game the participant had to discover what disease signed to test whether the participant could apply the information 240 ADAMS ET AL. broadly. publishers. allied disseminated its be of to one not or is and user Association individual the Psychological of Figure 3. Poster on the wall from the Crystal Island game. This information was also presented on a slide in the slideshow condition. use American personal

the presented in the game to new problems and situations that were not d. All of these are true about carcinogens. the by given in the game or slideshow. All four of the questions required

for some form of problem solving on the learner’s part, such as With a light microscope the smallest sized object you could developing a plan for testing a hypothesis or determining how to see would be:

solely distinguish between different classes of pathogens. Participants in copyrighted the slideshow presentation condition were not presented with a. 100 nanometers is ء questions during their lesson, and therefore they did not have

intended b. 200 nanometers experience in developing hypothesis-testing plans or distinguish- is ing between pathogens. The discovery game condition did include document c. 300 nanometers a scenario similar to those in the transfer questions (i.e., determin- article

This ing which pathogen was affecting a camp full of sick researchers); d. 400 nanometers This however the transfer problems required the learners to apply their problem-solving skills and acquired knowledge to new questions Each question was presented on its own on a computer screen, with analogous to the game scenario. the amount of time remaining to complete the multiple-choice test The retention test assessed the participants’ retention of funda- always being displayed at the bottom of the screen. The retention and mental facts. The retention test consisted of 20 multiple-choice transfer tests corresponded to those used by the team that developed questions, such as these: and field-tested the Crystal Island game (Spires et al., 2010). The apparatus consisted of five Dell computer systems with Which of the following is not true about carcinogens? 19-in. monitors and Sony headphones. The game was built through a modification of the game Half-Life 2, produced by the Valve a. They promote or facilitate cancer. Corporation. The slideshow quizzes, demographic questionnaires, b. They cause diseases that affect the cells of patients’ transfer test, and retention test were given on the computers using bodies. the software MediaLab. The final quiz for the slideshow replicated the in-game quiz using the software Game Maker, produced by .YoYo Games ء.c. They spread from one person to the next person NARRATIVE GAMES FOR LEARNING 241

Procedure. Participants were tested in groups of two to three Table 1 per session. Each participant was randomly assigned to a treatment Mean Scores and Standard Deviations for Narrative and group and was seated at an individual cubicle in front of a com- Slideshow Groups on Four Measures, Experiment 1 puter. First, the experimenter presented oral instructions stating that there would be two phases to the experiment. The narrative Group group was told that for the first phase they would be playing a Narrative Slideshow video game, given some basic orientation to what the game would group group be about, and given basic orientation to the headphones that they would use (to hear background sounds in the game). The slideshow Measure MSDMSD pd ء group was told they would be viewing a slideshow presentation Retention 13.1 3.1 16.8 2.3 .001 1.37 and that most directions would be provided within the slideshow. Transfer 11.8 4.2 14.3 4.5 .074 0.57 0.93 ءBoth groups were told that they had up to 75 minutes to Difficulty 4.0 1.3 5.0 0.9 .005 complete their educational experience (i.e., narrative or slide- Effort 3.6 0.9 4.1 1.3 ns 0.49 show). Both groups were told that for the second phase they would Note. An asterisk indicates significant difference (p Ͻ .05) between be answering questions about their educational experience and narrative group and slideshow group. ns ϭ nonsignificant.

broadly. answering questions about the information learned in their educational experience. All participants completed the game or slide- publishers. show within the time period. At the end of the 75 minutes, participants in the narrative group were handed a sheet that showed ratings of difficulty and effort. Students in the slideshow group allied disseminated

its the answers for the mystery, regardless of whether or not they had scored significantly better than did students in the narrative group be of solved the mystery. The sheet included the name of the pathogen on the retention test, t(40) ϭ 4.37, p Ͻ .001, d ϭ 1.37, and to ϭ ϭ ϭ one affecting the residents of Crystal Island, the three major symptoms marginally better on the transfer test, t(40) 1.84, p .07, d not or of the pathogen, how it was communicated, and whether or not

is 0.57. In terms of self-reported learning measures, students in the someone was responsible for the infection. narrative group rated their learning experience as significantly and When a participant completed the first phase of the experiment, more difficult than did students in the slideshow group, t(40) ϭ

user the experimenter went to the participant’s computer and started the 2.96, p ϭ .005, d ϭ 0.93. The narrative group also reported more Association computer-based questioning software for the demographic survey, effort in learning than did the slideshow group, although this transfer test, and retention test. Before the demographics survey difference was not statistically significant, t(40) ϭ 1.56, p ϭ .127,

individual was given, participants were informed onscreen that they would be d ϭ 0.49. These results offer no support for the claim of the

the taking a short survey about themselves. This questionnaire was discovery hypothesis that narrative computer games are superior or Psychological of presented onscreen for the participants to fill out at their own pace. easier venues for academic learning than conventional instruc-

use Once a participant had finished the questionnaire, the computer tional media. automatically moved on to the transfer test. For the first three Summary. The main findings are that students who learn American questions participants were given 3 minutes to answer each ques- about the biology of diseases by playing a narrative computer personal the tion, and for the fourth question participants were given 6 minutes game learn less and report more difficulty in their learning expe- the by to answer the question. The first three questions were shorter and riences than do students who receive a straightforward PowerPoint for required only one step in order to answer the question. The fourth presentation that is devoid of the game narrative and hands-on question contained multiple parts and asked the participants to activities. The results also demonstrate that even though the game solely describe a scientific procedure and use the information they had copyrighted condition included material and situations similar to those in the

is acquired on pathogens; therefore, more time was allotted for this transfer problems, learners in the game group did not perform question. Once a participant finished the transfer test, the computer intended better on these items. These findings are not consistent with the

is automatically moved on to the multiple-choice retention test. Be- discovery learning view, which holds that students learn best when fore the retention test began, participants were informed on the document they can engage in hands-on exploration and discovery of aca-

article computer screen that they would have 10 minutes to complete the This multiple-choice questions. Once finished with the retention test, demic content; these findings are consistent with the distraction This participants automatically received their debriefing onscreen. view, which holds that extraneous material and activity can over- load the learner’s information processing system and thereby de- tract from learning (Mayer, 2004). This study is an example of a Results media comparison study (Johnson & Mayer, 2010; Mayer, 2011; Scoring. The retention test was scored by tallying the number Mayer & Johnson, 2010), in which the instructional effectiveness of correct answers out of 20 items on the test. The transfer test of one delivery medium is compared to that of another. It is subject answers were scored by tallying the number of the correct idea to Clark’s (2001) criticism that instructional media do not cause units included in the participant’s answers across all four prob- learning, but rather instructional methods cause learning. One lems, out of a possible 31. The transfer tests were scored by an problem with this study is that it did not specifically focus on the alternate rater to check for rating reliability, and any differences role that narrative played in learning. Although it involved a were resolved by consensus. narrative game, there was no way to determine if narrative helped Do students learn better from a narrative computer game the learner in any manner. This issue is addressed in the next study, than from conventional media? Table 1 shows means and in which narrative and nonnarrative versions of a game were standard deviations of the two groups on retention, transfer, and compared to learning with a more traditional PowerPoint lesson. 242 ADAMS ET AL.

Experiment 2 Koenig’s study in order to give the fairest possible test of the narrative hypothesis. Experiment 2 featured a computer-based narrative discovery learning game called Cache 17 (Koenig, 2008) in which players Method seek to solve a mystery about the fate of a cache of missing Participants. The participants were 171 undergraduate col- paintings that disappeared during World War II. Players must lege students (81 male and 90 female) who were recruited through find their way through an underground bunker, occasionally the UCSB Psychology Subject Pool. Participants fulfilled a course building electricomechanical devices to help open doors and requirement by participating. The age of the participants ranged vaults and thereby learning something about how these devices from 17 to 26 years, with a mean age of 19.1 years and a standard work. We tested the discovery hypothesis by taking a media deviation of 1.3. comparison approach, in which we compared the learning out- Design. This study employed a pretest–posttest experiment comes of students who learned by playing the game (nonnar- method with participants being randomly assigned to one of three rative group) and the learning outcomes of students who viewed conditions: narrative game, nonnarrative game, and no game slide- a slideshow presentation containing identical information show presentation. Fifty-seven participants were in the narrative (slideshow group). We tested the narrative hypothesis by taking group, 58 participants were in the nonnarrative group, and 56 broadly. a value-added approach, in which we compared the learning participants were in the slideshow group.1 outcomes of students who played a version of the game with a publishers. Materials. The narrative and nonnarrative versions of the narrative theme (narrative group) or one without a narrative game used for this study were updated versions of those used in theme (nonnarrative group). allied Koenig’s (2008) research but without any of the pregame or disseminated

its Previous research examining the effect of narrative on learning in-game training mechanisms. The game is a single-player, third- be of with text has not had encouraging results. Two studies conducted

to person perspective, 3-D computer video game called Cache 17.

one by Graesser et al. (1980) found that students produced significantly

not The game was constructed using Visual Basic.Net along with the or higher retention scores for immediate cued recall and delayed is TrueVision3D game engine. Three-dimensional models used recognition tests from text with narrative than text without narra-

and within the game were created with Autodek’s 3D Studio Max 8 tive, but there was a methodological complication in which the and with Valve Corporation’s Valve Hammer Editor software. The user content of the passages differed. A later study by Wolfe and

Association educational purpose of the game was to teach the learners about Mienko (2007) found that when controlling for text content, there principles concerning electrical circuits and electrical energy was no difference between the narrative and nonnarrative text within the context of figuring out how to construct three electro-

individual groups on either free-recall or posttest measures. A similar study mechanical devices.

the by Cunningham and Gall (1990) examined the idea that learning The environment for both versions of the game is an under- Psychological of differences occur because students understand narrative texts bet- ground bunker, which includes multiple rooms, hallways, and

use ter. After creating narrative and nonnarrative textbook chapters objects, such as shown in Figure 2. The player is given various about archaeology and controlling for understandability, they resources to use to navigate through the game. All of them can be American found that there was no significant difference between the two

personal accessed through a menu bar, which is displayed across the bottom the textbook conditions on learning outcomes. Post-experiment inter- part of the screen. One resource is a map of the underground the by views showed that the majority of students preferred the narrative bunker to facilitate navigation of the environment. There is also a for text, although some said it provided too much unnecessary infor- Notes button, which provides the players with a file folder that mation on which they were not tested. Overall, this review of the includes information about the current task that must be solved. A solely text learning literature does not yield strong evidence for the copyrighted simulated personal digital assistant (PDA) provides information

is instructional effectiveness of adding a narrative theme to a pas- about all of the electrical devices and other educational informa- sage. Some researchers have proposed that multimedia learning intended tion that would assist participants with solving each of the prob-

is environments may benefit from narrative but have noted that this lems. The participant is also given a multimeter, which can be used area of research needs empirical studies comparing narrative to document to measure the electrical voltages of their devices. Finally, there is

article nonnarrative conditions, adequate control for content similarity a button that can be used to control the actions of the participant’s This across the conditions, and a focus on reliable measures of learning

This avatar, including picking up and dropping items. outcomes (Akkerman, Watson, Morrison, & Belfore, 2009; Lau- The participant had to complete three tasks in order to complete rillard,1998; Zumbach & Mohraz, 2008). the game. The first task consisted of constructing a wet-cell battery To date, only one previous study has involved a methodologi- in order to open a door (this is the task featured in Figure 2). This cally sound experiment evaluating the effects on learning out- required the participants to utilize information from the galvanic comes of adding narrative to a computer game. In particular, series of metals table to achieve a certain voltage using two Koenig (2008) explored the question of whether the presence of different materials. After achieving the correct voltage, the partic- narrative could facilitate learning in an action adventure game ipants had to connect the metals to an electrical door panel to gain called Cache 17. Koenig did not find a significant main effect for access to a room that contained materials for the next task and narrative on learning outcomes but did report a nonsignificant narrative elements for the narrative group. There were 30 possible trend in which the narrative group scored higher (71%) on the posttest than did the nonnarrative group (65%). Narrative led to a significant increase over the nonnarrative groups in enjoyment of 1 Eight participants in the narrative group and eight participants in the the game. In the present experiment, we used an updated version nonnarrative group failed to complete the game within the 2-hr time period of Cache 17 and solicited a much larger sample size than in and were not included as participants in the study. NARRATIVE GAMES FOR LEARNING 243

ways to combine the available materials, but only two combina- recover the painting; he discovers that Kate is a double agent right tions would cause the door to open. before he escapes from the bunker. In contrast, the nonnarrative The second task was a subtask for the third and required the participants were not given a cut scene and were simply asked to player to recharge a lead-acid battery using a direct current (DC) complete the tasks without any overarching goal. The physical electric motor and a gas-fired Stirling engine. The participants had bunker environment was the same for the nonnarrative group to connect the electric motor to the engine and then use jumper except there were no other characters in the bunker and the cables to connect the battery to the motor. The mechanical power documents they read contained information only about the electri- created by the engine is utilized by the DC motor to create energy cal devices they had to use without additional information about to recharge the battery. Working on this task was intended to help the story line. students learn the reciprocity between DC electric motors and DC The PowerPoint presentation consisted of 25 slides containing electric generators. Only one possible configuration for this task material taken from the game’s simulated PDA, such as shown in was possible; therefore, data on number of attempts and time Figure 4. The presentation was broken up into three sublessons, were not collected for this second task. Once the player has the which corresponded to the three tasks from the game. At the end charged battery, he or she can complete the third task, which of each sublesson the participants were given a question similar to involves creating an electric circuit. Only by creating the cor- the task completed in the game (e.g., “How would you construct a broadly. rect combination of serial and parallel circuitry can one produce wet-cell battery with the given materials?”). The participants in the correct voltage to power a motor to lift a heavy load. The publishers. this group were not asked to answer the questions. Answers for the materials needed for this task included the recharged battery, an problems were given on the slide following the problem slides, additional already charged battery, four jumper cables, and the allied based on images and text from the game. disseminated

its electric motor. A total of 12 possible combinations could be

be Measures. The measures included a demographic survey, of created using these materials, but only two of them would cause to pretest, posttest, and evaluation survey and were the same as those

one the heavy load to lift.

not used by Koenig (2008). The first menu in the game contained a or For the narrative condition, a cut scene was presented at the is beginning of the game to provide a background story for the demographic survey, which included questions on age, gender, and and protagonist and his ultimate goal. This cut scene lasted 3 minutes game-playing experience.

user and explained why the protagonist was in the game environment. The pretest contained 17 computerized multiple-choice ques- Association The story involves an insurance investigator named Alex who is tions, such as these: attempting to locate a stolen Raphael painting from World War II. Which of the following best describes the function of an

individual In the cut scene, the investigation leads Alex and his partner Kate electric motor?

the to an old underground bunker. The cut scene ends after Alex Psychological ء of descends into the bunker, and at that point the player gains control a. It converts electrical energy into mechanical energy.

use of Alex. The players in the narrative group continue to interact with nonplayer characters, like Kate, throughout the game by b. It converts mechanical energy into electrical energy. American following dialogue, reading about their actions in documents, or personal the finding items that belonged to them. The participants moved c. It converts potential energy into electrical energy. the by through the narrative by completing the tasks in the bunker and

for receiving more information. At the end of the story Alex does not d. It converts chemical energy into electrical energy. solely copyrighted is intended is document article This This

Figure 4. Slide from slideshow covering the academic material in the Cache 17 game. 244 ADAMS ET AL.

The questions covered topics such as different forms of energy, start the program. Participants were then prompted by the program batteries, motors, and electric circuits. The purpose of the pretest to fill out the demographic information. After the demographic was to obtain a measure of each student’s prior knowledge. questions were answered, the participants were given the pretest. The posttest contained 17 computerized multiple-choice ques- Questions appeared one at a time, and the participant had to click tions and was designed to assess each student’s learning outcome. a button to advance to the next question. Once the pretest was The posttest questions ranged from those mainly testing retention completed the actual game began. Participants in the narrative (i.e., simply requiring the participant to recall information pre- group viewed a 3-min video cut scene before starting the game. sented in the game to get the correct answer) to transfer questions The cut scene contained no educational information and was that required applying learning to new similar situations, such as designed to introduce the player to the main character and the the following: backstory. The regular game began after the cut scene. Once the participants started the actual game, the computer kept track of If you were constructing a small wet-cell battery and wanted in-game measures (e.g., time taken to complete each task, amount to maximize the voltage produced, which of the following of attempts per task, times the PDA information resource was would be the best choice for electrodes? accessed, and the total time used to complete the game). Once the game was completed, a window appeared asking a. two copper pennies broadly. participants to type in a summary of the perceived narrative ء publishers. b. a copper penny and a sheet of aluminum foil regardless of the condition to which they were assigned. These summaries were assigned narrativity scores ranging from 1 to 4, allied c. two pieces of aluminum foil with 4 signifying high narrative awareness and 1 denoting little to disseminated its no narrative awareness. The posttest was given after the participant be of d. a copper penny and salt water

to completed the summary and was administered the same way as the one pretest. Finally, participants were asked to fill out a game play not or After they finished the posttest, participants in the game conditions is experience survey. were given a computerized survey to evaluate their game play

and Participants in the slideshow group were seated at the same experience. The survey contained 12 Likert-style questions using a computer workstation cubicles and asked to read and sign a con- user 5-point scale (5 ϭ Strongly Agree,1 ϭ Strongly Disagree). The

Association sent form if they wished to participate in the study. Participants in questions addressed issues such as game difficulty (e.g., “The this condition were then asked to complete the same demographic game environment was intuitive and easy to use”), perceived information and pretest as completed by the game groups. Upon

individual educational value (e.g., “I learned a lot about how electrical completion, participants were instructed to open and review the

the devices are used”), and overall likability (e.g., “Overall, I enjoyed slideshow lesson. The lesson was self-paced, and participants were Psychological of playing this game”). told that they could have as much time as they wished to review

use Apparatus. The apparatus consisted of five Dell desktop the lesson and to feel free to reread the lesson or particular slides computers with 19-in. monitors and Cyber Acoustics headphones, if they found these difficult. Once the participant finished review- American each housed on a worktable within a cubicle.

personal ing the lesson, the experimenter started the posttest. the Procedure. Participants were randomly assigned to one of the the by three conditions. Participants in the slideshow group were tested for separately from those in the game conditions due to time differ- Results ences and the need to keep the participants blind to the purpose of solely the study. Upon their arrival, the participants were seated at a Scoring. For purposes of analyzing what was learned, we copyrighted focused on a measure of learning outcome: is computer workstation cubicle and asked to read and sign a consent form if they wished to participate in the study. intended Posttest score is the number of correct answers on the posttest is For the game groups, the experimenter gave each participant a paper copy of the game instructions and drew the participants’ (ranging from 0 to 17). document

article attention to the B key, which would take the avatar back to the

This For purposes of analyzing how learning occurred within the starting point if they got stuck at any point during the game (e.g.,

This game between the two game groups, we focused on six measures walking into a wall and being unable to get out). The experimenter also advised them to take advantage of the menus at the bottom of of learning process: the screen to assist them with solving the game problems. Partic- Wet-cell time is the amount of time from the start of the game ipants were asked to wear headphones during the experiment in to the completion of the wet-cell task. order to hear audio portions of the game and to block out extraneous noises. Heavy lift time is the amount of time from the start of the Instructions were given prior to starting the program because the game to the completion of the heavy-load lifting task. demographic and pretest portion of the experiment were self- paced. Only a cursory reading was required, along with the exper- Total time is the total amount of time taken to complete all of imenter drawing attention to particular aspects of the game that the tasks and navigate to the exit of the game environment. pilot participants had found difficult. The instruction sheet for the game was available throughout the entire experiment, so that Wet-cell attempts is the number of attempts the participant participants could review it at any point in time. Once participants made before constructing a wet-cell battery with the correct had reviewed the instruction sheet, the Next button was pressed to voltage to open a door. NARRATIVE GAMES FOR LEARNING 245

Heavy lift attempts is the number of attempts the participant ANCOVAs with pretest as a covariate revealing that the slideshow made before creating a circuit with enough electric current to group performed significantly better than the narrative group on power a motor to lift a heavy stone. posttest scores, F(1, 110) ϭ 4.71, MSE ϭ 23.80, p ϭ .032, d ϭ 0.31. In order to more directly address the discovery hypothesis, PDA use is the number of times the player accessed the PDA we focused on the whether the nonnarrative group outperformed in order to obtain information. the slideshow group on a measure of learning outcome. With respect to the discovery hypothesis, there was a significant differ- Two measures were included to examine participants’ level of ence between the nonnarrative group and the slideshow group on enjoyment in playing the game and their level of awareness of a posttest scores, F(1, 111) ϭ 9.54, MSE ϭ 52.98, p ϭ .003, d ϭ narrative theme in the game: 0.47, with the slideshow group scoring higher than the nonnarrative group. Satisfaction rating was based on the 12-item attitude survey With respect to that narrative hypothesis, there was no signifi- given at the end of the game. The satisfaction rating was cant difference between the two game groups for posttest scores, based on the average score from the twelve 5-point-scale F(1, 112) ϭ 1.00, MSE ϭ 5.74, p ϭ .32, d ϭ 0.16; therefore, the Likert questions. two game groups were collapsed into one game group. Using an

broadly. Narrativity score was based on story descriptions given by ANCOVA with pretest as a covariate, we found a significant difference between the combined game groups and the slideshow

publishers. participants after completion of the game, which were as- ϭ ϭ ϭ signed narrativity ratings by two independent raters, based on group on posttest scores, F(1, 168) 9.03, MSE 49.15, p .003, d ϭ 0.39, favoring the slideshow group. allied the participant’s awareness of narrative theme in the game. disseminated its Ratings ranged from a score of 1, which were assigned to In terms of how long it took to complete the lessons, an be of ANCOVA revealed significant differences among the three

to participants with no awareness of a narrative, to a score of 4, ϭ ϭ Ͻ one for high awareness of a narrative theme. The interrater reli- groups, F(2, 168) 77.42, MSE 19,838.05, p .001. A Tukey not or test (p Ͻ .05) showed that the slideshow group took less time to is ability for the raters was found to be ␬ϭ.84, p Ͻ .001. learn the material than either of the game groups, which did not

and Scores were averaged between the two raters for analysis. differ from one another. Overall, as in Experiment 1, there was no

user Are the groups equivalent on basic characteristics? A evidence in support of the discovery hypothesis’s prediction that Association preliminary issue concerns the degree to which the narrative, learning from a hands-on game is superior to learning with con- nonnarrative, and slideshow groups were equivalent on basic de- ventional media as a venue for learning.

individual mographic characteristics. The groups did not differ significantly Does adding a narrative theme improve learning with an

the on average age, F(2, 168) ϭ 0.56, p ϭ .57; pretest score, F(2, educational computer game? As mentioned in the previous Psychological of 169) ϭ 0.28, p ϭ .76; or the proportion of men and women, section, there was no significant difference for learning outcomes 2 use ␹ (1) ϭ 0.57, p ϭ .45. between the two game conditions. In order to further examine the Do students learn better from a narrative computer game narrative hypothesis, we compared the narrative group and non-

American than from conventional media? Table 2 shows the means and narrative group on measures of total learning time, learning pro- personal the standard deviations of the three groups on pretest, posttest, and cess and game satisfaction ratings. Concerning learning outcome the by total learning time. Overall, a t test comparing performance on the measures, the narrative and nonnarrative groups did not differ for pretest to performance on the posttest revealed that participants significantly on posttest score with pretest being used as a cova- had significant learning gains across the three conditions, t(170) ϭ riate, F(1, 112) ϭ 1.00, MSE ϭ 5.74, p ϭ .32, d ϭ 0.1.The solely 20.90, p Ͻ .001, with a mean pretest-to-posttest gain of 4.5 points. narrative and nonnarrative groups did not differ significantly on copyrighted ϭ ϭ ϭ ϭ is Concerning learning outcome measures, separate analysis of co- total learning time, F(1, 112) 1.46, MSE 543.07, p .23, d variance (ANCOVAs) with pretest scores as a covariate revealed a 0.22. Concerning learning process measures, the narrative and intended

is significant difference among the groups on posttest scores, F(2, nonnarrative group did not differ significantly on any of the ϭ ϭ ϭ in-game process measures except that the narrative group (M ϭ document 167) 5.04, MSE 27.45, p .007. In order to examine the

article differences with respect to a possible interaction between the 12.12, SD ϭ 7.48) used the PDA to look up information signifi- This discovery hypothesis and narrative hypothesis, we conducted cantly more times than did the nonnarrative group (M ϭ 9.21, This SD ϭ 5.15), F(1, 113) ϭ 5.94, MSE ϭ 244.43, p ϭ .02, d ϭ 0.45. It is unlikely that players used random trial and error as their Table 2 solution strategy to accomplish the in-game tasks: For example, of Mean Scores and Standard Deviations for Slideshow, Narrative, the more than 30 possible combinations for attempting to build a ϭ and Nonnarrative Groups on Three Measures, Experiment 2 wet-cell battery, players made an average of 2.63 attempts (SD 3.83), and of the 12 possible ways to arrange batteries and jumper Group cables for the heavy-load lifting task, players averaged 1.82 attempts (SD ϭ 1.69). Overall, there is no strong evidence for the Slideshow Narrative Nonnarrative narrative hypothesis, which predicts that learning is improved Measure M SD M SD M SD when a narrative theme is added to an educational computer game. In terms of game satisfaction, the narrative group (M ϭ 3.58, Pretest score 7.00 2.46 7.32 2.76 7.33 2.75 SD ϭ 0.73) did not report significantly higher game satisfaction Posttest score 12.39 2.46 11.61 2.59 11.17 2.77 ratings than the nonnarrative group (M ϭ 3.34, SD ϭ 0.79), F(1, Time (min) 19.66 3.99 54.09 21.56 49.74 16.82 113) ϭ 2.74, MSE ϭ 1.60, p ϭ .10, d ϭ 0.31. 246 ADAMS ET AL.

As expected, participants in the narrative group (M ϭ 2.75, this proposed causal chain. The theoretical rationale for excluding SD ϭ 0.96) provided more detailed narrative descriptions of the narrative in games is that it will distract the player from the game’s story and therefore received significantly higher narrativity educational content of the game; that is, it can create extraneous scores than the nonnarrative participants (M ϭ 1.46, SD ϭ 0.83), cognitive processing that leaves inadequate cognitive capacity for F(1, 111) ϭ 58.08, MSE ϭ 46.87, p Ͻ .001, d ϭ 1.43. This essential and generative processing (Mayer, 2009). Participants in narrativity effect provides a validity check confirming that players the narrative condition performed just as well as those in the were sensitive to the narrative theme manipulation, even though nonnarrative condition; therefore, we did not find evidence to the narrative theme did not affect learning outcomes. support this mechanism either. Perhaps the motivating benefits of Summary. As in Experiment 1, students learned better with narrative theme were canceled out by increased extraneous cogni- conventional media than with a narrative game, thereby suggesting tive processing. that the discovery hypothesis did not apply to computer game design in the present study. The primary new contribution of Methodological Contributions Experiment 2 is that adding a narrative theme to an educational computer game did not significantly improve student learning, The present study used what can be called a media comparison which suggests that the narrative hypothesis did not apply to approach (Johnson & Mayer, 2010; Mayer, 2011; Mayer & John-

broadly. computer game design in the present study. son, 2010), in which the goal is to compare learning the same content with different media such as by comparing the learning publishers. General Discussion outcomes of the slideshow and narrative groups. It also used what can be called value-added approach (Johnson & Mayer, 2010; allied disseminated

its Mayer, 2011; Mayer & Johnson, 2010), in which the goal is to

be Empirical Contributions of determine which features of games can affect learning, such as by to

one Concerning the discovery hypothesis, across two independent comparing the learning outcomes of the narrative and nonnarrative not or groups. This set of experiments demonstrates that the media com- is experiments, students learned academic content significantly better by viewing a slideshow presentation than by playing a hands-on parison approach and value-added approach can be useful tools in and narrative adventure game. The empirical evidence shows that future game research aimed at pinpointing “what works” (and what

user asking students to play the two narrative games used in these does not work) in game design. Association experiments was not the most efficient way to promote learning of academic content. Concerning the narrative hypothesis, adding a Practical Contributions

individual narrative theme to the computer-based adventure game used in this

the study did not significantly improve learning of academic content. Although strong claims are made for the benefits of discovery Psychological of These findings do not prove that all narrative discovery games are and narrative in improving the learning effectiveness of games,

use ineffective; rather, they show that the two well-designed narrative these experiments do not provide strong evidence to support these discovery games we used in this study were less effective than claims. An important practical implication for game designers is

American corresponding slideshows in promoting learning outcomes based these studies do not provide strong evidence to support incorpo- personal the on transfer and retention of the games’ academic content. rating high levels of discovery-based activity and a well-developed the by narrative theme in educational games. An important practical for Theoretical Contributions consideration, however, is that even if games are less effective than direct instruction in improving performance on immediate solely The theoretical rationale for incorporating hands-on discovery tests, games can serve a useful educational role when students copyrighted

is activity in a learning episode is that it promotes cognitive activity choose to play games during their free time, which otherwise in the learner, resulting in better learning outcomes. The rationale would not be used for educational purposes. intended

is for using direct instruction is that engaging in hands-on discovery activities can create extraneous cognitive processing during learn- document Limitations and Future Directions

article ing, thereby reducing cognitive capacity available for attending to This the relevant academic content of the learning episode. In the Why did hands-on activity in a discovery environment result in This present experiments, there was no evidence that hands-on discov- poorer learning than a simple slideshow presentation? In the ery activity improved learning, and thus no support for the dis- games, learners may have devoted a considerable amount of their covery hypothesis. To the contrary, the results support the distrac- cognitive processing to material that was not related to the instruction hypothesis, which holds that aspects of game playing can tional goal, such as determining how to control the movement of distract the learner from the academic content of the lesson. This the character or what to look at in a complex display. For nongame conclusion is strengthened by the fact that narrative games in both players, simply learning how to navigate a game environment is a experiments included a considerable amount of guidance, so play- learning task that detracts from actual instructional goal. Perhaps ers were exposed to the academic content. learners need more guidance within the game, such as asking Concerning the narrative hypothesis, the theoretical rationale for learners to fill out a worksheet as they play, asking learners to incorporating narrative into games is that it will increase player summarize their learning, or providing some sort of in-game motivation to play the game, which will result in better learning. signaling to guide the learner’s attention. Future research is needed There was a nonsignificant trend showing that participants in the to determine how to improve learning efficiency in a game envi- narrative condition did enjoy the game slightly more than the ronment. Finally, it would useful to determine if support for the nonnarrative participants, but learning outcomes did not support discovery hypothesis would be obtained when people play a game NARRATIVE GAMES FOR LEARNING 247

for 20 to 40 hours and are tested for self-regulation strategies and between game content and learner knowledge. Another limitation far transfer. of Crystal Island is that games generally do not contain embedded Why did adding a narrative theme fail to improve learning in the quizzes, so the efficacy of this approach warrants further investi- present game environment? We consider explanations based on the gation. In addition, the game module used in this study took a little narrative theme we used, the game we used, and the measures we over an hour to complete. If educators wish to use longer games used. The narrative hypothesis proposes that adding a narrative can that incorporate multiple lessons, a narrative may be an effective increase learning through increasing motivation, increasing under- tool, which can tie information together. Future work should also standability, or assisting the learner in organizing the material by address whether asking players to summarize the narrative theme using the story as a cognitive tool. First, the narrative theme in the could have detracted from their posttest performance in Experi- present study (e.g., solving a mystery) was unrelated to the learn- ment 2. The games in this study were played in a lab environment, ing objective of the game (e.g., figuring out how to use electro- so future research is needed to determine the most effective role of mechanical devices). According to Wolfe and Mienko (2007), games as an adjunct to normal classroom activities. when using a story to convey information it is important that the Third, the learning outcome measures may not have been sen- to-be-learned material is well integrated into the overall narrative; sitive enough, because some were based on multiple-choice rather otherwise, the learners may pay more attention to the story but than open-ended transfer questions, and the learning process mea-

broadly. ignore the factual content. Due to the disconnect between the sures may not have adequately tapped the player’s level of moti- narrative and the educational material in Cache 17, Experiment 2 vation. It may be particularly useful to consider well-designed publishers. may only have tested the motivating element of the narrative measures of transfer as most appropriate for testing the instruc- hypothesis. Although the theme may have been somewhat moti- tional effectiveness of game playing. In future research it would be allied disseminated

its vating, it was also distracting, leaving no net gain in learning useful to collect more fine-grained measures of game play includ- be of effectiveness. Without an educationally relevant narrative, partic- ing measures of the player’s sense of presence, enjoyment, moti- to

one ipants may not have seen the story as a useful cognitive tool with vation, and educational focus, as well as detailed measures of not or which to organize the material and increase coherence between the learning outcome. In addition, possible future directions for re- is separate facts (Dickey, 2006; Graesser et al., 1980). Future research on narrative in educational games might include examining and search should compare the effects of narrative themes that are whether they affect long-term retention of information. If the

user relevant and irrelevant to the learning objective to determine narrative provides cues that are more easily recalled at a delayed Association whether a relevant narrative theme could help learners organize the test, participants may retain more information. The narrativity material and encourage learning. Although Crystal Island con- measures may also require reexamination. Participants were asked

individual tained a theme that was closely related to the instructional goal to describe the narrative whether or not they were in the narrative

the (i.e., learning about pathogens while the player is trying to deter- condition. Some participants gave very long and detailed summa- Psychological of mine which pathogen has infected a research group on an island), ries but others offered only a few sentences about the actual or

use Experiment 1 did not include a condition in which the game was perceived plot, even when in the narrative condition. Instead of played without the narrative. A future study that includes this reflecting the narrative awareness of the participants, this may

American condition may help determine whether narrative relevance affects have been more a measure of the participant’s motivation level. In personal the learning outcomes. In addition, although the effects of adding the future, multiple-choice or short-answer questions about specif- the by narrative to an education game may not be apparent for immediate ics of the plot may be a better measure of which participants knew

for posttests, a relevant narrative may help organize material so that the story within the narrative group. performance on a delayed posttest would increase. solely Second, the game may have been somewhat tedious, in that the

copyrighted Conclusion

is players had to expend much effort in trivial tasks such as walking from place to place, picking up and carrying objects, and navigat- These findings contribute to the small but growing research base intended

is ing the underground bunker. The number of items they could carry of scientifically rigorous studies on the instructional effectiveness was limited to two items, but all three of the tasks required that the of educational games. In a wide search of thousands of games document

article participants use four to six items. This meant that the player would articles published in peer-reviewed journals, O’Neil et al. (2005) This have to walk back and forth from a supply area to the location of found only 19 that met basic scientific standards and concluded This the task multiple times in order to deposit the needed items. that “many studies claiming positive outcomes appear to be mak- Furthermore, many of the items, such as jumper cables, had to be ing unsupported claims for the media” (pp. 461–462). Clark et al. reused for all three tasks. Participants would therefore have to take (2010) summarized the empirical results in the game literature: apart their creations and move the items before being able to start “All of the studies that have been published in reputable journals the next task. This separation of materials from the task areas may have reached a negative conclusion about learning from games” (p. have caused a spit-attention effect, thereby increasing the cognitive 269). The present experiments tested the theory that educational demand of the task and sacrificing learning (Low, Jin, & Sweller, games can be powerful learning tools when they provide an 2010). Although the narrative theme was intended to improve environment for discovery learning and are grounded in a realistic player motivation, these effects may have been drowned out by narrative that gives life to the learning environment. Our results do these other elements of game design. Future research is needed that not provide evidence-based support for claims about the educa- examines the role of narrative theme in other adventure games. tional value of narrative discovery games, at least for the games Given the important role of prior knowledge in learning with that we tested. games (Adcock, Watson, Morrison, & Belfore, 2010), future re- In addition, the Cache 17 study represents an attempt to give a search should determine the effects on learning of the alignment fair hearing to the proposition that narrative theme can improve 248 ADAMS ET AL.

student learning from an adventure game. We chose Cache 17 Advance outlines, familiarity, and text genre on retention of prose. partly because Koenig (2008) had obtained suggestive but nonsig- Journal of Experimental Education, 48, 281–290. nificant positive results, perhaps attributable to a relatively small Harp, S. F., & Mayer, R. E. (1997). The role of interest in learning from sample size. We sought to overcome this problem and flaws in scientific text and illustrations: On the distinction between emotional other games studies by employing an adequate sample size, a interest and cognitive interest. Journal of Educational Psychology, 89, 92–102. doi:10.1037/0022-0663.89.1.92 rigorous controlled experiment, and an improved version of an Harp, S. F., & Mayer, R. E. (1998). How seductive details do their damage: established game environment. Yet, our results are negative in that A theory of cognitive interest in science learning. Journal of Educational adding narrative theme did not improve learning with this educa- Psychology, 90, 414–434. doi:10.1037/0022-0663.90.3.414 tional game. Other formulations of narrative games or other game Hung, D. (2002). Situated cognition and problem-based learning: Implica- formats may turn out to be more productive venues for academic tions for learning and instruction with technology. Journal of Interactive learning. Learning Research, 13, 393–414. These studies present a cautionary tale that although proponents Hung, W., & van Eck, R. (2010). Aligning problem solving and gameplay: may believe narrative discovery games will result in higher learn- A model for future research and design. In R. van Eck (Ed.), Interdis- ing, rigorous experimental research is needed to support or refute ciplinary models and tools for serious games: Emerging concepts and those claims. It is tempting to focus on exciting cutting-edge future directions (pp. 227–263). Hershey, PA: IGI Global.

broadly. advances in educational technology, which can be characterized as Johnson, C. I., & Mayer, R. E. (2010). Adding the self-explanation prin- taking a technology-centered approach to educational technology ciple to multimedia learning in a computer-based game-like environ- publishers. ment. Computers in Human Behavior, 26, 1246–1252. doi:10.1016/ (Mayer, 2009). However, it may be more educationally sound to j.chb.2010.03.025 take a learner-centered approach to educational technology, which allied Kirschner, P. A., Sweller, J., & Clark, R. E. (2006). Why minimal guidance disseminated its focuses on how the human mind works and how to adapt available during instruction does not work: An analysis of the failure of construc- be of technology to enhance human learning. The challenge facing ed- tivist, discovery, problem-based, experiential, and inquiry-based teach- to

one ucational researchers and educational game designers is to deter- ing. Educational Psychologist, 41, 75–86. doi:10.1207/ not or mine how to use educational games in ways that take advantage of

is s15326985ep4102_1 their motivating properties while still ensuring that instructional Koenig, A. D. (2008). Exploring effective educational video game design: and objectives are met. The interplay between narrative and game-schema construction (Un-

user published doctoral dissertation). Arizona State University.

Association Laurillard, D. (1998). Multimedia and the learner’s experience of narrative. References Computers & Education, 31, 229–242. doi:10.1016/S0360- 1315(98)00041-4

individual Adcock, A. B., Watson, G. S., Morrison, G. R., & Belfore, L. A. (2010). Lee, K. M., Park, N., & Jin, S. (2006). Narrative and interactivity in

the Effective knowledge development in game-based learning environ- computer games. In P. Vorderer & J. Bryant (Eds.), Playing video games Psychological of ments: Considering research in cognitive processes and simulation de- (pp. 259–274). Mahwah, NJ: Erlbaum. sign. In R. van Eck (Ed.), Interdisciplinary models and tools for serious use games: Emerging concepts and future directions (pp. 152–168). Her- Low, R., Jin, P., & Sweller, J. (2010). Learners’ cognitive load when using shey, PA: IGI Global. educational technology. In R. van Eck (Ed.), Gaming and cognition: American Akkerman, S., Admiraal, W., & Huizenga, J. (2009). Storification in Theories and practice from the learning sciences (pp. 169–186). New personal the history education: A mobile game in and about medieval Amsterdam. York, NY: Information Science Reference. the by Computers & Education, 52, 449–459. doi:10.1016/j.compedu Marsh, T. (2010). Activity-based scenario design, development, and as- for .2008.09.014 sessment in serious games. In R. van Eck (Ed.), Gaming and cognition: Barab, S. A., Hay, K. E., & Duffy, T. M. (1998). Grounded constructions Theories and practice from the learning sciences (pp. 213–225). New solely and how technology can help. TechTrends, 43(2), 15–23. doi:10.1007/ York, NY: Information Science Reference. copyrighted BF02818171 Mayer, R. E. (2004). Should there be a three strikes rule against discovery is Brown, J., Collins, A., & Duguid, P. (1989). Situated cognition and the learning? The case for guided methods of instruction. American Psy- intended culture of learning. Educational Researcher, 18, 32–42. chologist, 59, 14–19. doi:10.1037/0003-066X.59.1.14 is Bruner, J. S. (1961). The act of discovery. Harvard Educational Review, Mayer, R. E. (2009). Multimedia learning (2nd ed.). New York, NY: document 31, 21–32. Cambridge University Press. article Clark, R. E. (2001). Learning from media. Greenwich, CT: Information Mayer, R. E. (2010). Applying the science of learning. Upper Saddle River, This NJ: Pearson.

This Age. Clark, R. E., Yates, K., Early, S., & Moulton, K. (2010). An analysis of the Mayer, R. E. (2011). Multimedia learning and games. In S. Tobias & J. D. failure of electronic media and discovery-based learning: Evidence for Fletcher (Eds.), Computer games and instruction (pp. 281–305). Green- the performance benefits of guided training methods. In K. H. Silber & wich, CT: Information Age. W. R. Foshay (Eds.), Handbook of improving performance in the work- Mayer, R. E., Griffith, E., Naftaly, I., & Rothman, D. (2008). Increased place (pp. 263–297). San Francisco, CA: Pfeiffer. interestingness of extraneous details leads to decreased learning. Journal Cunningham, L. J., & Gall, M. D. (1990). The effects of expository and of Experimental Psychology: Applied, 14, 329–339. doi:10.1037/ narrative prose on student achievement and attitudes toward textbooks. a0013835 Journal of Experimental Education, 58, 165–175. Mayer, R. E., & Johnson, C. I. (2010). Adding instructional features that Dickey, M. D. (2006). Game design narrative for learning: Appropriating promote learning in a game-like environment. Journal of Educational adventure game design narrative devices and techniques for the design Computing Research, 101, 621–629. of interactive learning environments. Educational Technology Research Neitzel, B. (2005). Narrativity in computer games. In J. Raessens & J. & Development, 54, 245–263. doi:10.1007/s11423-006-8806-y Goldstein (Eds.), Handbook of computer game studies (pp. 227–245). Gee, J. P. (2007). What video games have to teach us about learning and Cambridge, MA: MIT Press. literacy (2nd ed.). New York, NY: Palgrave Macmillan. O’Neil, H. F., & Perez, R. S. (Eds.). (2008). Computer games and team and Graesser, A. C., Hauft-Smith, K., Cohen, A. D., & Pyles, L. D. (1980). individual learning. Amsterdam, the Netherlands: Elsevier. NARRATIVE GAMES FOR LEARNING 249

O’Neil, H. F., Wainess, R., & Baker, E. L. (2005). Classification of The Cognition and Technology Group at Vanderbilt. (1990). Anchored learning outcomes: Evidence from the games literature. Curriculum instruction and its relationship to situated cognition. Educational Re- Journal, 16, 455–474. doi:10.1080/09585170500384529 searcher, 19(6), 2–10. Papert, S. (1980). Mindstorms: Children, computers, and powerful ideas. Tobias, S., & Fletcher, J. D. (Eds.). (2011). Computer games and instruc- New York, NY: Basic Books. tion. Greenwich, CT: Information Age. Reese, D. D. (2010). Introducing flowmeter: A CyGaMEs assessment suite Wolfe, M. B. W., & Mienko, J. A. (2007). Learning and memory of factual tool. In R. van Eck (Ed.), Gaming and cognition: Theories and practice content from narrative and expository text. British Journal of Educa- from the learning sciences (pp. 277–254). New York, NY: Information tional Psychology, 77, 541–564. doi:10.1348/000709906X143902 Science Reference. Zumbach, J., & Mohraz, M. (2008). Cognitive load in hypermedia reading Shaffer, D. W. (2006). How computer games help children learn. New comprehension: Influence of text type and linearity. Computers in Hu- York, NY: Palgrave Macmillan. man Behavior, 24, 875–887. doi:10.1016/j.chb.2007.02.015 Spires, H. A., Turner, K. A., Rowe, J., Mott, B., & Lester, J. (2010, May). Game-based literacies and learning: Towards a transactional theoretical perspective. Paper presented at the meeting of the American Edu- cational Research Association, Denver, CO. Received January 14, 2011 Sweller, J. (1999). Instructional design in technical areas. Camberwell, Revision received July 13, 2011 Australia: ACER Press. Accepted August 21, 2011 Ⅲ broadly. publishers. allied disseminated its be of to one not or is and user Association individual the Psychological of use American personal the the by for solely copyrighted is intended is document article This This