Journal of Educational Psychology © 2011 American Psychological Association 2012, Vol. 104, No. 1, 235–249 0022-0663/11/$12.00 DOI: 10.1037/a0025595 Narrative Games for Learning: Testing the Discovery and Narrative Hypotheses Deanne M. Adams, Richard E. Mayer, Alan Koenig and Richard Wainess and Andrew MacNamara University of California, Los Angeles University of California, Santa Barbara Strong claims are made for the potential educational effectiveness of narrative-based adventure games, but evidence about how to construct effective educational games is needed (Clark, Yates, Early, & Moulton, 2010; O’Neil & Perez, 2008). College students played a computer-based narrative discovery learning game called Crystal Island (Spires et al., 2010), in which they learned about pathogens (in Experiment 1), or one called Cache 17 (Koenig, 2008), in which they learned how electromechanical devices work (in Experiment 2). In broadly. media comparison tests, participants who learned by playing the game performed worse than students who learned from a matched slideshow presentation on retention (d ϭ 1.37), transfer (d ϭ 0.57), and difficulty publishers. rating (d ϭ 0.93) in Experiment 1 and on posttest score (d ϭ 0.31) and learning time (d ϭ 2.89) in Experiment 2. In value-added tests, taking away the narrative theme concerning a detective story in the Cache 17 game allied did not significantly affect students’ posttest score (d ϭϪ0.16) or learning time (d ϭϪ0.22) in Experiment disseminated its 2. Overall, these results provide no evidence that computer-based narrative games offer a superior venue for be of to academic learning under short time spans of under 2 hr. Findings contradict the discovery hypothesis that one students learn better when they do hands-on activities in engaging scenarios during learning and the narrative not or is hypothesis that students learn better when games have a strong narrative theme, although there is no evidence concerning longer periods of game play. and user Keywords: educational games, discovery, narrative, computer-based learning, game design Association individual There is strong evidence that computer games are hugely pop- In the present set of experiments, we take a step toward provid- the ular. For example, as of 2002, more money was spent on computer Psychological ing evidence-based guidance for game designers by examining the of games in the United States—$6.9 billion—than on box-office instructional effectiveness of the game genre of computer-based use movies, and approximately 145 million Americans (or about 60% narrative discovery learning games, which has been proposed to of the population over age 6) regularly played computer games improve academic learning. In computer-based narrative discovery American personal (Lee, Park, & Jin, 2006). Advocates of educational gaming have learning games, players are faced with a challenge within a sim- the the proposed that educators should harness the appeal of computer by ulated environment in which they can control the actions of one or for games as a vehicle for fostering student learning, but reviews of more characters. We define narrative games as those having a clear the research literature have not yielded strong support for the story line in which characters’ actions are directed toward a goal in solely instructional effectiveness of computer games (Clark, Yates, Early, the story. We define discovery games as those in which players copyrighted & Moulton, 2010; O’Neil & Perez, 2008; O’Neil, Wainess, & learn academic content in the context of solving problems or is Baker, 2005; Tobias & Fletcher, 2011). In a recent review, for exploring within a game rather than by being directly presented intended example, O’Neil and Perez (2008) lamented that “there is almost with the academic content. is no guidance for game designers and developers on how to design Rather than trying to develop games on our own, we received document games that facilitate learning” (p. ix). permission to use two narrative discovery games that had been article This built by game design experts and field-tested over several years. This We sought high-quality narrative discovery games that were ex- This article was published Online First September 26, 2011. Deanne M. Adams, Richard E. Mayer, and Andrew MacNamara, De- pertly designed to be fun and to serve a clear educational purpose, partment of Psychology, University of California, Santa Barbara; Alan and we used two different, well-designed games to give our media Koenig and Richard Wainess, National Center for Research on Evaluation, comparison studies a fair testing. We measured learning outcomes Standards, and Student Testing (CRESST), University of California, Los through tests designed to tap transfer (i.e., ability to use the Angeles. material to generate solutions to new problems that were not part This research was supported by a grant from the Office of Naval of the game) and retention (i.e., ability to remember the material), Research. The game used in Experiment 1 was developed for use by Spires, consistent with calls to include measures of transfer in assessments Turner, Rowe, Mott, and Lester (2010). The game used in Experiment 2 of learning with games (Hung & van Eck, 2010; Mayer, 2011; was developed by Koenig (2008). Experiment 1 was based on a senior Reese, 2010). honors thesis by Andrew MacNamara. Correspondence concerning this article should be addressed to Deanne First, Crystal Island was developed and revised by James Lester M. Adams or Richard E. Mayer, Department of Psychology, University of at North Carolina State University, who is a well-known expert in California, Santa Barbara, CA 93106. E-mail: [email protected] or narrative game design. In Crystal Island (Spires, Turner, Rowe, [email protected] Mott, & Lester, 2010), the player is a visitor to a research team on 235 236 ADAMS ET AL. a remote island whose stay is disrupted by the spread of an “Even though the current chapter provides strong theoretical rea- unknown disease (shown in Figure 1). The player’s task is to sons why narratives are important, future studies should empiri- discover the source of the disease through interacting with other cally test the effects of narratives on various psychological vari- characters and using lab microscopes to run tests. The game is ables” (Lee et al., 2006, p. 271). We take up this challenge in the intended to help the player learn about how pathogens work within present set of experiments. the context of playing the game. Computer-based narrative discovery learning games such as Second, Cache 17 was developed by Alan Koenig at Arizona State Crystal Island and Cache 17 are based on two interrelated hypoth- University, using game design principles with a focus on narrative eses concerning educational game design, the discovery hypothesis theme (Koenig, 2008). Development required approximately 6 and the narrative hypothesis. The discovery hypothesis is that months of programming time, and the game has undergone several students learn academic content more deeply when they must cycles of revision based on field testing. In Cache 17, the player views interact within a realistic learning context than when they are a brief introductory cinematic that lays out the story line about a simply told the academic content. The narrative hypothesis is that long-lost painting that may be found in an old bunker system dating students learn better from computer-based games that have strong back to World War II. The player’s job is to make his or her way narrative themes than from games that do not have strong narrative through the bunker system to solve the detective story about the themes. The present set of studies provides experimental tests broadly. whereabouts of the painting, along the way constructing electrome- concerning the efficacy of the discovery hypothesis and the nar- chanical devices to help open doors (shown in Figure 2). The instruc- rative hypothesis for guiding the design of educational games. publishers. tional objective of the game is to help the players understand how various electromechanical devices work. allied The Discovery Hypothesis for Designing Educational disseminated its Lee et al. (2006) pointed to four reasons for incorporating Computer Games be of narrative themes such as these in computer games: They provide to one surrogate experiences, reduce cognitive load, motivate game play- not The Case for Discovery in Educational Computer Games or ing, and allow for a strong sense of presence. Yet, there is not a is sufficient body of empirical evidence for evaluating the educa- According to the discovery learning view, students learn better and tional effectiveness of narrative as a feature in computer games: when they can actively explore a learning environment than when user Association individual the Psychological of use American personal the the by for solely copyrighted is intended is document article This This Figure 1. Screen shot from Crystal Island game displaying a problem-solving exercise aimed to guide the learner to the solution. NARRATIVE GAMES FOR LEARNING 237 broadly. publishers. allied disseminated its be of to one not or is and user Association Figure 2. Screen shot from Cache 17 game displaying the wet-cell battery construction and the PDA entry on wet-cell batteries. individual the Psychological of use they are simply told what they need to know (Bruner, 1961; shown to have the potential to mimic real-world environments and Mayer, 2010; Papert, 1980), but research on