Exploring the Elements of Educational Video Game Design and Creation

Home , Educational game, Educational software, Educational video game

EXPLORING THE ELEMENTS OF EDUCATIONAL VIDEO GAME DESIGN AND CREATION

A DISSERTATION

SUBMITTED TO THE GRADUATE SCHOOL OF EDUCATION

AND THE COMMITTEE ON GRADUATE STUDIES

OF STANFORD UNIVERSITY

IN PARTIAL FULFILLMENT OF THE REQUIREMENTS

FOR THE DEGREE OF

DOCTOR OF PHILOSOPHY

Osvaldo Jiménez

June 2013

This work is licensed under a Creative Commons Attribution- Noncommercial 3.0 United States License. http://creativecommons.org/licenses/by-nc/3.0/us/

This dissertation is online at: http://purl.stanford.edu/xj238xb0164

ii I certify that I have read this dissertation and that, in my opinion, it is fully adequate in scope and quality as a dissertation for the degree of Doctor of Philosophy.

Shelley Goldman, Primary Adviser

I certify that I have read this dissertation and that, in my opinion, it is fully adequate in scope and quality as a dissertation for the degree of Doctor of Philosophy.

Amado Padilla, Co-Adviser

I certify that I have read this dissertation and that, in my opinion, it is fully adequate in scope and quality as a dissertation for the degree of Doctor of Philosophy.

Daniel Schwartz

Approved for the Stanford University Committee on Graduate Studies. Patricia J. Gumport, Vice Provost Graduate Education

This signature page was generated electronically upon submission of this dissertation in electronic format. An original signed hard copy of the signature page is on file in University Archives.

iii

Abstract It is difficult to create video games that are both entertaining and educational. Few educational video games have empirical evidence to support their claims of effectiveness (Honey & Hilton, 2011). Educational video games are explored through research and discussion aimed at making the development process more effective for other educational game designers. The discussion unfolds in several chapters.

First, concerns surrounding the making of educational games are discussed. For instance, educational game designers are often given the task of delicately balancing the fun and educational aspects of their games. This balancing is highlighted in a framework presented of the development journey that educational game developers often must take, navigating around these two aspects, which are called sirens.

Literature in psychology and other fields that make arguments for elements that can help make a game more educational and more fun are reviewed. On the educational side, the domain of fractions is explored, and the misunderstandings and difficulties encountered by students who are learning fractions are reviewed. The learning of fractions anchors the development and research since there is much work needed to improve student understanding. On the fun side, some of the motivational choices that have been used to increase the amount of fun are considered. Although there are elements in games that explicitly help either fun or learning, three areas are reviewed – choice, collaboration, and fantasy/story – that have properties that can help both fun and educational aspects of a game. It is the third area (fantasy/story) that this dissertation further explores as an element to be used in games.

Since fantasy/story warrants further exploration, the literature on fantasy and more specifically story are reviewed. The review of the literature concludes that story can be helpful with learning by assisting with comprehension. Research on story grammars and scripts provide evidence for story helping people comprehend information, making it more likely for these people to remember that information. On the other hand, story can also help persuade people. The persuasion aspect is notable v because it can help with both motivation and learning. Persuasion is important because it gives the audience a willingness to accept a different outcome or piece of knowledge.

Using this information, the dissertation then details the process that colleagues and I followed in order to create an educational game called Tug-of-War. After an initial study showing no significant achievement differences in learning, Tug-of-War was refined and demonstrated significant differences on pre-post measures in two quasi-experimental classroom studies. These three studies also provided evidence that children enjoyed playing Tug-of-War, based on survey results adapted from motivation scales. The process for creating the game as well as the study design and results for these three studies are provided.

Next, a qualitative, interview-based study of how students conceptualize story both in interactive and non-interactive media is reported. The methods are described, including how game were selected for children to play, and how the interview protocol that was accomplished with the children. The chapter ends by describing the analysis, and highlights recurring themes that were discovered around children’s notions of story.

The findings all of the initial studies resulted in a classroom study, where students were randomly assigned to play one of three versions of the Tug-of-War game. The first version was similar to the previous versions, having a context/story of the children participating in a tug-of-war match. The second version had similar game mechanics, but had no story or images related to the story. The third version embellished the first version by providing students with images of the relevant characters that are present in the game. The chapter provides analysis of pre and post measures of achievement, students reporting of fun, and a video analysis of ways students engaged in two of the conditions. Results of the final study indicate that story had a positive impact on the student engagement and involvement with the fractions content. The advantages that story brings to bear in educational games is discussed.

Acknowledgements I would like to thank my dissertation reading committee (Shelley Goldman, Daniel Schwartz, and Amado Padilla) for their invaluable support and wisdom during this entire process. To Shelley, thank you for keeping me going and for guiding me through the whole process. To Dan, thank you for helping me correctly think about how to conduct, study, and analyze a topic of interest for me. To Amado, thank you for always believing in me and helping me keep the entire process in perspective. I would also like to thank my oral committee (Mehran Sahami and Eric Roberts) for their valuable insight and questions.

Next I would like to thank Ugochi Acholonu and Dylan Arena for their help in creating the initial designs of the Tug-of-War game, and in helping conduct the research. I would also like to thank fellow colleagues and researchers Jessica Tsang, Ilsa Marie Dohmen, Jennifer Tackman, Doris Chin, Kristen Pilner Blair, Joe Prempeh, Min Chi, Nicole Hallinen, Jacob Haigh, and Rassan Walker for their advice and for their contributions in helping me conduct the research. I’d also like to thank the administrators, teachers, and users in the Bay Area that contributed to this project.

I would also like to thank the DARE Fellowship for helping make the goal of completing my dissertation a reality. If it were not for the support of my colleagues in that program as well as Anika Green and Chris Golde, I may not have finished.

Finally I would like to thank my family & friends for their unwavering support throughout this whole journey and the numerous ways they helped. Specifically I’d like to thank my nieces and nephews (aka “las crías”) for their constant playtesting, my parents Jaime & Alicia for telling me never to give up, and my friends Dan G, Daniel and Javier for their writing advice. Finally, and most importantly I’d like to thank my wife Cheryl Lynne for her incredible support at every imaginable stage of this process. I would not have finished if it were not for my family.

I will be forever indebted to all of you.

vii

Table of Contents

Abstract...... v

Acknowledgements ...... vii

List of Tables ...... xii

List of Illustrations...... xii

Section I – Difficulty in Making Educational Games ...... 1

Introduction ...... 1

Researching the Learning...... 7

Fractions ...... 8

Researching the Fun – Empirical Research on Motivation...... 14

Intrinsic Motivation ...... 15

Challenge...... 17

Other Structural Qualities ...... 18

Researching Elements useful both for fun and learning ...... 21

Control/Choice ...... 22

Illusion of Control ...... 23

Collaboration ...... 25

Section II – Reviewing the Literature on Fantasy and Story ...... 29

Fantasy ...... 29

Story ...... 32

Story and its impact on comprehension and memory ...... 37

Story and its impact on persuasion...... 39

Section III - The Process of Creating Tug-of-War ...... 49

viii

The Initial Tug-of-War Concept ...... 49

Study 1 ...... 53

Method ...... 53

Results and Discussion ...... 54

Our second physical prototype...... 56

Study 2 ...... 58

Method ...... 58

Fun Results and Discussion ...... 59

Learning Results and Discussion ...... 60

The Digital Prototype ...... 62

Study 3 ...... 63

Method ...... 63

Results ...... 65

Discussion ...... 66

Section IV – Investigating children’s conceptions of stories ...... 68

Theoretical analysis...... 68

Finding a contrasting educational game to Tug-of-War ...... 71

Interview Protocol ...... 76

Interview study...... 77

Participants ...... 77

Method ...... 77

Materials ...... 78

Interview Questions ...... 79

Description of Analysis ...... 80

Results and Discussion ...... 81

Characters ...... 81

The game mechanics can drive the story ...... 86

Tug-of-War’s story needed some work ...... 90

Kintsch and Van Dijk theory applied to educational games ...... 93

Section V – The Experimental Study ...... 97

From interviews to experiments: investigating the impact that story and particularly characters have on a game...... 97

The Three Versions of the game ...... 101

Participants:...... 104

Method ...... 105

Materials...... 107

Results ...... 108

Reporting data from paper and pencil measures ...... 108

Investigating Children’s levels of Fun ...... 110

Investigating the relationship between fun and learning across conditions ...... 110

Investigating in-game measures ...... 114

Investigating video ...... 116

Discussion...... 122

New Questions ...... 125

Study Limitations ...... 127

Next Steps ...... 129

Conclusion ...... 132

Bibliography ...... 134

Appendix A – Early Tug-of-War Fractions Assessment ...... 154 x

Appendix B – 2012 Fractions Assessment ...... 159

Appendix C – 2012 Additional Post-Test Items...... 164

Appendix D – Protocol for Interview Study...... 167

List of Tables

Table 1: Cohort 1 Fun assessment results from our two prototypes ...... 60 Table 2: Updated Fun Results with Digital Prototype findings in bold ...... 67 Table 3: My proposed dimensions applied to selected educational Games ...... 75 Table 4: Average Gain Scores from Pre to Post by Condition, (Standard Deviation in Parenthesis)...... 108 Table 5: Sub-test gain scores by Condition (Standard Deviation in Parenthesis) ...... 109 Table 6: Correlations of Fun Ratings with Gain or New Fractions sub-gain scores by condition ...... 113 Table 7: Select in-game results by condition...... 115

List of Illustrations

Figure 1: Screenshot of Donkey Kong Jr. Math...... 3 Figure 2: The Landscape for creating Educational Games...... 5 Figure 3: One of many possible voyages for creating an Educational Game...... 6 Figure 4: The Siren of Learning ...... 7 Figure 5: The Siren of Fun ...... 14 Figure 6: The Island of Choice and Control ...... 22 Figure 7: The island of Collaboration...... 25 Figure 8: The Island of Fantasy/Story ...... 29 Figure 9: Screenshot of intrinsic (left) vs. extrinsic (right) modes of Zombie Division ...... 31 Figure 10: My proposed relationship between story, comprehension, motivation and persuasion ...... 40 Figure 11: The Teammate Card ...... 50 Figure 12: Attack Card ...... 50 Figure 13: Pre- and Post- test scores for our first Physical Prototype ...... 55 Figure 14: Tug-of-War’s path for its first physical prototype ...... 56 Figure 15: Pre-, Mid-, and Post-test Means and Standard Errors ...... 61 xii

Figure 16: CST Subtest Individual Scores, Mean, and Standard Errors ...... 62 Figure 17: A screenshot of the work area students have when solving problems...... 63 Figure 18: Graph detailing test scores in both the physical card game and the computer version...... 65 Figure 19: Thorndyke's Theory (top) and Tug-of-War’s instantiation (bottom) ...... 70 Figure 20: Screenshots of the game Escape from Fraction Manor ...... 72 Figure 21: Screenshots of the game Math Blaster Hyperblast ...... 73 Figure 22: Screenshots of the game Carpenter’s Cut...... 74 Figure 23: The Number Eaters Game that was used as a contrast to Tug-of-War ...... 75 Figure 24: Number Eaters game applied to the Thorndyke Framework ...... 88 Figure 25: Manipulatives moving off-screen before being shown the question ...... 98 Figure 26: Animations detailing that 1/2 of 8 (left) are now stinky (right) ...... 99 Figure 27: The three conditions on the card playing screen...... 102 Figure 28: The three conditions after having provided help on answering ¾ of four.103 Figure 29: Student Gain Scores by Condition...... 111 Figure 30: Student Gains on New Fractions by Condition...... 112 Figure 31: Plotting Learning Gains by the Amount of Fun that a subject had...... 113 Figure 32: The conceptualization framework proposed early on ...... 131

xiii

Section I – Difficulty in Making Educational Games

Introduction Videogames have stormed into American homes. Virtually non-existent 35 years ago, videogames have quickly become a powerhouse media format for entertainment, generating more annual revenue than films at the box office (Holson, 2004). Children now spend a considerable amount of time playing video games, with boys spending an average of 13 hours per week, and girls spending an average of 5.5 hours per week (Carnagey & Anderson, 2005). Videogames are an important part of children’s lives and have high entertainment value. School learning, on the other hand, does not have the same entertainment value for most children. In fact, many children find school work to be boring. Replacing traditional school work with computer-based learning is not necessarily the answer, as some children find computer work boring as well (Anderson, Lankshear, Timms, & Courtney, 2008). Children like playing video games, and are willing to expend considerable time and effort into solving difficult video games-based problems – positive feelings which differ markedly from how they feel about school. Early in the videogame age, scholars noticed and discussed this discrepancy, reflecting on how to bring the motivation kids have for videogames into the classroom (Bowman, 1982; Malone, 1981).

One early effort to address this discrepancy came from the software industry, consisting of games meant to teach students about a topic, a field now known as edutainment. In the late 1980’s and early 1990’s, both aspiring and established software companies sought to make edutainment software by loosely embedding educational content into computer video games. One such example is Donkey Kong Jr. Math, released for the Nintendo Entertainment System and published by Nintendo, one of today’s pre-eminent software makers. The goal of Donkey Kong Jr. Math was to assemble an equation in order to reach a target number. Players would play as a monkey that would climb vines in order to grab both numbers and mathematical operators, using them to assemble an equation (see Figure 1). The edutainment 1 industry sought to combine the best of video games and education partly for financial gain.

However, the success of these edutainment software titles is inconclusive. Donkey Kong Jr. Math was the poorest selling title at launch for the system and was criticized both by academics (Epstein, 2005) and video game critics alike (Mackey, 2010) – while it may be argued as educational, it was not fun. James Gee (2004) notes that games from that era (such as Donkey Kong Jr. Math) led to an outcry among critics that video games and education should remain separate (Gee, 2004). The word “edutainment” developed a negative connotation insofar that scholars and developers now refrained from using the word for fear of backlash (Kirriemuir & McFarlane, 2004). Part of the trouble for edutainment has been the sheer difficulty of creating a game that both helps people learn and provides them with an entertaining experience.

Most educational games have trouble in one of two ways; either they are not fun or they are not educational. An example of a game that had trouble with being fun is Arden. Built by Edward Castronova and his research group, Arden is a 3-D game that has students live out Shakespearian works. Arden’s research group built the game with the hope that it would spark literature curriculum reform (Castronova et al., 2008). Although the premise sounded exciting, in a later interview (Naone, 2005), the research group proclaimed that the $250,000 project had one crucial flaw: it was not fun to play. While “fun” is not the primary purpose of an educational game, if “fun” is completely ignored, a game runs the risk of turning off students to critical fields, particularly STEM fields that could benefit from a more diverse setting (Metz, 2007).

Figure 1: Screenshot of Donkey Kong Jr. Math (taken from x-entertainment.com). In the game, the player controls one of the two monkeys on the bottom and makes them create arithmetic equations with the number and symbols displayed on the screen. The player’s intended goal is to create an arithmetic equation that equals the target value, displayed at the top by Donkey Kong.

Educational games are challenging to produce because it is arduous to make them fun. However, it is also arduous to make educational games that demonstrate any learning effects. There have been many commercial and educational games that have been studied or created by scholars. These scholars have shown that the games have diverse benefits, such as increased self-efficacy (Nelson, Ketelhut, Clarke, Bowman, & Dede, 2005), authentic scientific inquiry, (Barab, Sadler, Heiselt, Hickey, & Zuiker, 2007), and learning to use evidence to warrant claims (Steinkuehler, 2006). The evidence for these results comes both from the play in the games, as well as participation in activities outside the games. However, while some games have shown learning differences (Barab et al., 2007), there are relatively few studies demonstrating the learning benefits of educational games using traditional measures (Honey & Hilton, 2011). One game made by researchers Squire & Klopfer (2007) demonstrates the critical balance between creating a game and making it educational.

Squire & Klopfer decided to create an educational game that took the form of an augmented reality simulation. Augmented reality is more than just imagining a 3 virtual world - it places a “virtual” component of a game into real-life, with people walking around and interacting in the real world. In Squire & Klopfer’s augmented reality game, groups of high-school and college students worked together to move around a real campus or site, taking virtual water quality samples. Squire & Klopfer decided to scrutinize case studies at the discourse level to study the interactions that took place among the students. These researchers studied how students used each other to refine hypothesis and formulate additional questions. One finding from their work was the importance of scaffolding the formation of scientific inquiry. In the simulation, students were not given any prior information on the meanings of water quality readings. This caused many teams to wander aimlessly around campus trying to find “lower” or “higher” readings without stopping and trying to understand what such readings meant, and what exactly they were looking for. This provides an excellent example of how these self-made games could falter if players are not given the necessary information to learn, further demonstrating the delicate balance between a fun and an educational game.

The creation of an educational game is like a siren’s song. While many are lured to create a fun game that can help students learn, attaining and reaching such a goal is a laborious and unforgiving process fraught with potential disasters. Currently there are many aspiring developers lured to this very matter, accompanied by an increasing number of conferences such as the Game Developers Conference, Foundations of Digital Games, Games Learning & Society, and the Serious Games Summit. While many are interested in creating games, Honey & Hilton recently reported that there are few if any cases of educational games that have demonstrated benefits to science achievement in controlled studies (Honey & Hilton, 2011). This finding applies to the difficulty of making games: while numerous developers have created games, it is currently unclear if the games met their educational goals. This leaves unanswered the question for some of whether games should be made for learning.

To answer this question, the field of games and learning should add two goals to advance the study of successful educational game creation. The first goal is that educational game developers/researchers should meticulously detail the process of game creation. Such detail is necessary so that the educational games field can gain a deeper understanding of the potential ways to create a fun and educational game. The second goal is to conduct basic research around the elements and parts of educational games. If scholars study these elements, they can gain insight into how to improve current educational games and streamline future games. Some members of the educational games community may feel that such progress is being made, with introductions to the field such as the seven circumstances of game design (Arena, 2012) serving as a foundation for creating a common dialog amongst researchers. I agree that sharing broadly each game’s circumstances can help. I would add that more detail and evidence of the game-making process as well as basic research needs to be shared for the field to advance. By exploring these processes and elements further, it might be possible for future developers to navigate around the siren’s song.

Figure 2: The Landscape for creating Educational Games

Before moving forward on navigating a learning game that maintains a healthy balance of fun and learning, it is helpful to understand the previous research on the creation of successful video games. A familiarity with this research can help inform development processes and future research designs. In order to organize the information and to highlight the path that lies ahead for educational game designers, I have created a graphic (See Figure 2 above). This image provides a context for the voyage that educational designers (represented in the lower left) must take in order to reach the red “X”, which represents the successful educational game. Figure 3 represents one such voyage; however the red path that one takes may not necessarily be the same for all designers.

Figure 3: One of many possible voyages for creating an Educational Game

The next section of this proposal details a variety of research findings that can potentially inform the two main aforementioned criteria for educational games - the learning and the fun. These are the sirens that are represented in the image. Later on, I will describe elements which I feel are not sirens per se, but rather fields of research which can inform the creation of educational games – the fields of choice, collaboration and fantasy. To help jumpstart the research on learning, I first examine research around the domain of fractions. I report the fractions research both to detail

6 the issues with learning fractions and to provide designers ideas on how use academic research to improve their own software. After reviewing the siren of learning, I move to examining the siren of fun: examining empirical research and other psychological aspects of games that help bring the fun aspect, such as motivation. Finally, I will look at the islands/elements that can be introduced into educational games that can influence both the fun and the learning. I’ll start with the educational research first below.

Researching the Learning

Figure 4: The Siren of Learning

The Squire & Klopfer augmented reality game illustrated that educational game developers are not just creating a fun experience. Squire & Klopfer also found it necessary to design a custom curriculum, a task that most game designers have to take on as well. Indeed, scholars have discussed the inherent struggles relating to designing educational environments and the need to address varying perspectives in designing these environments. These perspectives include focusing on the needs of the learner, the knowledge to be learned, the ways to assess the learners, and the surrounding community/context of the learning environment (Bransford, NRC, 2000). If such a balance exists in an educational environment, then that balance amplifies in creating the curriculum for a game. Creating curriculum requires refinement and careful considerations by all educational game designers, yet it is a practice that is not 7 well researched (Honey & Hilton, 2011). Veteran and successful educational developers need to research and document the creation of their curriculum experience so that new educational developers can learn from them. Furthermore, researchers also need to document best practices and pitfalls for students in the particular domain that is at the center of the game. To help jumpstart this movement and understand what will be necessary to learn and share, I have worked on the design, implementation, and research with a game. I am interested in examining research on a contested domain—mathematics—and a topic that is burdensome to learn—rational numbers. Focusing on the challenges faced in learning rational numbers will allow consideration of the complexities and pitfalls that may arise in creating a game and its academic curriculum. This research comes into play and bears directly on discussions that follow and how issues about the fraction learning are addressed through game mechanics, strategies and curriculum.

Fractions The topic of rational numbers, also known as fractions, is a vital part of the school curricula in United States. Mastery of a variety of fractions operations is listed as part of the Common Core State Standards for Mathematics Education in grades three through five (National Governors Association Center for Best Practices, 2010). Besides being part of all US education standards, fractions are foundational and vital to future success in higher order mathematics such as algebra (National Governors Association Center for Best Practices, 2010; Nunes, 2006). Fractions are a fundamental set of concepts to master. Unfortunately, students have a extraordinarily difficult time understanding and mastering fractions, even in understanding basic concepts such as realizing that 1/2 of 8 is not the same as 1/2 of 12 (Nunes, 2006). A short review of the strategies and difficulties that children encounter when learning fractions is helpful to addressing basic learning needs that include developing an understanding around what a fraction means (Standard 3.NF.1 and 3.NF.2, National Governors Association Center for Best Practices, 2010), and the operation of taking a fraction of a whole (Standard 4.NF.4, National Governors Association Center for Best Practices, 2010). 8

One of the earliest difficulties cited by researchers is the diversity of acumen displayed by students when they learn about fractions. When they first encounter fractions, students tend to over-rely on their knowledge of whole numbers (Kerslake, 1986; Mack, 1990). For instance, when students interpret a fraction such as 3/4, they tend to think of that fraction as two distinct whole numbers (Behr, Wachsmuth, Post, & Lesh, 1984). This over-reliance on whole numbers can cause students to make other incorrect assumptions about fractions, such as thinking that 3/10 is greater than 2/5 because both 3 and 10 are greater than 2 and 5, respectively. This misconception is further exacerbated when students study the addition of fractions. When students have to add 1/2 and 2/3, some tend to add the numerators and the denominators together, providing 3/5 as the answer. These types of incorrect whole-number reasoning assumptions tend to highlight some of the struggles students have at the outset of fourth grade (Petit, Laird, & Marsden, 2010).

In addition to incorrectly applying their knowledge about whole numbers to fractions, students also bring with them real-world informal knowledge around partitioning and equivalence (Mack, 1990). Other math researchers also agree that this phenomena is more general; children come to the classroom with varying amounts of math knowledge that they have informally organized in their minds (Baroody & Ginsburg, 1990; Griffin, Case, Siegler, & McGilly, 1994). Children are not blank slates when they learn new information; Piaget argued that children often would assimilate new information so that it can be organized into their existing models (Piaget, 1952). Piaget claims that assimilation is vital to learning: “Learning is possible only when there is active assimilation” (Piaget, 1964, p. 185). Thus, assimilation itself is worthy of discussion as a part of learning. Other scholars also agree, referring to assimilation as a problem of direct instruction – “with a direct instruction approach, new information is often too abstract for children to assimilate” (Baroody & Ginsburg, 1990, p. 58). This notion of starting with abstract concepts has been further criticized by Davis, who writes: “Abstract definitions are quite properly seen as a glory of modern mathematics, but they are NOT useful for the instruction of beginners, where one must build on things that the students already understand” 9

(Davis, 1984, p. 313). As an alternative, Davis agrees with the other scholars about using students’ prior knowledge to bear on solving problems, which includes the concept of sharing with fractions (Davis, 1984).

Though some of this informal knowledge is leveraged with fractions, there can be times where the formal knowledge students learn in fractions conflicts with their informal knowledge. Nunes explained that in interviews, students could not understand that 1/3 is bigger than 1/5, but they did know that a cake shared by five people would result in less cake per person than a cake shared by three (2006). This notion of sharing has been noted as the first step towards fractions understanding (Petit et al., 2010), followed by halving and then dealing with other even and then odd fractions. It is this informal knowledge about sharing that starts them at the first level in Petit’s representation of fractions. Other scholars have also stated arguments close to those of Petit, arguing for performing fair share activities. These fair share activities they argue, where students create equal-size groups or parts of a whole early on, are critical for a deeper understanding of formal mathematical properties in later years. This fair share activity, also referred to as equipartitioning, is a topic that children understand as early as pre-school (Wilson, Myers, Edgington, & Confrey, 2012).

In addition to their prior knowledge, children also have trouble with the way that fractions are represented in the curriculum. One popular representation used in fractions curricula is the part-whole representation, where fractions are represented as parts of a unit, such as pies or pizzas. The part-whole representation provides specific difficulties for students. If explained correctly, the part-whole representation can allow students to recognize the relationship of a fraction as some parts of a whole. Nevertheless, scholars have documented the difficulty people have in understanding what the whole is in many situations (Confrey & Smith, 1995; Lamon, 1994). For instance, when people are asked to circle 1/2 of four hearts, some people will circle half of each of the four hearts, rather than just circling two hearts (Payne, 1976). Circling half of each heart implies that those subjects believe that one heart constitutes

10 the whole, rather than all four. If students believed the half that they were circling was of the four hearts, then they would have circled two complete hearts rather than half of each heart.

Student’s use of their own partitive-whole diagrams can also produce incorrect results. Scholars have demonstrated this in analyzing questions that have been answered by 3000 11-14 year old students in England. In one such question, where the students are asked if 1/2 is the same as 3/6, a student who had created a pie with 6 slices decides to put a line directly down the middle. Because of the placement of the line and the fact that two of the six pieces were fully enclosed on one side of the line, the student incorrectly deduced that one-half is less than three-sixths (Hodgen, Küchemann, Brown, & Coe, 2010). The authors found similar mistakes with students incorrectly drawing sixths and thirds when the whole was considered a bar. Based on these mistakes, the researchers argue for the importance of having reliable references for creating sixths even though they also claim that students who make their own diagrams have a better understanding (Hodgen et al., 2010). Other researchers agree that students should create their own custom representations: researchers believe students should actively construct their representations in order to reach genuine academic accomplishments in mathematics (Putnam, Lampert, & Peterson, 1990).

Although custom representations are important, scholars have argued that teachers and curriculum developers should strive to teach multiple understandings of fractions in order to get the highest level of understanding (Petit et al., 2010). In fact, there are three classes of fractions representations that are beneficial for elementary students to master. These representations are the aforementioned partitive-whole, the set representation and the number-line representation (Petit et al., 2010).

The set representation is a representation in which the idea of the fractional unit is a set of items rather than a single item. The set representation has support from authors who find that the partitive-whole is better understood by taking the unit to be a larger number of items, which favors using the set representation (Lamon, 1994). For example, instead of thinking of the whole or unit as one pizza, the whole could be the 11 set of four pizzas. This would translate to 1/4 of 4 pizzas to be 1 whole pizza. In the partitive-whole model, 1/4 of a whole pizza, would be one slice of four equally split slices. The set representation is not only valuable because it serves as an alternative to the partitive-whole representation. The set model can also help students gain a deep understanding of proportional reasoning, and generalize their understanding of a fraction. Nevertheless, children’s earlier notions of representation still can be seen as hindering their progress in developing the set model. Researchers have noticed that children had difficulty relating a number to a different set of objects, such as 2 dozen meaning 24 cookies. Because students think that one number can only take on one value (2 is 2), it makes it difficult for them to gain an understanding of how “2 dozen” equates to “24 cookies” (Ball, 1990). This is in direct contrast to the partitive-whole, in which one pizza would serve as the whole unit.

The number-line representation is simply representing fractions on a number line. Translating a fraction to a number-line can support students’ notions that a fractional quantity is indeed a number that sits somewhere on the number line, a concept that is hidden in both the partitive-whole and set models (Kerslake, 1986). Students’ over-reliance on whole numbers also comes to light here as well. For instance, when children were given a number line from 1 to 3 and then asked to place roughly where 2/3 should go, some of the students would place it between 2 and 3 (Kerslake, 1986). The number-line helps children relate the magnitude of the fraction and to other numbers. The number-line also prepares students to understand mixed number fractions (Petit et al., 2010). Scholars have noted that students often have trouble understanding that there is an infinite amount of numbers between two fractions, such as 1/4 and 1/2 (Kerslake, 1986; Orton et al., 1995). All of these issues demonstrate the struggles children have with the number-line.

Based on the literature, an educational game developer can then use this knowledge to ensure that their game avoids a couple of pitfalls. A few examples are illustrative of both the knowledge and how to avoid these perils. First, students come in with prior knowledge about fractions, most notably around sharing. However,

12 students also have misconceptions based on that prior knowledge. For example, they tend to over-rely on whole numbers. In creating a game on fractions, we know that having multiple representations would serve students well as the new information can be assimilated by students into their own mental models. Additionally, educational game designers should clearly identify to players what the whole or unit is, since students have difficulty understanding what the whole is. Academics also argue for starting children off with concrete representations of fractions. Since we know that the pie and bar part-whole representations are pervasive, it may make sense for developers to use other concrete models in their games, like the set or number-line models. Using the set model can help combat students’ over-reliance on whole numbers and their misconceptions about a whole, while the number line model can help students realize that a fraction is a number with a value. Finally, in testing, designers should evaluate a student’s own conceptions of what the “whole” would be when interpreting the fractions in their game. The result of using all of the research would be a game that serves as a complement to what students learn in a traditional classroom curriculum.

Using the rational numbers literature as a basis, developers can make educationally informed decisions about how to approach fractions and then brainstorm a context or way of making fractions fun. In fact, educators are concerned with providing engaging contexts for children’s learning. Baroody and Ginsburg argue that these engaging contexts are important to aforementioned topics like assimilation, writing that assimilation is inherently tied to interest: “Like adults, young school children often do not make the effort to assimilate new information unless it makes some sense and is important to them. When a task piques their curiosity, children will spend considerable time and effort working at and reflecting upon it" (Baroody & Ginsburg, 1990, p. 56). With assimilation being linked to a fun task, and an aforementioned quote by Piaget stating learning can only happen when children actively assimilate (1964), we can tie appealing contexts to learning. Specifically, if educational games pique the curiosity of the children, then children will make more of an effort to learn the material. 13

While educators are concerned with providing engaging contexts, choosing the wrong context could also lead to confusion with the students; students can have different interpretations of what a context could be. One researcher came to this conclusion in an intriguing exercise where they asked students to figure out the fraction of people in their family that were children. In that exercise, researchers found that conversations arose among the students about who does and does not constitute as a member of a family (Ball, 1990). Ball further concluded that the creation of an entertaining and meaningful representation/context is difficult, arguing: “…a focus on making mathematics fun will justify some representations that are not grounded in meaning, that offer little opportunity for exploration or connections” (Ball, 1990, p. 33). Thus, the tension between fun and education is not a phenomenon exclusive to video games. Hopefully this review of the literature can lead educational game designers to create educational representations of fractions that are grounded in meaning – while at the same making a representation that is both fun and educational. Having reviewed literature that informs the decisions affecting the educational aspect, I will now shift to reviewing the literature that affects the fun aspect of learning games.

Researching the Fun – Empirical Research on Motivation

Figure 5: The Siren of Fun

While we have been using the term fun thus far, I want to introduce another word that I will also be using to discuss fun: intrinsic motivation. Intrinsic motivation is often described as a drive that one has towards an action that does not require external reinforcement (Harter, 1981). In addition to not requiring external reinforcement, there have been some scholars who link intrinsic motivation to the concept of fun (Malone & Lepper, 1987; Malone, 1981). Their work is relevant and warrants discussion.

The reason to focus on motivation is based on the issue that some academic games have had with creating fun experiences. In addition to engagement, motivation is thought to be significant to learning, with one scholar arguing: “Motivation is the most important factor that drives learning. When motivation dies, learning dies and playing stops” (Gee, 2004, p. 23). Therefore, if educators want children to spend more time practicing activities that they are not engaged in, it would help if they motivated those children. There needs to be more research and understanding of motivation in video game design experiments. Sustaining motivation is not a trivial task; even commercial game developers have had difficulty in sustaining motivation across time (Gee, 2004). If the goal is to create highly motivating environments, then those working in the fields need to better understand highly motivating games, leverage prior motivation research and understand it as a contributing feature of games. That research is discussed below:

Intrinsic Motivation In 1981, Malone published three studies that were conducted to examine how computer games might be intrinsically motivating. In Malone’s first study, Malone interviewed children on their game preferences and concluded that the game preferences were different for each child; there was no defining game that all children marked as their favorite. Malone then conducted his second study where he picked eight features to focus on in providing different variations of the game and measuring motivation. In the third study, Malone used results from the two earlier studies to refine experiments where elementary aged children played different versions of a

15 video game. Students were assigned to play with a version of a game. Each version of the game had certain features added, such as points, music, challenge, or fantasy elements. Students were left to choose whether they would play with the version of the game they were given, or a control game (Hangman). Malone then measured the amount of time they spent on the game versus playing Hangman, as well as students self-reported liking of the game, and whether they preferred the game they played versus the Hangman game.

Examining the results reveals that as features like points or music were added to the program, the motivation scores increased for both boys and girls. With each change or addition that Malone made to the program, a small change in the motivation score occurred. Although few changes seemed to provide a significant effect, when all the changes were compiled the difference in motivation was large. This leaves the question of which changes were most effective for learning unanswered.

Malone’s (1981) publication is a strong example and case for creating empirical studies on video games and motivation. Malone yoked conditions based on different versions of a similar game and had strong controls – a research element previous scholars had mentioned as needing to be present more often in the educational games research (Honey & Hilton, 2011). Malone’s initial study also led to follow-up studies that looked at using video games to increase intrinsic motivation. One set of follow-up studies was conducted by Cordova and Lepper (1996). Seventy- two students were placed in one of five varied conditions in order to play a number game. In this game, students had to use “order of operations” in order to create large numbers to win a race. The researchers decided to have one non-fantasy context and four fantasy conditions involving a space or pirate theme and back-story; the fantasy conditions were divided into a 2x2 design, with the variables being personalization (the subject’s name being used in the program) and choice of non-trivial factors of the game, such as their spaceship marker. The study then measured the student’s intrinsic motivation by measuring their self-reports of intrinsic motivation along with behavioral measures such as time on task, willingness to take on more challenging

16 options, and preference for other tasks. In comparison to the no-fantasy context, students tended to like the game more, were more willing to stay after class, would choose more challenging options, took fewer hints, and would show more signs of developing strategic play when they were given all three options (fantasy, choice, and personalization).

Using these two studies as the base literature on learning games and motivation, let’s examine how the aforementioned factors of fantasy and choice, along with challenge and structural qualities of games, tend to benefit the intrinsic motivation that players have when playing video games.

Challenge While they seem separate, challenge and motivation have been inevitably linked by prominent theorists (Csikszentmihalyi, 1990; Harter, 1981). In Harter’s work to devise a scale for measuring intrinsic motivation, she found that challenge had a direct link to motivation, and thus used challenge as a subscale to motivation. Challenge, along with curiosity and desire for mastery, were the three subscales that Harter found to be positively related to motivation. Using one element (challenge) as a proxy to measure another element (intrinsic motivation) illustrates how associated the two concepts might be. Cziksentmihalyi’s work also demonstrates the link between them.

Csiksentmihalyi is most famous for his theory on flow, which describes the experience some people feel when deeply involved in an activity where they are in their optimal state and highly motivated. One aspect of Cziksentmihalyi’s theory on how to get people to attain flow is to match one’s challenge to one’s skill level. Cziksentmihalyi argued that if people did not have the sufficient amount of challenge for their ability levels, they often fell into boredom. However, if the amount of challenge was vastly greater than their ability level, participants would become anxious. To maximize progress, ability and challenge must grow together, and be continually matched as they grow (1990).

Cziksentmihalyi’s stance on video games and flow, however, is a cautionary one (1988). He found that, for some games, the activities that are worthwhile in real life are trivialized into button presses in video games. Nevertheless, Cziksentmihalyi’s research on flow remains influential in video game design (Chen, 2007). His findings translate into emphasizing the importance of providing the right amount of challenge into an instructional game so that players are not alienated or frustrated. Perhaps the field might benefit from cutting edge research being conducted in computer science that helps adapt a game’s level of challenge to the player’s ability in real time (Hunicke & Chapman, 2004). Recent advances in these areas can hopefully support future efforts by game designers to provide the optimal level of challenge for their players, allowing them to sustain their motivation at peak levels.

Other Structural Qualities While challenge has been noted in the literature for keeping motivations high, there are also other structural qualities that have empirical backing for raising or sustaining motivation. These are ideas on winning, goal mentality, and simplicity. These are discussed briefly through referencing articles in a variety of fields from gambling to psychology that implicate games and motivation.

Structure of Winning Symbols One structural quality that derives from gambling literature is the structure of winning symbols. In an experiment in which Strickland and Grote studied slot machines, forty-four 16-year-olds were randomly assigned to play slots. The researchers modified a slot machine so that the likelihood that the machine would land on a winning symbol was 70% for the left column, 50% for the center column and 30% for the right column. These percentages were manipulated to be in reverse on a second machine, with 30% being in the left, 50% in the center and 70% on the right. The researchers measured how much longer subjects would play after they finished the required number of trials of playing the slot machine. Strickland and Grote found that subjects who were in the 70-50-30 condition tended to play longer than those that were in the 30-50-70 condition. This could provide an argument for making sure that

18 designers should try to keep participants involved in the game for as long as possible, without showing them their potential outcome (Strickland & Grote, 1967).

Goal Mentality Games provide a framework for play by providing rules and including at least one goal. Because goals are critical elements of games, it is also important that learning designers pay close attention to goals. Research has revealed that the mindset that one has when approaching goals can have large effects into the amount of motivation and continued persistence toward a goal. Elliot and Dweck conducted a study (1988) in which they had 101 fifth graders from a rural area perform a pattern recognition task. These fifth graders were divided into a 2x2 design. One manipulation was about goal mentality: whether they approached the goal with a performance-oriented mindset (Studying to get a good grade), versus a learning- oriented one (Studying to learn). The other condition was the learner’s pre-conceived ability manipulated with a prior exercise. Students were then measured on choice of the tasks, strategy improvement during tasks, and type of verbalizations during the task. They found that students in the performance-goal orientation would be highly affected by their pre-conceived ability while the learning-goal orientation would be unaffected. From their findings, the researchers urge an increase in promoting learning goals instead of performance, as learning-goal-oriented students performed the same regardless of pre-conceived ability and showed signs of high motivation. I do agree that most educational games could be constructed to promote a learning-goal orientation; one finding that also existed in their data was the high motivation and improvement scores of the performance group who had a high pre-conceived ability. Since video games are often extremely performance-focused, even in non- performance oriented games, game designers can still maintain motivation by giving players a lot of success early on. Having a lot of success can induce players to have a high pre-conceived ability, which could then be maintained. Furthermore, one can have these students perform well and improve in the face of adversity.

Simplicity Creating a game that is at its core simple eases research time and costs. On the other hand, creating a complex game is desirable for researchers because if done correctly, it could challenge commercial software that is comparable to commercial offerings. One issue with a complex game is that the game can become so large in scope that it becomes more difficult to manage and makes it much more difficult to complete (Brooks, 1975). An added challenge that occurs when creating a complex game is not only the time it takes to create the game, but the subsequent sunken costs of trying to refine the game later on. Sarama and Clements write: “research indicates that technological ‘bells and whistles’ should not become a central concern: While they can affect motivation, they rarely emerge as critical to children's learning” (Sarama & Clements, 2002, p. 101). One can relate the bells and whistles that the authors refer to as the complexity that is created in new games and making the case for simplicity. To avoid this demise, it is valuable to construct an instructional game that is as simple as possible in order for it to be manageable. Having a simple game on a focused topic also allows researchers and developers to understand the domain space and gain an understanding of the dos and don’ts for that domain quickly. There have been excellent examples of software that have been constructed that are strikingly simple, yet provide plenty of replayability and offer rich opportunities for discussion. One such project was the Envisioning Machine, created by Jeremy Roschelle.

The Envisioning Machine was a simple program that had a split screen: one side had velocity and acceleration vectors, and the other had a series of points to follow a curve. As their goal, students tried to move the two vectors to produce the sample curve that was displayed to them. Roschelle wrote an article which looked at how the Envisioning Machine allowed a pair of learners to construct shared knowledge as they worked together to match the curve (Roschelle & Teasley, 1994). Roschelle’s study provides evidence for the power of simple games and an existence proof that simple games can work. The software goes against providing cutting-edge graphics and modern gameplay, and as such reduces the amount of work needed to make the software. Reducing the complexity or scope of the project frees educational 20 game designers to concentrate on the two principles that are the most important to creating instructional games: 1) helping students learn and 2) entertaining those students while they learn.

Researching Elements useful both for fun and learning The last point in simplicity, while mostly focused on development, also hints at informing learning. If developers stick to working on and mastering a small concept, any game that is created will be more defined, providing a greater chance of producing an entertaining game and a refined curriculum. Increasing learning and increasing intrinsic motivation are often seen as two separate constructs that often affect one or the other. Ball’s argument cautioning against the use of fun contexts is just one example of how scholars position learning and motivation as competing for the same resources. Learning and motivation do not have to compete against each other. There are elements of a game that could contribute both to learning and to entertainment. For example, we know that feedback can help with instruction, however, providing variable rewards or feedback can also be thought of as providing a motivator for difficult work. Challenge and mastering more challenging topics while motivating can also be thought of as learning. As a prominent video game designer puts it, “Fun is just another word for learning” (Koster, 2005, p. 46). So far I have discussed elements that could help students with learning and motivation. There are a few elements of a game that I believe are both useful for fun and learning: control/choice, collaboration, and story. I will now discuss these mutually beneficial elements below.

Control/Choice

Figure 6: The Island of Choice and Control

Control and choice have already been mentioned to be beneficial in both learning and motivation (Cordova & Lepper, 1996). Though choice seems beneficial, additional work by Lepper and his students has shown that the issue of choice is more nuanced. For instance, in other work, Iyengar and Lepper (2000) have found that providing too much choice tends to work against the satisfaction that one may feel. In order to demonstrate this idea, Iyengar and Lepper discussed three studies they formulated to analyze the nuances of choice. In each of the three studies, adults were divided into conditions that would either allow them choice between a limited set of conditions, or choice among an extensive set of options. Using behavioral measures and self-reports of satisfaction, the researchers concluded that people in the limited choice condition would be more likely to be satisfied with their choice.

In designing future video games, the study suggests that we cannot simply increase the amount of choices players get in order to increase their intrinsic motivation with an instructional game. However, offering an extensive set of choices to users could be desirable in certain contexts. In their work, Iyengar and Lepper reported that people who had an extensive amount of choices would, on average, spend a longer time exploring the different choices. Thus, there could be scenarios in which time on task for some curriculum is vital, and one can envision designing a

22 game where students have an extensive set of choices to ensure that children spend the critical time learning in that environment.

In addition to quantity, another nuance of choice that Iyengar and Lepper examine is the notion of culture. A set of studies were conducted in which preferences and intrinsic motivation were compared between Asian-American and European- American families (1999). In one of the studies, 80 fifth graders were randomly assigned to one of three conditions in which they play a space-themed video game. The first condition allowed the student to pick their spaceship for use in the game, while neither the second or third conditions allowed it. The second and third conditions were told that someone else had chosen their spaceship for them. The difference in the second and third conditions was about who made that choice: a member of their in-group such as their peers or a member of an out-group such as younger students. Students were measured on the number of games they attempted, as well as pre- /post- test measures in math, preferred level of challenge and self-reported liking. They found that while Asian-American children tend to be more motivated when presented with the choice that their peers made, European-American children benefitted when they were allowed to make their own choice. Based on the results of this study, culture of the players should be another important factor that one must consider when thinking about designing choices for instructional games.

Illusion of Control Illusion of control is a theory that Langer began to write about based on a series of experiments. In the article by Langer (1975), Langer describes illusion of control as a process: “By encouraging or allowing participants in a chance event to engage in behaviors that they would engage in were they participating in a skill event, one increases, the likelihood of inducing a skill orientation; that is one induces an illusion of control” (Langer, 1975, p. 313). Langer’s statement means that certain outcomes that are often seen as events of chance can be made to seem as acts of skill. In one of Langer’s studies, 52 adult subjects participated in a lottery. The 2x2 experimental design was created, where Langer assigned people independently to the

23 two conditions. The first condition (choice) was manipulated by having the subjects participate in a lottery where they were given a choice to pick a specific card. The second condition (familiarity) was manipulated by having the lottery tickets be identified by letters or esoteric symbols. The day that the lottery was to be run, the experimenter would casually bring up into conversation the lottery and would offer the participants a chance to trade in their lottery ticket for another lottery ticket that had slightly better odds. The study measured the proportion of people that decided to trade in their values for the second lottery, and found a double main effect: both choice and familiarity were significant in their manipulations. People were less likely to give up a lottery ticket with familiar symbols that they chose.

The fascinating part of this story was that when people were given a choice among outcomes strictly governed by chance, people would still translate these chance outcomes to skill bets. As game designers, there are some ethical issues that surround such choices. One cannot simply transform the player’s notions of chance-based outcomes into perceived-skill outcomes, but we may exercise this with caution.

While choice and control are beneficial to motivation, it is also beneficial as a locus of control. Weiner and his colleagues have discussed how events can be perceived by students as either being in or out of their control. The important part comes in failure. Although not completely focused on learning, if a student fails and they believe the failure was due to their own performance, they are less likely to persist if they believe their failure was for internal reasons (i.e., they aren’t good at it). This is in contrast to if they believe the cause of their failure was due to external factors, in which they will not lose their aspirations (Weiner, 1985). Therefore, a student can attribute victories to their skill and ability but also attribute their failures to events outside of their control. While this seems to be a motivation story, it especially applies to learning because it helps us address and keep children who normally struggle in games to keep playing. With so many games and game designers wanting to equate game mechanics with skills, it is important that we do not lose this aspect of creating games. Using chance events and attributing those events to being in or out of

24 your control helps students who have a greater need to learn the material. Other than control, another element that can inform both learning and motivation in games is collaboration.

Collaboration

Figure 7: The island of Collaboration Working with others has profound effects on learning (Aronson, Blaney, Stephan, Sikes, & Snapp, 1978). Collaborating with others or even simply thinking that one is collaborating with another person improves the amount that one learns (Okita, Bailenson, & Schwartz, 2008). Collaboration has also been found to directly impact learning when children are playing games. In one such experiment, Guberman and Saxe (2000) help explain the phenomena of children playing games and learning complex materials, by claiming that when children are engaged in an activity they tend to work through the mathematics involved in the activity, putting their feelings about the topic aside. To prove their hypothesis, 96 third and fourth graders participated in a study where two of three conditions played a base-10 banking game made by the researchers called Treasure Hunt. The researchers split the third and fourth graders into mixed-grade and same-grade groups. They found that third graders who played with older students used higher-level strategies more often for arithmetic than children who played with third graders. The researchers attributed this finding to observations that were made in which the younger students had a distributed role in calculating how much change they should receive (Guberman & Saxe, 2000). This study highlights 25 how collaboration – specifically whom one collaborates with – can have an impact on strategies that one learns, while playing the game. The study also aligns with other research demonstrating the benefits of mixed ability collaborations in games, even when players are competing against each other (Nasir, 2005) as well as the benefits of having mixed ability groups in classroom settings (Cohen, 1994; Hooper & Hannafin, 1988).

Though the previous research has suggested that collaboration in mixed ability groups supports learning, merely introducing such collaborations in technology may not work without some thought on the outer context of the activity (Barron, 2003; Cohen, 1994). Thinking about and researching the learning that occurs outside of games has been advocated as needing more research (Stevens, Satwicz, & McCarthy, 2007). Stevens et al. found that there were many opportunities in which players were collaborating and learning from each other outside of the game context. Goldman and colleagues also found a similar result of having to structure organization and rules in the classroom in order to promote collaboration among students. They found that simply having technologies that supported the collaboration were not enough. They had to socially engineer situations in which the students were held accountable for the group’s knowledge and were given different roles and responsibilities that caused students to collaborate with each other (Goldman, Pea, & Maldonado, 2004). This type of social engineering aligns with conclusions made by other scholars: if there is no reason for students to interact, the students may work individually instead (Cohen, 1994). Cohen further states that the collaboration cannot be simple interdependence: “When group members are simply dependent on one another for resource (sharing information in the case of a navigation task) but do not share a goal, achievement is also impaired because interaction consists of one person trying to get information from another but perhaps wanting to avoid wasting time by giving information” (Cohen, 1994, p. 12). This statement poses a challenge one finds in such a situation which perfectly mirrors the typical prisoner’s dilemma situation, which is that one person acting selfishly can have detrimental effects for the collaboration of the group.

To avoid the possible failures in collaboration, scholars have tried to determine the optimal way of making sure that children’s achievement on collaborative learning tasks is high. For example, Slavin concluded that students be rewarded both as a group and individually after doing a review of the collaborative learning literature. Slavin wrote that holding the individual accountable just as much as the entire group enhances knowledge, calculation and application of principles (Slavin, 1980). Slavin’s theory on rewarding the group and the individual was also demonstrated by Yueh and Alessi, where they investigated three math classes using different systems for rewards. One class only received individual rewards, another only group rewards, and the last received both individual and group rewards. Yueh and Alessi found that the class that received both the individual and group rewards performed significantly better on direct learning items compared to the other two classes that had different reward systems (Yueh & Alessi, 1988).

In addition to its effects on learning, scholars have also argued for collaboration’s effects on motivation. For example, collaboration is cited as one of four important characteristics of motivation in a classroom (Paris & Turner, 1994). Video game design scholars also argue for introducing multiplayer aspects into games, stating that competition helps increase motivation (Fullerton, Swain, & Hoffman, 2004). Having multiplayer is not just important to designers, but to players as well. Self-selected participants of an online video games survey responded that having a multiplayer component was an important structural component of a video game – along with character development, customization options, rapid absorption rate, and realism (Wood, Griffiths, Chappell, & Davies, 2004). While “multiplayer” and “competition” should not be considered “collaboration”, any collaboration game with human players can be considered a multiplayer game. Furthermore, the video game industry press has noticed the increase in the number of commercially prominent software titles in which players can work together (Sessler, Webb, & Herter, 2009). Because this uptick is a relatively recent phenomenon, it could be that collaboration had yet to emerge amongst video game scholars as part of the multiplayer repertoire – as a complement to competitive video games. While competition and cooperation are 27 different entities, the recent rise in cooperative components in pre-eminent video game titles suggests that the commercial game industry is noticing the demand for cooperation and collaboration. One can infer that the demand would mean that people enjoy cooperating with others in these games.

While this discussion has pointed to collaboration and choice as supporting both fun and learning, I am interested in the ways that fantasy and stories in games support motivation and learning. Because of its importance to both learning and motivation, the topic will be reviewed in-depth below.

Section II – Reviewing the Literature on Fantasy and Story

Fantasy

Figure 8: The Island of Fantasy/Story

The educational benefit of adding elements of fantasy to an instructional, game-like technology was first researched by Parker and Lepper (1992). They conducted two studies looking at embedding fantasy of playing as pirates, detectives or astronauts, into LOGO environments. After a preliminary study, they conducted a follow-up investigation in which they were successfully able to link the fantasy contexts with improving student learning. Thirty-two third graders were assigned to one of three conditions, which were a student-chosen fantasy context (to be a pirate, detective, or astronaut), an experimenter-chosen fantasy context, or a no-fantasy context. After students had been assigned to a condition, they were repeatedly exposed to their assigned environment over multiple sessions, and finally were assessed via motivation measures and pretest/posttests scores from a LOGO Test. From the pretest-posttest scores on their LOGO assessment, Parker and Lepper found that both fantasy contexts provided significantly stronger gains in comparison to the no-fantasy context (Parker & Lepper, 1992).

This finding coincides with the earlier research done by Malone purporting fantasy as an important element to educational games. More recent work by Habgood and colleagues has argued for a refined model to fantasy. In their earlier work, Habgood, Ainsworth and Benford argue that merely introducing a fantasy context like Parker and Lepper (and Malone) had done is neither sufficient nor beneficial (2005). Rather, Habgood et al. suggest that intrinsic integration is what is vital to an educational game. Habgood et al. argue that intrinsic integration is important because it refers to how fantasy ties more directly into the game context. To demonstrate their point, Habgood and Ainsworth ran two studies with children playing a game called Zombie Division. The game has players navigate a protagonist through a 3-D maze slaying zombies along the way. As players go through the maze, they must pick the right weapon to slay a zombie. The way that players choose this weapon is at the heart of the difference between the two main conditions. In the intrinsic integration condition (see Figure 9: left), each zombie and weapon has a number, and players must choose the weapon with the corresponding number that will evenly divide the zombie. In the extrinsic integration condition, the numbers on the zombie are replaced with a set of symbols that represent which weapons the players could use. In the intrinsic-integration condition, players perform the fractions while they play the game. Contrast that to the extrinsic-integration condition, where players answer division questions at the end of each level, similar to a quiz (see Figure 9: right).

In their first study, these researchers randomly placed students in three classrooms (controlling for pre-test scores and gender) into these two conditions along with a control condition. The control condition looked just like the extrinsic condition except that it lacked the quiz at the end of each level. All students received a pre-, post-, and delayed post-test. This test consisted of 60 problems related to division – 40 multiple choice division questions and 20 general division knowledge questions. What they found was that the accuracy of the students in the intrinsic integration condition was significantly higher than of the students in the extrinsic and control conditions. The authors argue for operationalizing percent correct rather than the number correct in order to not penalize any students who took more time to use 30 elaborate strategies to find their answers. Habgood & Ainsworth used the results from this study to claim that the intrinsic integration of fantasy allowed the game to have an educational benefit. Their second study measured time on task to see how motivated and engaged students were in the different games. To measure motivation and liking, Habgood & Ainsworth measured time on task in free-choice play sessions for 16 students in an after school club and conducted interviews. The researchers found that the students played the intrinsic-integration version seven times more often than the extrinsic-integration version, allowing them to argue that the intrinsic-integration condition was also motivational (Habgood & Ainsworth, 2011).

Figure 9: Screenshot of intrinsic (left) vs. extrinsic (right) modes of Zombie Division (taken from Habgood & Ainsworth, 2011)

These studies demonstrate why fantasy is an effective element for making educational games more fun and more educational. While fantasy could be integrated with the core game mechanic, one area that is in need of more research is what Habgood & Ainsworth refer to as the outer-context for the game. Habgood & Ainsworth recognize this by stating that their fantasy of slaying zombies with numbers on their chests “could easily be replaced with an entirely different fantasy context (e.g.

31 evil robot enemies with monitors on their chests)” (Habgood & Ainsworth, 2011, p. 176). They argue that the context is largely irrelevant. That argument should be investigated further. If this outer context is largely irrelevant, then this means that it could be absent, or could be replaced with a fantasy that is not appealing. Habgood’s conclusions beg the question of how important fantasy really is, given Malone and Parker & Lepper argued that the fantasy context is important while Habgood and colleagues believe the integration with the game mechanic is what is important. More work needs to be done to investigate the importance that fantasy has on games.

Determining whether the fantasy context is important seems like a fruitful endeavor that can inform game designers’ future games. Determining this issue via research will also allow us to contribute to the field of games and learning by moving the field forward through research. Game designers and developers can benefit immensely from knowing whether or not the “outer context” is important to the creation of the game. If the context is not important, this means that they can spend that valuable time on other processes that can impact the game. However, if it is important, then it is wise for the educational game community to think critically about the fantasy context into which they place their educational game.

I review the literature on story, allowing for a better understanding of how to formulate an outer context that will improve the learning through a game. I mention only learning and not both learning and motivation since we have already mentioned how fantasy has improved the motivational aspects of games. Afterwards, I will discuss the process of how we created a game called Tug-of-War that has leveraged this research. Finally, I will detail two studies that help inform the question of how if at all a fantasy context and story impact the learning and motivation of an educational game.

Story Before we delve into story, we need to make a minor distinction between story and the fantasy contexts just discussed. Parker and Lepper operationalize fantasy by creating cases where: “a brief story and a few simple icons were used to establish a 32 context in which problem solutions would be seen as a means of fulfilling fantasi[z]ed goals” (Parker & Lepper, 1992, p. 626). This quotation implies their view of fantasy as being a story, and quite similar. While there are differences between story and fantasy, my conjecture is that all fantasy can be considered story. While certain fantasy scenarios may not make any mention of a story, we cannot ignore the reader or listener’s own story that they apply to such a situation. While I believe fantasy can be considered story, not all stories can be considered fantasy. Stories that depict day-today situations have not been considered to be fantasy by other scholars (Saltz, Dixon, & Johnson, 1977). Because these day-to-day situations can serve as valuable contexts for educational games, I will transition to using the term “story” and examine its research more deeply.

Story has many links to play and learning. For instance, story connects to play via role-play and thematic fantasy play, where children act out stories in a play fashion (Saltz et al., 1977). Researchers have argued that acting out stories in a play condition helps the youngest and lowest performing students understand stories better (Pellegrini & Galda, 1982; Williamson & Silvern, 1991). Many of the learning games have some type of fantasy or story embedded in them, with some games going as far as having the players role-play as another person. For example, one branch of educational games that use role-play is called epistemic games. The ideology for creating epistemic games is that students learn and take on the role of a doctor, scientist, or other professional worker (Shaffer, 2006). Providing a story has also been demonstrated to make boring tasks more interesting, which would provide a reason to include stories to make a task more intrinsically motivating (Sansone, Weir, Harpster, & Morgan, 1992). While most educational games apply a story, I have found only one research article mentioning story in educational games (Barab et al., 2007). With prominent authors advocating that story is one of the reasons video games have such great promise in transforming our society (Reeves & Read, 2009), it is important to review in-depth the story literature and consider its applications to educational games.

Many popular authors have argued that story helps people learn or remember information. For example, in a best-selling book Made To Stick, the authors theorize why some ideas tend to stick in people’s minds while other ideas are lost (Heath, Heath, & Kahlenberg, 2007). These authors conclude that there are six tenets for making ideas stick. One is Stories; tell stories to your audience to help them remember or rehearse events in the future. The other tenets are related to elements of a good story. For example, their tenet of Unexpected can be associated with plot twists and surprises that are often part of good stories (Truby, 2007) while their elements of Concreteness and Emotion, have been argued as crucial to a story (McDaniel, Hines, Waddill, & Einstein, 1994).

Popular authors have also argued that story is a critical component of a successful presentation. In his book, Reynolds (2008) documents his strategies for creating successful presentations. While one would expect points detailing the use of white space and imagery, the author also argues for story. Specifically Reynolds claims that presentations are more like documentaries, and the key to a good documentary is a good story. A good story that connects with an audience will go a long way in getting the main idea of the presentation to the users (Reynolds, 2008). Whether as sticky information or as a presentation, story serves a vital role in communicating ideas and more generally in education.

Story has applications in a variety of fields from courtroom and jury deliberations (Sunwolf & Frey, 2001) to therapy (Celano, Hazzard, Webb, & McCall, 1996; Gardner, 1971; Mills & Crowley, 2001). In therapy, people advocate for the use of stories in a variety of different therapies. One of the most famous therapy processes in popular culture is the twelve step program. Most American adults anecdotally know there are meetings that members have to attend as part of that program. In those meetings, people get together and recount their struggles with the challenges they encountered in battling their former vice. These struggles are stories – participants are telling stories to each other about how they overcame a difficult obstacle in their lives.

Popular culture aside, there are many therapy articles which discuss the powers of storytelling in helping patients cope (Celano et al., 1996; Gardner, 1971; Mills & Crowley, 2001). These argue that stories are great because they can be suited to an intellectually and culturally diverse audience. Kirkwood & Gold also made this point, when they wrote that such stories can: “…convey several levels of meaning, they appeal to a wide range of students and encourage each to explore his or her interpretations in greater depth” (Kirkwood & Gold, 1983, p. 352). With such a wide and diverse cultural tradition of stories (Sunwolf, 1999), stories have an almost universal appeal that can be leveraged when people tell stories to one another.

For a more concrete example, Smith and Celano (2000) reported on a small case study in which they were able to help an African-American child from a low- income neighborhood, who was in the care of his grandparents. They ended up using a mutual storytelling technique in which the therapist and the child constructed stories together, which were often analogous to the problems the child was facing. Rather than prescribing one solution, Smith and Celano wrote about how therapist provided a variety of solutions for the child in the mutually constructed story. This enabled the story’s protagonist to cope with problems in a variety of ways that the child would choose. The problems and strategies mirrored the child’s daily life and the therapist ensured all of the possible outcomes were desirable. From these sessions, the authors reported that while the child was initially uncooperative, that child later became engaged with the stories and reduced the amount of negative behavior that he was displaying after his treatment. The Smith and Celano article provides an example in which story was used to overcome barriers that were faced via cultural diversity.

While story has been used to overcome cultural barriers, cultural forms of stories have also been highlighted as useful. One such example is cuento therapy, one of the only techniques in the therapy literature mentioned to work empirically with Latinos (Altarriba & Santiago-Rivera, 1994). Specifically, cuento therapy used folktales taken directly from popular Puerto Rican folklore and relayed them to Puerto Rican Children either as/is or adapted to promote values like delay of gratification

(Mischel, Shoda, & Rodriguez, 1989). Recipients of cuento therapy as compared to a control and other therapy were rated as having significantly less anxiety, based on the State-Trait Anxiety Inventory (Costantino, Malgady, & Rogler, 1986).

Using stories to overcome cultural barriers has also been mentioned in Nursing education (Koenig & Zorn, 2002). Specifically, Koenig & Zorn argue that stories bring situations to real life, allowing different people to get different content specific to their needs and demonstrating it to others in the classroom. This works well with a diverse student body as the student body can cater the story to their own specific needs, something that was also highlighted in philosophy classrooms (Kirkwood & Gold, 1983). In addition to working with diverse students, Bartol and Richardson (1998) claim that stories allow one to become more culturally competent, citing the Color Purple as an appropriate cultural story that provides you with a microscopic window of another culture.

While stories have been mentioned as helping to address issues of diversity, others have simply argued for their use in teaching (Green, 2004), since stories used for teaching are present in so many cultures (Sunwolf & Frey, 2001). For example, some authors argue for the use of stories specifically meant for instruction, called teaching stories, which Kirkwood defines as: “brief, oral narratives told primarily to instruct, guide or influence listeners, rather than to entertain” (Kirkwood, 1983, p. 60). Kirkwood & Gold also promoted teaching stories by stating, “teaching stories can provide an intellectual challenge to students with differing cognitive skills, because they embody not just a mixed assortment of different themes, but a succession of meanings ranging from the readily apparent (although not insignificant) to the less obvious” (Kirkwood & Gold, 1983, p. 352). This statement illustrates that these authors believe stories are powerful because they can be used and interpreted at multiple levels. Kirkwood further argues for the use of stories to challenge people intellectually because stories can evoke an experience of non-rationality, an experience where a person’s awareness is confronted (1983). It seems the author

36 would argue that confronting student’s awareness, and confronting that awareness on multiple levels, would help with teaching content that a student may oppose initially.

As seen from the variety of fields, much has been written about stories because they are intriguing, and a valuable place to investigate more closely in order to inform educational games. Thinking more closely about the power of stories, Anette Simmons, author of The Story Factor (2006), spends the whole book making claims for why story is so powerful. Two arguments relevant to educational games are: 1) stories can provide vivid images and 2) stories can be used to persuade. These two factors have broad implications for education and educational interventions. Vivid images are important to educators because vivid images can be a factor of long term memories – a topic important to education. Persuasion is important to educators because it is a factor for changing behavior, a primary goal in education. Let us look more deeply at the literature concerning these two aspects. I will do so by first examining the literature on stories and comprehension followed by the literature on stories and persuasion. I will start with the former literature below.

Story and its impact on comprehension and memory As stories have long been a de-facto method of passing information down from generation to generation, stories have served as a vehicle for comprehension and transmission of information. Many scholars that have explored why people recall stories so well (Kintsch & Van Dijk, 1978; Mandler & Johnson, 1977; Rumelhart, 1975; Thorndyke, 1977). The most heavily cited theory comes from Mandler & Johnson (1977), who developed a model (also known as a grammar) which they felt stories should use. Mandler & Johnson tested first graders, fourth graders and adults on their ability to recall a story by telling it to someone else the following day. The researchers told each subject two stories: one that aligned to their model and another that disregarded the model, coding each person’s retelling of the stories for recall accuracy. They found that the story that aligned closely to their grammar led to much greater recall when compared to the other that had the erratic structure. Mandler & Johnson concluded their model for structuring stories was valid. This finding is

37 supported by other cognitive research around scripts, as a story can be thought of as a series of events or a causal chain that is well-known by the subjects (Schank & Abelson, 1975).

Black & Bower took the Mandler & Johnson model and others as inspiration to create what they claim is an improved model (1980). This story grammar, which is called the Hierarchical State Transition Model or HST, was a theory of encoding simple stories. In their model, Black & Bower claim that people remember with greater ease: higher level actions, completed sequences, and setting information that is part of the plot. Black & Bower referred to experiments that they had previously published to provide support for their points. They found that subjects would remember the main plot better than details, would more often remember a successful character sequence, and would more often be able to recall the setting information if it were integrated with the plot (Black & Bower, 1980). Black & Bower also argued that if a story has a goal, and the goal is not known, then the story becomes incomprehensible. The authors arrived at this point from an earlier set of experiments done by Bower. These findings are useful for stories and learning. Story authors that create a story with goals, tightly integrated settings, and completed sequences offer their audiences a set of information or rules that can be more easily remembered.

Out of the four experiments that Bower did, one experiment merits further discussion. For this study, subjects came in and read a boring text that consisted of five basic scripts for a young adult to participate in, such as going to the doctor for a checkup and getting groceries. Bower divided these subjects into four groups. The groups differed on two conditions, the gender and role of the story’s protagonist, and whether the character had an unusual motive or not. For example in the female gender condition, the unusual motive was “Nancy woke up feeling sick again and she wondered if she really were pregnant. How would she tell the professor she had been seeing?” (Bower, 1978, p. 216). After subjects read the boring text, they had to return the next day and retell the story. What they found was that while the unusual character conditions remembered more, they also remembered more false sentences

38 and would convert general scripts into character-specific scripts. For instance, the routine doctor checkup the subjects read in the unusual motive condition was converted and recalled as a pregnancy checkup. This finding is further supported by another theory claiming that gaps that occur in knowledge may be inferred and filled by the existing structures that students have (Kintsch & Van Dijk, 1978). The fact that people would not remember parts that did not fit into a goal was also demonstrated. The group that read about Nancy rarely remembered the grocery store script, yet the group that read about a hungry football player would remember the grocery store script. Bower then goes on to argue that if the reader conceives of a character with a specific problem, they use that as a base, and subsequently use that base to fill in possible gaps in their story (1978). This claim aligns well to the aforementioned gaps theory by Kintsch & Van Dijk (1978).

This series of experiments suggest that educational game designers who create a story with a clear motive or goal can benefit and influence what readers remember. Having tools available that can help players remember concepts is essential to educational game creators. Oftentimes they are tasked with making a game, the accompanying rules, and the curriculum - all parts that the student must remember for the educational game to be successful. Educational game developers can use story to assist readers in navigating the wealth of information needed to play their games. However, the research also demonstrated that we must be careful about the stories we use, as they can cause users to remember incorrect information with what they “think” they remembered. A way to circumvent this is to align stories with well-known scripts, thinking ahead to ensure the goals of the game and its story align closely to information that makes sense to the user and applies to how the game works.

Story and its impact on persuasion While I have contended that stories can help transmit knowledge, I also want to proclaim that stories can persuade. While persuasion and comprehension may seem like distant cousins, they are very interrelated because the knowledge that is comprehended and gained from stories can help supplant older knowledge. When

39 stories introduce new knowledge, they can be used to persuade people of new ideas. It seems intuitive that the arousal from stories and plot-twists (Bryant & Miron, 2003) is also instrumental in making a story an enjoyable experience. I believe a story can provide a fun environment, which when mixed with comprehension can help persuade someone or help them to learn a behavior or change a behavior. To help understand this relatedness, Figure 10 demonstrates how I would claim that stories can help with persuasion via comprehension and fun (motivation). When a story is first presented, one understands the information presented in a certain way. This understanding can then prompt an emotional response based on the story. This emotional response, combined with the information, can cause the story to persuade the reader. Persuasion as a matter is interesting to educational games and to education itself, because it can be used to help change behaviors and provide education on topics that are viewed more negatively or more often paralyzed as was discussed earlier.

Figure 10: My proposed relationship between story, comprehension, motivation and persuasion

Based on this claim, the substantial literature discussing the link between stories and persuasion will be reviewed below.

In the persuasion literature, the content and the viewer’s original viewpoint have been shown to be important in various models of persuasion (Hovland, Janis, & Kelley, 1953; Petty & Cacioppo, 1986; Sherif, Sherif, & Nebergall, 1981). We have

40 seen a link to this in the aforementioned quotes given by Kirkwood about teaching stories, which help introduce an idea to someone while they have their defenses down.

Stories can be used to persuade, and conversely, stories are frequently used to provide evidence or justification for facts. It is quite common to hear anecdotal stories as evidence for theories by a wide range of cultures at a variety of educational levels. The power that stories have as a persuasive form of evidence has not been lost by certain groups in the academic community. There is an entire field devoted to the effects and to designing interventions based on soap opera style stories, the field of Entertainment-Education or E-E. While the field of entertainment-education does embrace the edutainment and video game interventions as part of its field, the field is broadly defined as “the intentional placement of educational content in entertainment messages” (Singhal & Rogers, 2002, p. 117). Historically, E-E is noted to have started in the 1960’s by the work of Miguel Sabido (Nariman, 1993).

The catalyst for Sabido’s work came from a 1960’s a soap opera style drama, called a telenovela, which began in Peru and was rebroadcast in dozens of countries around the world (Singhal, Obregon, & Rogers, 1995). What made the soap opera unique is the message of empowerment that it gave the people of lower socio- economic status. The soap opera, which was about a working class maid who became illegitimately pregnant, but then climbed her way out of poverty to become a famous Parisian designer, motivated its viewers to earn an education and to make something of themselves. This message along with other factors such as the advent of the television drama made this a huge success around the world (Singhal et al., 1995). When Miguel Sabido had noticed this apparent phenomenon in Mexico’s rebroadcast of that telenovela, he built a team and created a few telenovelas around a series of topics that the government had asked for. Sabido created six telenovelas, which ranged in topic from family planning to adult education. Sabido also used principles from psychology, the arts, and communication to formulate the message and research the outcomes of his telenovelas, which were aired as commercial television programming (Nariman, 1993). The format and its message of hope had captured the

41 imagination of both researchers and producers. Researchers have gone on to define and study the world of entertainment-education, and producers have followed Sabido’s model to produce mass media telenovelas with social justice messages in their respective countries around the world (Singhal & Rogers, 2002).

Effects from telenovelas such as these are evidence that stories can serve as powerful justifications. In fact, researchers have compared stories to statistical information to examine which medium is a more powerful form of evidence to the public, with literature reviews on the topic being unclear as to which format was more persuasive (Allen & Preiss, 1997; Reinard, 1988; Taylor & Thompson, 1982). Another scholar who read the three reviews and used the information from the literature to analyze 17 articles based on the persuasiveness of health literature also found mixed results as to whether stories or statistics were more persuasive (Winterbottom, Bekker, Conner, & Mooney, 2008). The finding of mixed results should not deter one away from the simple fact that for some, stories rival statistical information in their persuasiveness. There has been other research that has demonstrated that stories do not even need to be real or true stories, the stories can be claimed as fiction, and yet still be influenced by the story (Green & Brock, 2000; Strange & Leung, 1999).

To demonstrate the blurred reality of fact and fiction, Strange & Leung ran an experiment where they had 95 people read a 900-word story about a student who dropped out of school. The subjects were divided into four different conditions on two different variables. The first variable was related to the content of the story. While all subjects read about a high school student who drops out of school, the study manipulated the student’s reason for dropping out. For half of the subjects, the reason was focused on dispositional factors such as the student not liking school. For the other half, the reason was focused on situational factors, such as the school not having the resources to successfully support the student. The subjects were also split amongst a second variable: telling the students whether the story was fact or fiction. After reading the story, they were asked to recall the story and were given some survey

42 items to respond to. The survey items included engagement with the story, urgency of the dropout problem, its need to be addressed in setting future policy, the common causes of high school failure and the factors associated with that failure (Strange & Leung, 1999).

What Strange & Leung found was that while the fact versus fiction manipulation did not have an effect on the subjects’ beliefs about the causes of dropping out, the situational versus dispositional factors did have an effect. The situational/dispositional factors also were associated with differences in responsibility judgments. For instance, the subjects who read the dispositional stories thought the causes for dropping out were based more on the individual; subjects felt the individual in the story was responsible for dropping out. In contrast, those subjects that read the situational text felt the responsibility and the cause was on the school and the teachers. However, how strong they felt one way or another was also moderated by how engaged subjects were towards the passage (Strange & Leung, 1999). Strange & Leung’s findings in this research also shed light on a new and important point. No matter how much evidence is presented via research about stories and how persuasive they are, if the story’s content is not engaging to its audience, the story will be ineffective at persuading its audience. This point applies to educational video games and is analogous to aforementioned troubles that certain games can have if they are not fun or engaging.

Strange & Leung’s finding about the content being an important factor for stories also arises in an analysis done by Winterbottom and colleagues (2008). Their mixed results finding led the author team to postulate factors for w hether a story is more persuasive than statistical information. These variables are vividness of the information, credibility of the message content, and credibility of the speaker. All of these variables are related to the content of the message, which other scholars have argued to be just as important. While credibility is at the forefront of most persuasion literature as being one of the factors for being persuasive, vividness is a topic that is highlighted as critical to persuasiveness by other authors (Sunwolf, 1999). Stories

43 have the ability to create a vivid experience for the reader that is completely fictionalized. Video games can augment such an experience by allowing players to assume the role of characters in these stories.

Using another term and theory that is similar to vividness, another set of authors decided to use a different framework for examining the persuasiveness of stories (Green & Brock, 2000). This framework is called transportation. Green & Brock use the word because it evokes the idea that it transplants the user to another world. In a set of four experiments, the authors use the idea of transportation to combat a heavily-cited persuasion model called the Elaboration Likelihood Model, or ELM. ELM is a model for persuasion that discusses two routes for persuasion: the central route and the peripheral route. A person who is about to be persuaded can decide which one of the routes to use to process new information. If one uses the peripheral route, then they will use peripheral characteristics such as credibility of the speaker or mood to process the information. By using the peripheral route, one will not examine the content of the message closely. It is easier for one to accept information if it is processed via the peripheral route, but the idea or concept is also more likely to be changed by another idea later on. If one uses the central route, then they will closely examine the content of the message, generating counterarguments as they process the message. This means that it will be harder for one to be influenced if they process a message through the central route, but it will be more likely to stick (Petty & Cacioppo, 1986). Green & Brock did not feel that people who read a good story would not generate many counter arguments. However, they did feel that readers deeply process the information in the story, causing them to be transported. According to ELM, if readers process the information from the story through the central route, they must generate more counterarguments. In a set of four experiments, Green & Brock found the opposite. These scholars were able to manipulate the amount of transportation that readers felt, and found that the more transported a reader felt, the less amount of counter-arguments they generated (2000). This finding provides evidence again for the content of the story being important. It also demonstrates that stories are different. When stories have the ability to transport 44 the user, stories can have the effects argued by Kirkwood, stories can bring the defenses down of a person who objects.

However, while we have focused on the persuasiveness that stories have, we only know based on these studies that the effects are immediate. There are multiple studies that demonstrate that the effect of persuasiveness that comes from stories decreases over time. Concerned with the rise of hate groups online, Lee and Leets measured teenager’s responses to online-hate narratives and the effects of those narratives over time. To answer their question, the researchers had 180 self-selected adolescents complete a survey measuring their predisposition to white supremacy. After completing the survey, the adolescents read one of five types of counterarguments. Each counterargument was given a low and a high narrative version, and an implicit or explicit final conclusion. Afterwards, researchers asked the subjects to rate the persuasiveness of the argument, to describe their feelings, and to generate counterarguments. Two weeks after having been introduced to the argument, subjects were asked to fill out the same attitudinal survey they filled out at pretest. While the researchers reported many interactions in their findings, the most novel finding from this research is that the high persuasiveness that the stories had on subjects often faded after two weeks (Lee & Leets, 2002). The fact that persuasive benefit received from reading stories faded has also been demonstrated in another research study.

In a study done by Watts, students read a persuasive argument or constructed an argument based on a prompt. Watts used three issues that he found to have a relatively equal number of proponents and opponents, and assigned each section (six sections, 140 people total) of an introductory psychology class to participate in one of the issues as passive or active participants. Participants rated three attitudes per issue (1-15 scale), for both the one they completed and the other two. Six months later, the participants re-took the attitude survey, indicated subsequent involvement in issues, and were asked to recall the attitude they had six months earlier. What Watts found was that the initial opinions of each attitude shifted significantly for both the reading

45 and the writing conditions compared to a control condition. However, six months later, the difference in opinion for the writing position increased from their initial point of view. In contrast, the scores in the reading condition attenuated to the point of no longer reaching statistical significance from the other subjects who had not completed an activity on that issue. The difference after six months became more pronounced when the researchers took into account whether subjects had correctly recalled the stance they were asked to take (Watts, 1967). Both Lee & Leets and Watts seem to determine that simply reading a story will initially be persuasive, but the persuasive effect will attenuate over time. However, Watts study does point out another finding, which is that active participation in the study allows one to keep being persuaded. Active participation by Watts was defined as writing a paper for or against the issue (1967). These subjects’ persuasiveness did not attenuate. It is this active participation or direct experience that has also been shown to be a quite powerful motivator among other scholars (Fazio & Zanna, 1981; Regan & Fazio, 1977).

After reading the previous paragraph, a reader may then discount stories being used in video games. However, video games and the use of a story in a video game is not the same as reading a story. One way that video game stories differ from traditional stories is that oftentimes videogame stories are an accessory of the game. Videogame stories are often made to drive the videogame experience. Players do not simply listen to a videogame story; they engage with the story and actively participate with it. This role-play has already been demonstrated with serious games such as PeaceMaker, which places players into the shoes of rivals and asks them to engage in a peace process. It is my theory and hypothesis that stories in games do not equate to stories given as texts. While stories that contain a text can be thought of as passive experiences, stories that are embedded into a video game not only provide a passive context, but also provide a direct experience or role play on the story that a player can participate in. This causes the story to be a different experience. Educational game scholars have yet to explore pedagogically how story interacts with role play. While all the literature on story does not directly apply, it can help the educational games community research the most pressing topics. Then, running studies to advance 46 educational games knowledge can inform the community of future directions. Studying how story research can be applied to video games and whether it makes an impact at all is a great first step.

To summarize the review, I have found evidence for proponents of stories in a variety of fields. Scholars have argued that story can help people remember information when it is correctly structured and when stories are given a clear motive. Other scholars have also thought of stories as being persuasive, often comparing their effectiveness to statistics. Stories have spawned movements to empower adults and create social change via mass media. From an experimental view, the persuasive power that stories have – has been linked to the vividness of those stories and how they are framed. How engrossed one is in a story suggests how persuasive that story is to them. How stories are framed provides readers with different beliefs without many counter-arguments. Other scholars have found though that these persuasive effects are mostly immediate and that they attenuate over time, providing a challenge for stories. The limitations of passive stories’ persuasive effects have also been argued as not being as powerful of an indicator when compared to direct experience. We do not yet know whether video games can be construed as passive stories or as active experiences. I believe stories in video games are well poised to be effective in persuasion and learning. Players are not participating in a passive experience, but rather are actively a part of the story. This literature review demonstrates the fertileness of stories and videogames as an area of further research.

With literature as background, I now delineate the process of creating an educational game and providing new knowledge through experimentation. First, I detail the process of a game called Tug-of-War. I document the creation of the game, including how Tug-of-War leverages the research discussed in this paper to create a successful educational game. Presenting the process in detail addresses three goals: 1) it makes the process of game creation transparent, 2) it demonstrates the delicate balance between fun and education, and 3) it provides background for the readers to understand later studies. It is my hope that the process and studies will help future

47 game creators and developers navigate around the sirens and provide them a better path to finding the successful educational game.

Section III - The Process of Creating Tug-of-War Tug-of-War was born from the desire of a team of Stanford educational researchers (Dylan Arena, Ugochi Acholonu and I) to create an educational game. We wanted to use the perils and previous failures as guides to help us navigate the gentle balance between fun and learning. We felt that leveraging insights gained from the design community would help us easily achieve this goal.

The Initial Tug-of-War Concept We started our design of “Tug-of-War” by agreeing to three ideas. The first was that we were going to concentrate first on creating an uncomplicated physical, non-technological prototype (Rick et al., 2010; Sefelin, Tscheligi, & Giller, 2003). Like the envisioning machine (Roschelle, 1987), the result of starting with such a simple, yet functional prototype enabled us to refine the game play without having to deal with the complexities of writing software. Second, we chose the domain of teaching fractions to students, as fractions is one of the most difficult concepts for children in elementary education (Petit et al., 2010) and also one of the first topics that causes children to stray away from a STEM pathway (Nunes, 2006). Finally, to help ensure the game was enjoyable, we wanted to build our game using an existing and popular style of game, the card game (Parlett, 1991). Specifically, we thought to leverage the rules and gameplay of certain games like Magic: The Gathering, Yu-Gi- Oh!, and Pokémon, the latter which have been studied by educational researchers and found to be both entertaining and complex (Buckingham & Sefton-Green, 2003; Ito, 2006; Vasquez, 2003). Gameplay in this card genre involves assembling “troops” and choosing cards from one's hand to attack an opponent's troops or defend one's own. In Tug-of-War, the “troops” are groups of teammates on either side of a tug-of-war line, and the “attacks” and “defenses” are pranks (e.g., stink bombs) or fibs (e.g., “I hear the ice cream truck!”) and their countermeasures (e.g., air fresheners and radios).

Once we settled on the game genre, we embedded the learning objectives within the game's narrative and mechanical structure. This technique, described by Malone (1981) as intrinsic fantasy, has been more fully explored by Habgood and 49 colleagues (Habgood et al., 2005), who expressed it in terms of flow, core mechanics, and representations. The basic notion is that the game elements that are essential to learning should be incorporated into the narrative flow of game play, linked to the core mechanical operations players undertake in the game, and enacted using pedagogically sound representations to anchor thinking about the learning objectives.

Consistent with this notion, the cards for attack and defense in Tug-of-War (stink bombs, air fresheners, etc.) fit into the narrative of a playground tug-of-war. Mechanically, each card contains a rational number (represented as a fraction, decimal, partially filled meter, or ratio) that is applied to one of the whole-number teammate groups, weaving our two learning objectives seamlessly into the basic game mechanic. An example shows how the game represents fractional operations on whole numbers. Figure 11 shows a group of 8 teammates (the Johnson Family), and Figure 12 shows an attack card (a Stink Bomb) with the value 3/4.

Figure 11: The Teammate Card Figure 12: Attack Card

To play the Stink Bomb on the Johnson Family, players would first decide how to split the Johnson Family into four equal groups (because that is the denominator of the Stink Bomb fraction); once those groups were formed, players would choose three of those groups (the numerator of the Stink Bomb fraction) to be scared away from the Tug-of-War by the Stink Bomb. This process of forming and choosing subsets of whole-number quantities is our main representation for fractional operations on whole numbers in Tug-of-War, and is part of the partitive-whole representation researchers 50 found to be a critical component missing in fractions knowledge (Confrey, Maloney, Nguyen, Mojica, & Myers, 2009; Kerslake, 1986; Petit et al., 2010) necessary to help students learn. Tug-of-War uses the “set model” to discuss fractions by having students deal with fractions using a set of figures (Petit et al., 2010).

To address the issues of control we had mentioned, we gave players a limited set of choices to make sure they are not overwhelmed or unsatisfied (Iyengar & Lepper, 2000). We gave players the illusion of control (Langer, 1975) by having them select a chip from a bag that they cannot see, essentially a random event. We included this action to make sure that students engaged with the game, and could use that chance event as a source of blame if the players performed poorly, which also references the locus of control when playing a game (Weiner, 1985). So if a player loses in the game, that player could blame it on the chance event. The randomness also helps with the structure of winning, as the wild swings in the number of points makes the students feel like they are still in the game (Strickland & Grote, 1967). This randomness and ability to blame outside circumstances also allows some of the lower performing students to stay engaged with the game, and not think of the game in a performance-oriented mindset (Elliott & Dweck, 1988). We expanded on this dis- association by making sure that winning in the game was not directly tied to their fractions performance. While linking in-game skills to academic performance is a common mechanic in most games, we avoided this to keep the lower performing students trying. If students felt that the mechanic was internally related to their performance on fractions (“I lost this game because I’m bad at fractions”), then according to Elliott and Dweck they could be less likely to play the game (1988).

Our basic design in place, it was tested internally with colleagues, and then with sixth or seventh grade students, who were older than our ultimate target demographic. This allowed us to work out issues and refine the game so that it could work well for a fourth grader. Each time that we had a user study, we would make modifications and interview users on the problems they had. This process allowed the game to move from being complicated to being understandable and playable within a

51 short amount of time. After this initial period, we conducted user studies with children that were closer to the target age, introducing them to the game for a one or two hour session getting input from children as well as parents while checking for enjoyability and understanding. We used these user studies to iterate further on our design. Finally, we had extended play sessions in local after-school clubs that were recorded and analyzed to resolve any problems with understanding of the rules, boredom, or unsatisfying game play.

In addition to the game content, we looked for problems with the educational fractions content. We realized quickly that children had trouble executing the fractional operations that occur in game play. To help students visualize and think through the operations, we added manipulatives. Tangible objects (e.g. small pie- wedge pieces) have been shown to support students’ transition from natural to rational number interpretations (Martin & Schwartz, 2005). The researchers also added instruction to go along with the manipulatives, teaching them a fair-share dealing strategy and leveraging their prior knowledge about partitioning (Confrey et al., 2009; Wilson et al., 2012). The addition of stylized miniature plastic people both supported our narrative and offered a concrete (instead of abstract) representation for our central mechanic (Davis, 1984). In addition, we also thought about the challenge our game poised. To ease the challenge into a more flow like state (starting easy and getting more difficult), we started the students only using fractions cards that had halves so that children solely needed to know the halving strategy. That strategy is one of the initial strategies students encounter with fractions (Petit et al., 2010). Our interaction and subsequent changes to our game from that point made us feel confident that we had an enjoyable game that would help fourth graders with fractions. This process and feeling led us to run the first study.

Study 1

Method

Subjects In the first study, we used thirty fourth-grade students (fifteen boys and fifteen girls) from one classroom in the San Francisco Bay Area. Based on the students’ third-grade Standardized Testing and Reporting (STAR) score, approximately 83% of students were categorized as economically disadvantaged, and 73% were categorized as English language learners.

Experimental Design With the two samples, we conducted a quasi-experimental study, where one group got the treatment for four sessions initially while the control group completed another, unrelated task with other researchers. Subject’s assignment to condition is documented in the procedure section.

Procedure We administered a pre-test of fractions concepts to the class. The results of that pre-test were used to help divide the class in half (15 and 15) to balance gender and fractions achievement. One half (the Game group) played Tug-of-War for three 60-minute sessions per week for three weeks, in place of the students' regular math work, while the other half (the Control group) participated in unrelated research. We then re-administered the same math test to the entire class as a post-test.

In the first one-hour session, researchers explained the game and divided students into single-gender, mixed-ability groups of three or four students each. One researcher acted as the “dealer” for each group, structuring the game session, breaking the group into two teams, assigning tasks (e.g., dealing cards, organizing manipulatives, and keeping score) to individual students, helping clarify rules, and adjudicating conflicts. In each of the next two weeks, students played for another one- hour session with the same groups (possibly changing teammates); researchers

53 continued to moderate for each group but made a conscious effort to withdraw slowly to more peripheral roles as students became familiar with the game.

We addressed the collaboration issues by having the teacher assign students of mixed ability into one team (Cohen, 1994; Guberman & Saxe, 2000; Hooper & Hannafin, 1988). Players worked in teams of 2 and played against another group of two. In the case where the teams were uneven, one person would play against two; we made sure that individual players were paired in the subsequent session. At times, the experimenter served as a teammate if the student needed help. Students were given different teammates each session.

Measures We constructed a fractions-concept assessment with 14 standard items relevant to basic fractions concepts (e.g., ordering of 1/2, 2/3, and 1/3; identity of 1/2 and 2/4; etc.) and administered it as a pre- and post-test. This instrument served as a pilot for our instrument in later studies.

To measure fun, we administered periodic surveys of students' enjoyment of Tug-of-War in this and subsequent studies. These surveys had five items, shown in Table 1 (on page 60) that were designed to assess students’ intrinsic motivation with the game. We adapted our measures from components of Harter’s (1981) intrinsic motivation work. Each item had a five-point Likert scale whose response options were anchored as follows:

• NO WAY! (1) • Not really (2) • Maybe (3) • A little (4) • YEAH! (5)

Results and Discussion Figure 13 below displays the average pre- and post-test scores for each student group. The results from the fractions assessment did not differ significantly between

54 groups or across time. At the outset of the study, the hypothesis was that the game would provide evidence of learning gains through the assessment. It did not. However, we did find that the children’s average scores on all the fun items approached ceiling each time they were surveyed. Our survey results, group meetings, play-testing, and classroom observations convinced us that we had designed a fun game.

Figure 13: Pre- and Post- test scores for our first Physical Prototype

Nonetheless, our brazen, year-long attempt at creating an educational game humbled us. The result of our careful design process led to having a game that could not demonstrate kids having learned anything via a paper and pencil assessment. While our observations led us to believe that the children had gained fractions knowledge, we could not substantiate those claims with the evidence we had collected. Having experienced the same troubles of the early edutainment titles led us to understand the sheer difficulty of creating a game that was both fun and educational. We had not been able to dislodge ourselves from the siren of fun (See Figure 14).

Figure 14: Tug-of-War’s path for its first physical prototype was very fun, but did not demonstrate learning Our second physical prototype We reflected on what we thought worked and didn’t work. After reflecting on observations and data, our team found three areas in which we might have fallen short of our goal. These were: 1) the process the children had to take to learn the game’s rules, 2) the actual educational content that we wanted the children to learn, and 3) the choices made about how to assess the game’s effectiveness.

To address these failures, we simplified the game even more and streamlined game-rule teaching by creating short videos to introduce the game to all of the children at the same time rather than teaching groups separately. Next we decided that the game’s strength was in providing a meaningful and situated context for children to practice finding fractional amounts of whole numbers (e.g., 3/4 of 8). Identifying this critical learning component helped us refine the game to maximize exposure to this mechanic: game elements that were not helping with this concept were reduced or eliminated. For example, our game had money cards, which could be used to buy more cards. Students enjoyed using the cards, but the cards did not succeed with the

56 educational goal we had intended for them (understanding of decimals) and did not support our now-central mechanic of taking fractional amounts of whole numbers, so we cut them from the game. Eliminating the money also allowed us to make the story have a better narrative argument (“Why should money allow you to buy extra fibs?”) (Fisher, 1985). Focusing on a single mechanic and concept also helped us focus our assessment: we included more items that required finding fractional amounts of whole numbers while eliminating items that were less relevant to the primary learning content of the game.

One surprising finding was the difficulty we found with creating a gradually more challenging experience. We introduced the children to halves only – thinking that it would match children’s levels of challenge and would align to their initial real- world representations. After introducing halves, we would introduce fourths and thirds into the set of cards after each session, making the card set gradually more complex. However, what we noticed was that children would confuse the halving operation and types of cards with the game’s mechanics. Because the fractions of the stink bomb and air freshener were always one half the first time they played the game, children would not perform the fraction, but would rather think of the air freshener as cancelling the effects of the stink bomb. In this case, trying to provide an easier initial state caused the students to be confused with how the game worked. From a story standpoint, it also makes sense that they thought the story was that the air freshener cancelled the stink bomb, rather than reduced the stink bomb by a certain amount, which could have demonstrated a fault in how integrated the mechanic was to the story of the stink bomb. With these changes in place, we decided to run a second study the following year.

Study 2

Method

Subjects Thirty-one students (15 boys and 16 girls) from one fourth-grade class participated in the study. On the students’ third-grade Standardized Testing and Reporting (STAR) report, 91% were categorized as economically disadvantaged, and 75% were categorized as English learners.

Experimental Design We administered a pre-test of fractions concepts to the class (with no feedback about right or wrong answers), which was then divided in half to balance gender and math achievement. One half (Cohort 1) played Tug-of-War for one 75-minute session per week for seven weeks, in place of the students' regular math work, while the other half (Cohort 2) participated in unrelated research. We then re-administered the same math test to the entire class (again without feedback) and switched conditions; Cohort 2 played Tug-of-War for six weekly 75-minute sessions while Cohort 1 participated in unrelated research. Finally, we administered our math test a third time.

Procedure In the first weekly session for each cohort, students learned how to play the game by watching a short instructional video. They were then assigned to mixed- ability groups of three or four students each. (We used videos to ensure that both cohorts received identical instruction.) One of four researchers worked with each group, forming teams, assigning tasks (dealing cards, organizing manipulatives, and keeping score) to individual students, clarifying rules, and adjudicating conflicts. Subsequent sessions proceeded similarly: videos were played to reinforce various aspects of gameplay, and groups were periodically rearranged to provide novel opportunities for collaboration. Our team continued to moderate for each group but gradually withdrew to more peripheral roles as students became familiar with the game. The “paper method” was introduced via video in the fourth session. By the

58 sixth session students were encouraged to rely entirely on the paper method or mental calculations and use the manipulatives only if they became stuck; students were also encouraged to run the entire game session on their own, with researchers observing but not interacting unless necessary.

Measures We devised an assessment that was administered as a pre-, mid-, and post-test. Our assessment was revised from the previous 14-item assessment. The new assessment contained 22 dichotomously scored items testing our learning objectives: performing fractional operations on whole numbers (e.g., “Circle 1/4 of these 12 stars” or “What is .8 of 10?”) and reconciling different representations of the same quantity (e.g., “Which one has the same value as 2/3?”). (For this instrument, see Appendix A – Early Tug-of-War Fractions Assessment)

The classroom also underwent its annual state-mandated California Standards Test (CST) administration after only one session of Cohort 2’s gameplay, thus providing a rough natural measure of external validity for our assessment game. The specific measure we looked at was the CST subtest dealing with decimals, fractions, and negative numbers.

Fun Results and Discussion The average scores from both studies, as well as the standard deviations (in parenthesis), are displayed in Table 1.

Physical Physical Prototype Survey Item Prototype 1.0 2.0 I had fun playing this game 4.88(.39) 4.96 (.29) This game is boring 1.72 (1.28) 1.26 (.77) I would want to play this game again 4.65 (.9) 4.78 (.66) This game is too hard 2.12 (1.52) 1.59 (1.15) Sometimes I got frustrated while I was 2.44 (1.61) 1.76 (1.12)

playing Table 1: Cohort 1 Fun assessment results from our two prototypes

From this table, one notices that the students had fun in both iterations of the game. There are trends that indicate the game was not as hard and frustrating the second time around. This could be due to the streamlining of the rules: both in the number of rules and in how they were taught to the students. Teaching the rules felt more successful because we now had experience with knowing what students found confusing about the game. To supplement the data, we also had anecdotal information about the fun level of the game. Several children asked us whether they could buy our game in stores to play with their friends and family. The changes to the game did not diminish the fun.

Learning Results and Discussion As shown in Figure 15, the two Tug-of-War cohorts did not differ at pre-test, t(27.55) = -.17, n.s. Students in Cohort 1 showed significant gains from pre- to mid- test, t(15) = 9.05, p < .0001, whereas students in Cohort 2 did not, t(14) = .83, n.s. Once Cohort 2 got to play Tug-of-War, they showed significant gains from mid- to post-test, t(14) = 6.71, p < .0001, while Cohort 1's scores did not change significantly from mid- to post-test, t(15) = -1.35, n.s., even though they had not played the game for almost 3 months. Thirty out of thirty-one students showed learning gains from playing Tug-of-War.

Figure 15: Pre-, Mid-, and Post-test Means and Standard Errors

As shown in Figure 16, Cohort 2’s performance on this subtest did not differ from that of the classroom teacher’s students from the year before, t(24.81) = 0.32, n.s., while Cohort 1 outperformed both the previous year’s students, t(40.14) = 3.17, p < .005, and Cohort 2, t(21.46) = 2.13, p < .05, despite the fact that Cohorts 1 and 2 had been created to balance math achievement, including achievement on their third-grade CST math scores, t(25.84) = .61, n.s. The “champagne graph” style of Figure 16 (inclusion of individual observations in the bar graph) illustrates that gameplay seems to have reduced bimodality in Cohort 1: one interpretation of this is that playing Tug- of-War especially helped lower achieving students. 1

1 The results for this second study and a brief explanation of Tug-of-War used here have been published in Jiménez, Arena, and Acholonu, Tug-of-War: A Card Game for Pulling Students to Fractions Fluency, Proceedings of the Games, Learning, and Society Conference 7.0, 2011.

Figure 16: CST Subtest Individual Scores, Mean, and Standard Errors

We had two different measurements displaying a difference between the students who had played the game and those who didn’t, allowing us to feel more confident in our results that the children learned the fractions content.

The Digital Prototype The success with both fun and learning in the fourth-grade classroom provided a strong foundation for creating a digital implementation of the game. In the transition from a physical lo-fi prototype to a digital version, our main goal was to preserve as much of the game’s flow and mechanics as possible in the new medium. We relied on our own experience as Tug-Of-War moderators to help us implement the mechanics and dynamics of the computer game (Hunicke, LeBlanc, & Zubek, 2004). The mimicry of our human performances proved difficult, not least because the digital version of the game presented many new game possibilities. These desired features might have made the game easier to play or been great extensions in their own right, but we decided to be cautious and did not include features that could potentially compromise players’ strategy building and practice with fractions.

One example of a new feature’s impact on mechanics was how illegal moves would be handled. Initially, we thought it would be great for the computer game to immediately correct the student whenever he or she played an incorrect card, but this

62 convenience came with a side-effect: students would miss the opportunity to understand why such a move was wrong and to perform the fraction calculation needed to realize their mistake. Instead, our digital version mimics what the moderators of the physical prototype would do: allow the child to discover the mistake by asking him or her calculate the fraction. Another example of letting the player see the reasoning unfold is with the “Show Me How” button (see Figure 17: A screenshot of the work area students have when solving problems.). This button provides answers to students who are stuck, but instead of producing an answer immediately, the system steps through a slow animation illustrating the process of finding the answer. Once pilot testing with older students in classroom settings confirmed that the digital game was relatively stable, we were ready for the next classroom study.

Figure 17: A screenshot of the work area students have when solving problems. Study 3

Method

Subjects The participants for this study were the third cohort of children from the same fourth-grade classroom used for the two prior studies. Of the 30 students in the classroom, two students classified as special-needs children were excluded from this analysis, leaving 17 males and 11 females.

Experimental Design We used the same experimental design as the previous study - a treatment/control crossover design with stratified random assignment. Participants were ranked according to their third-grade standardized-math-test scores and their initial scores on our fractions assessment. Then participants were placed randomly into two groups to balance math achievement. The first group, the Treatment Digital group, played the game while the other group, the Delayed Digital group, participated in unrelated research on negative numbers. After six sessions with the game, the groups switched conditions and the Delayed Digital group played our game while the other group participated in unrelated research.

Procedure In the first session, students watched two explanatory videos, one video presented the mechanic of how to perform a fraction while the other video introduced students to the gameplay. Because there was not a separate paper method that was presented, they were shown a modified version of the paper method designed for the computer. Students were assigned to work in pairs on one laptop and played against another pair sitting directly across from them, similar to the setup given in the physical prototypes. Afterwards, 2-3 researchers roved around between the groups, answering any questions and making sure the gameplay between the groups was going smoothly and solving any technical or user interface issues with the game. The researchers did not have to serve as moderators for the game, so they served more flexible roles. Also, a review of the gameplay was given in the second session. In the fourth session, we faded a button called the “Show me how” button, which gave the students the answer. We also began to encourage them to use paper and a pencil instead of the computer to come up with their answer. In the last session, we again modified the program so that they could not use the computer’s tools to generate the answer.

Measures The same 22-item test from the second study was used to measure fractions knowledge before the study started (pre-test), at the time the groups switched

64 conditions (mid-test), and at the end of the study (post-test). No feedback was given to any of the students about any of their three performances on the test.

Results

Figure 18: Graph detailing test scores in both the physical card game and the computer version.

Figure 18 displays the results of all three examinations for these groups, as well as the results from the previous prototype, Physical Prototype 2.0. (This previous study had used the same crossover design with stratified random assignment, so the two groups from Physical Prototype 2.0 are labeled Treatment Physical for students who played the card game initially and Delayed Physical for students who played the physical card game later.) At pretest, there were no significant differences between any of the groups. The Physical-Prototype-2.0 results shown on the graph above are results that have been previously reported (Jiménez, Arena, & Acholonu, 2011) and are only displayed to demonstrate that the physical and digital conditions were similar in their effectiveness. Therefore, we will only report numbers for the digital version.

Pre-test scores for the digital groups did not differ, t(27) = 0.68, n.s. By the mid-test, however, Treatment Digital (the group that played the card game) had shown significant gains, t(13) = 6.03, p < .0001, while Delayed Digital had not, t(13) = .33, n.s. Once the Delayed Digital group was allowed to play the card game, they showed

65 significant gains from mid-test to post-test, t(13) = 9.71, p < .0001, while the Treatment Digital Group, who had not played the game in over six weeks, maintained their mid-test achievement levels, t(13) = -1.6, n.s.

Discussion The figure above shows that the performance of children who played the digital game was very similar to the performance of children who played the physical game. In both groups, the mid-test comparisons show large differences between the groups who had played the game and those who had not. While in our initial prototype we were not able to demonstrate any effects in learning, by redesigning certain aspects of the game and revising our experiment we are able to show now that both the physical and digital versions of our game are capable of helping children with fractional operations on whole numbers. The similar outcomes also indicate that our adaptation from a lo-fi prototype to a computer game was a faithful one. Nevertheless, the two versions of the game have different affordances. Although we took steps to ensure that most of the digital game mechanics stayed close to those of the physical game, there are some rules/features in the physical game that because of time constraints were not incorporated into the computer game.

Table 2 shows the fun results from the digital prototype. What one notices is that while there is a little bit more variation in the scores, the children on average are still scoring relatively highly in terms of the game, with scores in line with the other physical prototypes. This has allowed us to maintain our belief that the game remains entertaining for the children to play as a computer game.

Physical Physical Digital Survey Item Prototype Prototype Prototype 1.0 2.0 I had fun playing this game 4.88(.39) 4.96 (.29) 4.71(.93) This game is boring 1.72 (1.28) 1.26 (.77) 1.54(1.12) I would want to play this game again 4.65 (.9) 4.78 (.66) 4.62(.95)

This game is too hard 2.12 (1.52) 1.59 (1.15) 1.43(1.02) Sometimes I got frustrated while I was 2.44 (1.61) 1.76 (1.12) 2.23(1.48) playing Table 2: Updated Fun Results with Digital Prototype findings in bold

Having produced a game that was both fun and educational for the students, it is important to examine what parts of the game were critical to its success. Finding the components or elements that made our game successful can inform future game design for us and for others. One element worthy of study is the use of story. As we saw the children playing the game, both in the classroom and in our user tests, we noticed the amount of discussion the players would have around the term “Stink bomb”. For example, it was typical for players to place a stink bomb on their opponents and then start giggling, uttering before or after “I stink bombed you”. This little phrase would imply that they were engaged in this element of the story, which was to play pranks on each other before they do the tug. According to previous research, it is unclear whether this context should have little or any effect, as some researchers may argue that the context could be entirely arbitrary (Habgood & Ainsworth, 2011). Therefore, we will more closely investigate the role that children place on story in the next section.

Section IV – Investigating children’s conceptions of stories While the discussion thus far has explored the various benefits of stories, one thing remains clear, the fields concerned with learning games could still benefit from knowing the aspects of story that are important to children. Identifying the critical components of story is important because it can help designers more accurately manipulate the conditions in which children interact with a story. A study was conducted about how children think about and relate to a story. What is known from previous research and what is gleaned from this new study can inform this research on games. I will use this deeper understanding, along with previous research, to hypothesize the elements that children find necessary for story in educational videogames.

In this exploratory study, children were asked to examine and explain the stories behind two educational games: the Tug-of-War game and another game called Number Eaters. Number Eaters was chosen due to its lack of story along with a few other factors, such as it being a game where the rules do not make a lot of sense to children. These criteria were derived when I conducted a theoretical analysis of how to apply the story grammars to educational games. Once the interviews were done I coded and analyzed the interviews and arrived at three important conclusions. First, I found that the children prioritized mentioning characters in their recollections of stories. Second, children would often generate information based not only on stories but on the mechanics of the educational games as well. Third, I noticed a couple of areas where the story used in Tug-of-War could be improved for the children. After reporting on the findings, I have a discussion of how to integrate existing theory with the study results.

Theoretical analysis The working hypothesis was that children who played games lacking story would not be able to remember the game as well as children who played games with a story. The hypothesis is based on story grammar literature which argues that stories have beneficial effects on comprehension and memory (Black & Bower, 1980; Bower,

1978; Kintsch & Van Dijk, 1978; Mandler & Johnson, 1977; Rumelhart, 1975; Thorndyke, 1977). My initial problem was to find a story grammar that could be applied to educational games.

The search started by examining story grammars listed by Black and Bower (1980). Specifically, I looked at the Black & Bower (1980), Mandler & Johnson (1977), and Thorndyke (1977) grammars. The challenge in finding a grammar that fits with educational games is that educational games often have very brief stories that accompany them; therefore, I needed to ensure that the grammar that I used from a theoretical standpoint was well suited for the brief stories often given in educational games. After looking at various frameworks, the Common Core Standards for English and the Annenberg Foundation’s explanation of story, I decided that the Thorndyke grammar was the most applicable because of its brevity and adaptability. I chose Thorndyke’s grammar because of how it deconstructed plot, as a series of episodes, rather than as a beginning, middle and end mentioned in other grammars (Mandler & Johnson, 1977). The fact that the plot is composed of various episodes fits nicely with the fact that the Tug-of-War game also has multiple “tugs” that happen throughout the game. Each tug and the result of each tug have an impact into the game.

The Thorndyke grammar breaks a story into four components, Setting, Theme, Plot and Resolution. The two images below (Figure 19) provide a representation for the Thorndyke grammar. The image on the top represents one possible theoretical instantiation of the Thorndyke Grammar, while the image on the bottom demonstrates how Tug-of-War would fit into such a grammar. After studying the two grammars, one can notice that the Tug-of-War grammar fits fairly well into the representation given. The only discrepancy is with Time, which is not explicitly mentioned or detailed in the Tug-of-War game. This discrepancy will be discussed below.

Figure 19: Thorndyke's Theory (top) and Tug-of-War’s instantiation (bottom)

Applying the Thorndyke grammar to Tug-of-War, one notices that in the setting, the characters would be the player and her opponent, the location would be the school-playground, and the time could be imagined as recess or some recreational time. Recess is represented as an orange cloud because it is implied in the story; there is no direct mention of it. Following the Setting, is the Theme, which Thorndyke explains as being, “the general focus to which the subsequent plot adheres” (Thorndyke, 1977, p. 80). In the case of Tug-of-War, the Theme would be to win tug matches so that one earns points. The first to earn 20 points will win the game. The focus of winning the game and scoring points adheres to the mechanic of having a lot of Tug-matches to score those points, which is what the plot adheres to.

The plot consists of these Tug-of-War matches where the players have the sub- goal of winning the match. With the sub-goal of winning the match, users play fractions and deal tricks in an attempt to have more people on their side than their opponent. After they finish playing trick cards, then the outcome of that tug match is calculated and followed by another tug match. While the figure above shows three matches, the number of matches is wholly dependent on whether or not they have reached the main goal of winning the game. Also while each match is relatively independent of each other, the way that the players play their cards may be influenced by previous tug matches.

Finding a contrasting educational game to Tug-of-War Having found a grammar and applying it to Tug of War, I concentrated on finding another educational game that children could play that could serve as a contrast and found several candidates2. In order to choose one game from the list of

2 I conducted this search by searching for games on the Internet using a search engine and the keywords “math games”, and using the repository from hoodamath.com based on my search results. A few games were chosen that contrasted nicely with the Tug-of- War game. These were: Math Blaster Hyperblast (www.mathblaster.com, June 18, 2012), Escape from Fraction Manor (mathplayground.com/HauntedFractions/HFGameLoader.html, June 18, 2012), Multiplication Grand Prix (academicskillbuilders.com/games/grand_prix/grand_prix.html, June 18, 2012),

71 candidates, I developed a set of three dimensions that would serve to discriminate the games from both Tug-of-War and each other. The three differences are called extrinsic/intrinsic integration, providing reasons for that math, and calculating the math to advance the story. These three dimensions are explained in detail below.

The first dimension – intrinsic and extrinsic integration – relies primarily on Habgood’s interpretation of intrinsic fantasies and how they are tied to the core mechanic of a game. In a game that is intrinsically integrated, the educational content is fed in the game via the game’s most exciting moments. I would argue that Tug-of- War follows this intrinsic integration because users must calculate fractions during the exciting moment of playing a card. Contrast this to integrating the content extrinsically, where the educational content and the game are largely separate. An example of an extrinsically integrated game from the candidates I chose would be Escape from Fraction Manor (Figure 20). In the game, players first navigate a platform world avoiding monsters and collecting clues (Figure 20: left). After they collect all the clues, the game shifts to a set of problems they must answer to continue playing (Figure 20: right). The educational content presented is separated from the exciting aspect of jumping on platforms through a world.

Figure 20: (left) the exciting platform phase; (right) the math equation phase, extrinsic to the game environment.

Fraction Poker (hoodamath.com/games/fractionpoker.php, June 29, 2012), and Number Eaters (hoodamath.com/games/numbereaters.php, June 29, 2012).

The second dimension relies on whether or not the games provide story-based reasons for encountering the math. In Tug-of-War, children are introduced to fractions because the stink bombs that they can use have different quantities. The 1/3 is presented as part of the stink bomb metaphor in which they play a trick on another group of kids. While the fraction could be replaced with another quantity, it follows the narrative argument and falls in line with the story. Contrast this to a game like Math Blaster Hyperblast, where while children are shooting enemies and navigating a sci-fi tunnel (Figure 21: left), and then suddenly a giant octopus appears that presents children with arithmetic problems to solve. While the math and octopus are being presented during the gameplay, there is not an immediately clear argument for why an octopus is not letting the player pass without solving the arithmetic problems (Figure 21: right). One may also question why an octopus appeared in the tunnel in the first place. Furthermore, the addition or math could easily be replaced with other mathematical content. For instance, the equation at the bottom of the screen could be swapped for a fraction, algebra, or geometry problem, without changing the mechanics in the game. Such a replacement could not happen in Tug-of-War without some rebranding of the story and what the numbers mean.

Figure 21: (left) player navigating Hyperblast, (right) octopus appears with math equations to solve

The third and last dimension relates to how the quantity being calculated in the story relates to the rest of the plot. In the instance of Tug-of-War, the answer to the fraction has an effect on the outcome of the story. For instance, a 1/3 stink bomb played on six people means that two people become stinky and subsequently no longer

73 participate in the tug of war. This mechanic is in direct contrast to other games where it is unclear that the outcome of the calculations is linked to the subsequent narrative. For example in Carpenter’s Cut, players play as a carpenter who must cut pieces of wood that have the correct fraction on them (Figure 22: left). This game does well on the first two dimensions. The game presents the math during the most exciting part of the game (Figure 22: center) and provides a reason for doing the math; home- improvement tasks like cutting have strong links to home-based math activities (Esmonde et al., 2013). Nevertheless, it is unclear from the game why the players should perform the calculations. Players who solve the fractions and perform cuts for the carpenter receive nothing for their calculations, other than moving on to the next level or cut (Figure 22: right).

Figure 22: (left) a story basis, with an intrinsically rewarding activity (center), but only used to level up (right)

Having come up with these dimensions, I created an analysis chart for the games searched to understand how they compared with Tug-of-War. The goal of the analysis was to ensure that these factors, explained above were sufficient to discriminate Tug-of-War from the other games listed. Having such factors would provide a good starting point for development of an interview protocol. It also helped make sure that the game that I chose as a contrast to Tug-of-War would not be the same on all dimensions. The table is displayed below (Table 3).

Math during Story provide Calculating number or after? reason for math? advance story? Escape from Fractions Manor After Weak Weak Multiplication Grand Prix During No Yes Fractions Poker During No Yes

Math Blaster Hyperblast During-After No Weak Tug-of-War During Yes Yes Carpenter's Cut During Yes No Number Eaters During No Weak Jiji Cycle During No Weak Table 3: My proposed dimensions applied to selected educational Games After I had looked at the factors, Number Eaters was chosen as a contrasting game. Number Eaters is a clone of a popular game Number Munchers, a classic edutainment game (Ito, 2007). The goal of Number Eaters is to control a Green Monster who has been tasked with eating all of the numbers that correspond to a specific equation given in the upper right corner (Figure 23). While the Green Monster moves around, it also has to avoid landing on the same square as the Purple Monster while finding the numbers. Once it finds all of the squares that have the correct answer to the equation, the player then moves on to the next level. The next level presents the player with a new grid of numbers and a new equation for the player to calculate.

Figure 23: The Number Eaters Game that was used as a contrast to Tug-of-War

If one looks at Number Eaters based on the dimensions, one notices that Number Eaters shares both similarities and differences to Tug-of-War. Number Eaters and Tug-of-War are alike in the first dimension – they both integrate the math into the exciting parts of the game. Nonetheless, Number Eaters differs from Tug-of-War in the other two dimensions. Number Eaters does not provide any story-based reason for doing the math problems that it presents and calculating the number does not advance the story aspect of Number Eaters. I labeled this third dimension as “weak” because

75 even though calculating the number does not advance the story, it does advance the game. Aside from the dimensions, Number Eaters also stood out because it could cover fractions content; it is seen as a classic educational game, and served as a nice contrast because it has a minimal story, yet still appealed to children. I believe that children would find it appealing to control a green monster that is in charge of eating numbers in a grid world. I also chose Number Eaters over the classic Number Munchers because of the former’s free availability on the Internet, making it easy to access via different computers. Having come up with factors that I believe are important to Tug-of-War’s success and chosen Number Eaters as a contrasting game, I now will discuss how these dimensions and theories influenced the questions I chose for the interview protocol.

Interview Protocol For the interview protocol, two types of interview questions were developed. The first set of questions were meant to “break the ice” for children to talk about story in general. Children were asked to discuss their favorite cartoon show. Then they were asked to provide in detail what happened in either their favorite episode or the last episode they watched. Follow-up questions were asked to ensure that the children were finished with their recount of the show. This first stage of the interview served to gauge children’s recounting and storytelling capabilities, to help make visible what matters to children, and to understand what children recount as part of their stories. Finally, this first stage also informed me of children’s ability to understand traditional media (television/film) that place an emphasis on story. While both Tug-of-War and Number Eaters have a semblance of a story, is it not their central component, so having a source of comparison to another medium where story is central was important.

The second part of the interview concerned the two games the children played. The children were asked approximately the same set of questions for both games. The questions were: “Tell us what the game was about” and “Did the game have a story? Children were then asked to elaborate on their responses. Finally, they were asked

76 questions that addressed the last two dimensions of the Tug-of-War game analysis, which were to discuss why they thought the math was present in the game and why performing the math equation was important. To understand how they conceptualized the calculations with respect to the game, children were also asked what happens after they calculate the answer and its significance. To help the children in answering these questions, the interviewers presented the children with relevant screenshots of the game. These screenshots were shown at the end of the interview in order to not influence what children told us initially about the game and the story. The interview study was piloted with two children, and a few questions were modified. Similarly, a few questions were introduced to the protocol, most notably to ask children specifically about the characters in the game and their responsibility or interaction with those characters. Once the protocol was finished, it was then printed for each interviewer to have while conducting the interview (see Appendix D – Protocol for Interview Study).

Interview study

Participants Sixteen students participated in the study – ten females and six males. Five of the students were incoming third graders, six were incoming fourth graders, and the remaining five were incoming fifth graders. All participants were part of an after- school club; the club’s stated mission is to place after-school sites in what they term “the most challenged areas” of nearby cities. The after-school club is located relatively close to the classroom used in previous studies. This was beneficial and also a challenge. Being close to the school gave me greater assurance that they were similar to the children I had in previous studies. However, it was a challenge because I had to make sure that none of the participants had seen Tug-of-War. They had not.

Method First, children played both educational games for approximately one hour each. The order in which participants played the games was counter-balanced, so that half of the students played Tug-of-War first, while the other group played Number Eaters.

Each student was introduced to the two games by one researcher. The researcher introduced and explained to each of the students how to play the game, but did not mention anything about the story. During the playtime, the researcher was present only to assist the child or answer any questions they may have, but largely to stay out of the student’s way while playing.

The children were then interviewed after having played the educational games. The length between the time that the children finished playing the game and were interviewed ranged from one to three days. After the students had played the games, they were then invited to meet with a researcher individually. Next, the researcher followed the interview protocol during the meeting. Fourteen out of the sixteen interviews were recorded using a video camera. In the other two interviews, a video camera was not used based on the privacy choices outlined by the parents. In those two interviews, the researcher took field notes of their responses.

Materials The children played both the Tug-of-War and Number Eaters games. A simplified version of Tug-of-War was used for this study. This version excluded the chance cards and kept the focus on children’s understandings of stink bomb cards, air freshener cards and the fractions written on those cards. While I initially wanted children to play solely the fractions version of the Number Eaters, children played any version of Number Eaters they wanted, which included children playing games where they worked with basic arithmetic. This decision was made because one of the researchers noticed immediately that students did not have the requisite knowledge to solve the fractions problems in Number Eaters. Based on a researcher’s observations, most children played a Number Eaters version that used single digit addition equations. Most of the children decided to play this Addition version of the Number Eaters game, which meant their goal in the game would be to look for spaces that had an addition problem equal to the target number provided to them in the game.

Interview Questions Each interview lasted approximately 20-30 minutes and involved a one-on-one session between the interviewer and interviewee. A PhD colleague and I conducted all of the interviews3. Before interviewing, time was spent with my colleague going over the interview protocol. The protocol followed a semi-structured interview, with a set of questions that I wanted to ask each student and sets of questions to use to prompt students to elaborate. Agreement and the questions were presented in an initial meeting with my colleague before the interviews, and progress in the interviews was discussed afterwards.

The process for the interviews were based on first asking the children initial questions about how they view stories in more conventional media that are known for having stories, such as retelling a favorite cartoon episode or movie they had watched recently. After students had described their favorite traditional show, students were then asked about the games they have played and were also asked to explain the rules and the story of each game. Next, the interviewer proceeded to ask students questions that were directly related to the aforementioned dimensions. Finally, the interviewer sought further information regarding how the children viewed and thought of the story in the different games.

3 Because parents of two of the children had privacy concerns about videotaping, those two interviews were not videotaped. In those two interviews, I performed the interview process and took extensive notes in real time on my laptop as students provided answers to the questions. While not extremely fast, I do feel that my typing is fast enough to get most speech patterns. I decided to do a quick typing test and noticed that my speed is roughly about 80 words per minute, according to an online typing test. With speech being on average of 150 words per minute for adults (http://www.ncvs.org/ncvs/tutorials/voiceprod/tutorial/quality.html, September, 10, 2012), it means that with children I was able to jot down the majority of what they are saying. In my memory of the transcriptions, I had a good feeling that I recorded most of what they said, if not the main ideas of what they discussed. Because I was leading the interviewing questions, I would also pause at times before moving on to the next question in order to make sure that I recorded their response as accurately as possible without disrupting the flow of the conversation too much.

Description of Analysis All of the playtime and interviews were done with the children over the course of two weeks, with most students taking one or two days to finish the study. All interviews were uninterrupted and took 20-30 minutes to complete. Some of the playtime had to be split for some of the students due to insufficient time at the program.

Shortly after finishing all interviews, tapes were watched, and an open-coding indexing scheme was employed. In open-coding, I wrote down surprises and general themes (such as creative interpretations) that I noticed from looking at their responses. Each videotape was then transcribed. These transcriptions were done at the conversational level and did not include non-lexical utterances, except long pauses. The transcripts were recorded into a web-based form that I created. In regards to the two non-taped interviews, I decided that my field notes looked close enough to the transcriptions that I could place the notes into the form as well3. The transcriptions enabled me to analyze and more quickly look up information from each interview.

After transcribing the interviews, the Thorndyke grammar was applied to each child’s conceptualization of each game. In order to apply their conceptualization to the game, first I had to make sure that I was comfortable with my own interpretation of the Thorndyke grammar for the two games. To ensure this, I decided to construct several practice grammars and compare them to Thorndyke’s grammars. I carefully re-read Thorndyke’s article and tested my knowledge of the grammar by comparing the grammar to one done by Thorndyke himself. While I had previously read the article, I formulated the grammar in earnest. Next, I compared the two grammars based off the same story and noted the differences. The main differences between the grammars were that the Thorndyke grammar had multiple resolutions and deeper levels in the plot; in contrast, I had certain plot events in succession.

Because the two grammar maps were similar, I decided to apply the Thorndyke grammar to other stories to gain a deeper understanding and explore its nuances. I applied the grammar to two popular children’s stories, Little-Red Riding Hood and

Wall-E. I chose these stories because they are famous stories that I felt children would know, as the former is a popular children’s tale in the US, and the second is a popular Disney Film released in the last five years. Applying the grammar to these stories helped me ensure that the relevant details of the story arose from the grammar interpretation. After applying the grammar and analyzing the results, I understood more of the limitations. More specifically this grammar was meant to be used in stories with a single protagonist or character. With a deeper appreciation for the grammar, I then applied the Thorndyke grammar to both Tug-of-War and Number Eaters. From those applications, I then developed a set of codes that I used to count and structure children’s interpretations of the two games. The result of that analysis and the interview caused me to generate three major findings, which are discussed below.

Results and Discussion

Characters One theme that came about early on in the interviews was the importance of the characters to the children. In relating to the first interview questions, fifteen out of the sixteen subjects were able to recount a show or movie that they had mentioned as being their favorite. After the children would mention their favorite show, the other researcher and I would ask them to describe the show to us, and thirteen out of the fifteen children started their description by naming one of the main characters, or a few main characters and how they were related to each other. For example, here is one such child’s explanation of their favorite show, called iCarly.

“There’s this girl and her friend and the girl Carly. She has a brother and the brother is bigger than her, and they videotape funny shows every time, and they show how they play on the other cartoon, the same video they had.” “Can you say more?” “Like there’s this cameraman but he’s a friend of Carly, and he videotapes them doing funny stuff and they put it on the website, and at school a lot of people like them so they’re famous at school, and they always hang out at other places and there’s always crazy people coming in.” (Incoming 3rd grader, Female)

In this retelling, one immediately notices that the 3rd grade female first mentions Carly and Carly’s brother, and then retells the story. Later on when the interview prompts the student for more information, the student then introduces an additional character (the cameraman) first before mentioning anything. Mentioning the characters first was a common occurrence in the interviews. For some of the students who would mention the same show, the same characters would often be the only consistent information. Here is the initial description from one boy for a Comedy Central show, named the Regular Show:

“It's about like a blue jay and a raccoon start working at a job, but then they start slacking off too much so their boss Benson pressures them to work harder” (Incoming 5th grader, Male)

Now contrast that description to an account for the same show made by another boy.

“Basically about this blue jay and squirrel and they like in the commercials. It’s anything but a regular show, like everything new happens in episodes” (Incoming 5th grader, Male)

While both have come up with different explanations for the question, “What is this show about?”, what one can infer from each description is that both children recognize that the show is about a blue jay and a furry mammal. Both children described the characters first, rather than describing the setting or other details about the show, which reflects the general pattern shown by the thirteen students who mentioned characters. The two students who did not mention characters initially, one started off by mentioning that they did not know how to speak English very well, and the other mentioned that the show was fun but hard to explain. Nonetheless, the fact that students mentioned characters first suggests their importance to the children.

The importance of the characters to a story was also demonstrated in the Number Eaters game. Because Number Eaters had two monsters that children could find appealing, I found children generated many facts to help them explain the game that were based on the characters. Although not all were immediately forthright with mentioning the monsters, all of the students did identify when prompted that there

82 were two characters on the screen, a “protagonist green” monster that they control, and an “evil purple” monster that they must stay away from. Students then generated scenarios that used the characters and the game’s rules to explain their encounters in the game. For example, one student mentioned that a character would become sick if they ate the wrong number, and a majority of the students would say that the purple monster would eat the green monster (nine out of 16) if they ever came into contact, although this is something that was never explicitly mentioned in the game.4

Students who did not mention the purple monster’s responsibilities would also mention other facets that were not part of the game. For example, two children mentioned that the purple monster tries to block the green monster from getting the numbers, and another two mentioned more accurately that the monster could get the green monster, rather than eat it. The remaining children gave erroneous statements, but even some of those were invented, such as one student who stated that the purple monster was there to protect the green one. Whether correct or inaccurate, the children’s statements about the role that the purple monster has in the game, provides evidence that the children are remembering the characters. I would propose that the children are then using the characters and their experience in the game, to generate reasons that align to their own stories about the game. I believe that students perceive the characters and game mechanics, and then use those two facets to infer actions for those characters that align to some story.

This finding about generating sentences that were not part of a story falls in line with previous literature done by Black & Bower about characters and their motives, where they showed that when characters had a motive, subjects would add more detail to the story depending on the character’s motive. The difference here is that rather than the children knowing the motives of the characters beforehand, the

4 In the game, when the purple monster reaches the same square as the green monster, a noise plays that sounds like a haunting noise, and it subtracts one life from the player, and moves the player back to the center of the grid, while causing the purple monster to disappear. In our interviews, not one child makes any claim or statement about the purple monster disappearing, nor do they provide any reasoning as to why the purple monster disappears after reaching the same square as the green monster.

83 children are filling in the goals of the monsters by the actions that they perceive the monsters to be taking and the game play. I would argue that children are using the actions of the game to determine motives for the characters in the game. This finding is important because it makes it salient to me that in game design, the characters are a central focus for children who play a game.

While having salient characters is an asset of Number Eaters, it is a weakness in Tug-of-War. In Tug-of-War, children were more likely to mention themselves as a character, or would mention simply people or kids. Tug-of-War’s characters could be noted as the teams that could be recruited to a side of the Tug-of-War. However, only three of the 16 students identified any of the specific characters that were on the cards, with only one naming most of the cards with people on them. An additional person made an erroneous reference to a card, calling the first graders as the kindergartners, which means 25% of the students mentioned at least one of the character cards. Even though the children would not name Tug-of-War’s characters by names, the children would often discuss the characters in the story using an aggregate term like players, children, students, or people. By having appealing characters, students who play the game could then use the character to organize their knowledge around the game more clearly.

Nevertheless, children do not always need characters to memorize the game’s rules. Characters could serve as an anchor point around which students could organize their information. However, students could also use the general game process to store most of the information they needed about the game. This is evident in Tug-of-War, as students who played the Tug-of-War game still had a surprising amount of knowledge about the game and the rules that the game provides. Below is an excerpt of one interview with one student, whom I will call Gloria. Gloria is an incoming 5th grader who self-professed early on that she didn’t “know how to talk a lot of English”. While she does mention the characters in the general form as “kids”, she does a commendable job explaining the game. Part of Gloria’s conversation about Tug-of- War is below.

“What’s this game about?” “It is about fractions, so it says like, Gloria what is ½, (pause) something like that.” “So what happens in the game?” “So they have like cards like here, and then up, we are down, they are up, and so we have to win the thing” “What thing?” (Pause) The war thing?...Oh yeah Tug-of-War, you have to win the Tug-of-War. “So you have your cards and they have their cards, and then what happens?” “So the card says one (pause) third, so it asks Gloria, what is 1/3 so then you divide and then you do the show me how, and then like this third, and then you do three thirds, 1, 2 three, and then you circle one, the answer will be one.” “Oh, Ok and then what happens after that?” “And then there are like kids in here and then the kids get eliminated because we already got them out.” “Oh how did you get them out?” “Because we put one of the smells that smells bad” “How do you know how many that smells bad?” “Because I put 1/3 as the answer if I get two or one that’s how many people are getting out.” “Why does it matter if two of them get out?” “Because we have more people and they have less people.” “And why does that matter?” “Because we win the points.” “How come you get the points?” “Because we won the Tug-of-War”

This transcript demonstrates that, with a little bit of prompting, Gloria was able to provide the entire line of reasoning for how playing cards and fractions affected the tug of war. She went from discussing the fractions questions that she had to answer, provided a strategy for answering those questions, and providing reasoning for how the fractions apply to the story elements of the game. Gloria told us that based on a fraction associated with smelling bad, kids get eliminated from being able to participate in a Tug-of-War. Those cards and getting people out provide a means for winning the Tug-of-War and consequently earning points. Other students provided similar stories about playing stink bombs and air fresheners, as well as providing strategies and identifying when certain moves were legal or illegal/impossible. It

85 could be that the stink bombs and air fresheners were in a way the characters for the children that they anchored their responses around. 11 out of the 16 children mentioned stink bombs and air fresheners when they were asked “What is Tug-of-War about?” in some capacity in their first sentence. Only two of the sixteen children failed to mention the air freshener or stink bomb directly, and Gloria is one of the two. While Gloria’s account does not include any mention of the word, one does notice that she does have some conceptualization of the stink bomb in the game and its consequences. The representations that the students gave and their anchoring around the stink bombs and air fresheners demonstrated that Tug-of-War had a sufficient enough story for children to remember and use to explain causal relationships. I hoped then that, with Number Eaters, students would not have that same facility. At the beginning of the experiment, I had hypothesized that students would not remember the rules for Number Eaters as well as they had for Tug-of-War. I will discuss whether or not the results confirm my hypothesis next.

The game mechanics can drive the story As previously mentioned, almost all students who explained Tug-of-War stated that a stink bomb would cause a certain number of students to be stinky; this action would in turn affect the Tug-of-War. Because students would state this, I would argue that Tug-of-War gives them at least a partial reason for doing fractions. The stink bomb card contains a fractional amount; performing that fractional amount (by placing the stink bomb on a group) will determine how many of those people will not be able to participate in the Tug-of-War match. Before doing this study, my hypothesis was that the students who played Number Eaters would not have a good recollection of the game’s rules because they did not have a good story to use as a basis on which to organize the game’s information. My analysis of their interviews demonstrates that they do use the story as reasoning for the game’s rules. However, contrary to my hypothesis, children still had a good understanding of the game and its rules. For Number Eaters, it seems that students use the game’s traditional mechanics and their experience with the game to conceptualize a story. Contrary to enjoying the game and

86 story together – like in Tug-of-War – Students are using their experience in Number Eaters to come up with a story for the game.

Number Eaters’ story can be illustrated as having many limitations and missing parts. Refer to conceptualization of the Number Eaters map below (Figure 24). This conceptualization is based on the Thorndyke grammar made previously (Figure 19); areas marked as orange clouds are areas where there was little information provided. For instance, the setting’s location for Number Eaters is very abstract, to the level that it could be considered non-existent, which is why I labeled it in the framework as “Grid”. There is also no mention of the time at which this game is occurring, and there are no episodes or events that lead up to the theme for the game. The areas that are marked using brown circles are elements of the game that do not contain any story elements in them, but are rather game elements that I believe make more sense when applying the Thorndyke framework to Number Eaters.

Figure 24: Number Eaters game applied to the Thorndyke Framework

For instance, the theme of the game, according to Thorndyke, would be to win the game, but there is no reason for winning the game, other than a person’s innate desire to win at a game. You win the Number Eaters game by clearing the correct spaces, solving fractions or math equations along the way, and once you clear the board you move to the next level. These lines of reasoning mirrored what I found in the interviews. While only three children mentioned the overall goal of winning the game, the majority did mention that they were out to eat numbers (13), and that the goal of eating a number and calculating the number was to clear the numbers from the board (13). Moreover, half of the children mentioned the reason for clearing the board was to move on to the next level: a mechanic found in many classic games. This last statement makes it clear that, for many of the children, the goals of the game provided

88 enough of a context for doing the activities. When no story was present, children would align themselves to some of these classic game mechanics and would, when appropriate, apply stories to the mechanic. This argument of the children using the game mechanic as a framework and adding story when appropriate could serve to explain the aforementioned finding that the majority of children would make up answers about how the purple and green monsters would interact in Number Eaters.

For Tug-of-War, the children’s answers were more story-based. Fourteen of the children mentioned having the most people on their side of a tug of war was important, and to do that, you would play a trick card (mentioned by 15 children) and figure out a fraction based on that trick card (mentioned by 13 children). Twelve of the children mentioned that the goal was to win the tug of war match. Fifteen of the sixteen children explicitly mentioned stink bombs causing Tug-of-War participants to be stinky, and when they were questioned about why it was important for the children to be stinky, nine mentioned that when they became stinky they could no longer participate in the tug of war. Thus in this game, the children relied more heavily on the story elements of the game. Rather than saying it was to move to the next level or to get more points, most children provided reasons that aligned with the story. The interviews reveal that when children do not have these stories, they base their reasoning on the gameplay. Children seem to leverage and remember characters they find appealing. Children also seem to use their game playing experience to fill in gaps in their understanding of a character’s action.

What remains to be seen (and the important question from an education standpoint) is whether the work to organize the story line interferes with the time children might be spending figuring out the fractional content. Does a story help children organize the rules of a game and subsequently, allow them to focus more time on the educational content? In order to perform such a study, I need to make sure that there is an example of a game that has a contiguous and well-formed story that the children understand. Since Tug-of-War is a candidate to be such a game, I need to make sure that its story is contiguous enough that children playing Tug-of-War

89 understand its story and gameplay. The third set of findings investigates areas where children did not use the story to supplement their reasoning; areas where the story in Tug-of-War could be improved to justify the gameplay.

Tug-of-War’s story needed some work In reading the responses of the children, I found three major areas where students struggled relating the gameplay with the story. The first area is around connecting the fraction problem to the resulting match. Most of the children understood that you wanted to win a tug of war match by having the most people on your side, you play trick cards and figure out fractions, and playing a stink bomb takes away people. However, less of them were able to state a key causal relation between the fraction and the match. Only five out of the sixteen students stated that the fraction on the stink bomb card relates directly to the number of children that leave the Tug-of-War. While the children understood that the stink bomb takes away children from the tug of war, and that they had to answer a fraction afterwards, not all of the children linked together that the answer in the fraction is the number of children that do not participate. To me, this means that more of an effort needs to be concentrated in linking some of the plot specifics (figuring out the fraction) with the overall story. Currently in Tug-of-War, after one answers the fraction, the application immediately jumps back to the game with only a subtle indication of what happened. If there were a better elaboration of what happens to that number, then it may help more children link the story to the math calculation.

Another trouble for the children was that many did not have an understanding of the formula for scoring points in the game. In Tug-of-War, the scoring is based on the disparity of people, which means that the number of points scored after a Tug-of- War match is the difference in people you have vis-à-vis your opponent. For example, if I had 14 people and my opponent had 5 people, then I would score 9 points for that match. When asked about the scoring, students would at times mention erroneous point schemes “there’s just one left, and that means they get two points and we got zero still” (Incoming 3rd Grader, Female, 0005). More interestingly, is that multiple

90 students mentioned that points equaled the people standing: “sometimes they give you two [points] for putting two [people] there” (Incoming 3rd Grader, Female), which was also mentioned by another student:

“I have eight and they have…eleven…so they have 11 points, cuz they have 11 persons” “How many would you get?” “I would get 8” (Incoming 5th grader, Female)

A stronger connection needs to be implemented that helps the children notice the difference between the two teams corresponds to the number of points given.

Finally, the last area concerns the overall theme of winning 20 points. The 20- point limit is an arbitrary amount and this whimsical rule is not salient to most students. Only six out of the sixteen children in the study mentioned that the goal of the game was to get 20 points. Obtaining 20 points is a random game mechanic that could be refined to align closer to the story.

In truth, the previous theory that I proposed about students relying on game mechanics when story did not make sense is also evident in Tug-of-War, based on another interview where a child explains his version of the Tug-of-War story:

“The story is that you have [to] pass all the levels from pulling the rope, and there’s the math you have to do the math, so the story is you have to do everything so you have to get 20 points so you can be passing the levels.” (Incoming 4th Grader, Male)

The last part of their sentence is interesting because this student ended up incorporating a common game mechanic, the mechanic of a game with many levels, and their desire to pass the level by gaining a certain amount of points. From their speech, one could infer that the reason students believed they needed to get the 20 points in the tug of war matches was to move on to the next level. Therefore, they used a common game mechanic of moving on to the next level in order to justify why they were trying to achieve a certain amount of points. Tug-of-War has no reason for obtaining the goal of having 20 points, nor does it have a good system of tying the matches together.

These two flaws about the points and matches also were raised a couple of students in their interviews. As part of the interviews all students were asked whether or not they thought Tug-of-War had a story. When students were asked this question, seven of the children stated that they thought Tug-of-War did not have a story. After these seven interviewees had mentioned the lack of story, they were then probed to provide reasons for why there was no story and if they could create a story for the game5. Out of those seven children, one student mentioned that the game did not have a story because they were unsure why the tug of war was happening (Incoming 4th grader, Male), while an additional student developed a story about two schools wanting to see which one was the most athletic. Based on these two answers, one can notice that they were also unsatisfied with the game mechanic of winning 20 points and wanted to know why they were playing to 20 points in the first place, and why there were so many matches. One student identified this as a problem and the other provided a suggestion that would fix the problem. The second student also gave a possible solution to the problem of how each tug of war match could relate to each other more seamlessly.

Based on the issues raised, one would hypothesize that children understand some of the basic mechanics in the game, but may not understand how those mechanics tie together. My results demonstrate that many children lack the information in Tug-of-War that ties the matches and points together, as well as providing more fluidity in how to get points and how to figure out the fraction’s effects for the tug of war match. Looking back at the Thorndyke theory and comparing it to the results, I notice that I do not have a way of understanding causality in the Thorndyke framework that I know how to work with. The Thorndyke framework has served me well in helping me understand the basic parts of the story,

5 When children were asked what Tug-of-War needed to have a story, two of the students did not give any specific details about what was wrong with the story, while another mentioned needing words and people, and another mentioned needing an explanation for how to play the game as necessary for a story. The last student provided a story about a little girl being in an ice cream truck and being thrown a stink bomb which made her stinky.

92 but I need to think of a better way of representing the causal relationships that are inherent in stories for educational games. Therefore, I went back in search of a framework and thought more about other theoretical frameworks for stories that could be used for educational games, and I along with the help of my advisors, decided to switch to using a different theoretical framework, that of Kintsch and van Dijk.

Kintsch and Van Dijk theory applied to educational games I decided to shift to using the Kintsch & van Dijk framework because I felt it aligned more closely with this study’s findings and my proposed dimensions. Kintsch & van Dijk characterize text in the form of two levels, which they call microstructures and macrostructures (p. 365, Kintsch & van Dijk, 1979). The authors define a microstructure as: “a structure in the individual propositions and their relations,” (p. 365) while a macrostructure is: “characterizing the discourse as a whole.” (p. 365) My interpretation of their theory is that a microstructure is a sentence or the link to an individual sentence. A macrostructure is all of the sentences put together. If I were to apply this interpretation to Tug-of-War, a microstructure could be the single point that you need more people to win the tug of war, while the macrostructure would be the entire narrative behind the tug of war, the overarching purpose that organizes all of the propositions.

Many of the points that Kintsch & van Dijk make align with the results and findings of this study. For example, using their own architecture the authors argue that a discourse (story), “…is coherent only if its respective sentences and propositions are connected, and if these propositions are organized globally at the macrostructure level.” (1978, p. 365) Based on the results, I have discussed how children do not always have an organizing structure for the Tug-of-War game, which could affect what they remember about the story. In addition to the coherence argument, Kintsch and van Dijk also argue that readers make inferences and generate knowledge based on summaries of information, just like Black and Bower did. Black and Bower argued that subjects can use existing facts to infer other parts of the story in order to make it coherent (1980). These arguments also match some of this study’s findings: namely

93 that children used the facts that they were given and game mechanics (like passing levels) to infer aspects of the story (i.e. you clear the board to get to the next level, or you get 20 points to get to the next level). This study also demonstrates evidence of children inferring unstated rules about the game (i.e. the purple monster would eat the green monster), with some children even mentioning radical interpretations of the rules6.

The Kintsch & van Dijk framework is different from the Thorndyke framework because it carries less specificity than the Thorndyke one, and yet aligns more closely with the dimensions that I found with educational video games. If I were to try to apply the Thorndyke framework to the Kintsch and van Dijk framework, it would make the most sense to link Thorndyke’s Theme concept with Kintsch and van Dijk’s macrostructure, while Thondyke’s Plot (and each of the propositions or squares in the plot) could be linked to microstructure. While I lose some of the specificity that the Thorndyke framework provided categorizing the game’s story elements, I gain a higher level of understanding and distinction. It is my thought that while Thorndyke discusses the theme as the point on which the whole plot abides by, Kintsch & van Dijk’s macrostructure introduces the concept of a meaningful whole. For example, in the case Tug-of-War, the theme of getting 20 points would make sense from Thorndyke’s perspective because the rest of the plot is about how you score 20 points. However, getting to the 20 points in and of itself does not establish it as a “meaningful whole”, demonstrating that the macrostructure is missing. I can arrive at this conclusion because of the lack of children who remember that 20 points was the goal of the game, or used it as a basis for saying that it did not make sense from the story perspective. Because there is no meaningful whole for them to organize the tug

6 In the tug of war game there exists an animation that details the tug of war match, with both sides pulling on a rope to determine who wins. At the end of the animation, the game demonstrates how many and which side received points by having that number of people remain standing after the tug of war is done. Everyone else that participated in the tug is placed horizontally on the screen, near their position for pulling the rope. When a student was explaining the tug of war game and they explained this animation, they said that some of the children “die”.

94 matches around, which they remember for the most part, they do not have a sense for why the tug matches are occurring.

This lack of macrostructure is not unique to Tug-of-War. In Number Eaters passing all the levels to win the game would also not be a meaningful whole, as only two of the children mentioned passing all the levels, and 3 mentioned winning the game as part of the goal. Children understood parts of the microstructure of eating a number, and trying to stay alive, but there was no way for them to organize this facet either. Having demonstrated gaps in both Tug-of-War and Number Eaters using the microstructure/macrostructure framework, I feel that it is a more applicable framework to educational games. This structure also works well with the three dimensions for game selection that I previously outlined. Having a coherent set of microstructures and having them organized by the macrostructure link well with providing reasons for doing the math (our 2nd dimension) and for explaining how the math will play out with the story (our 3rd dimension). Therefore, I moved to using this framework as a theoretical backing to the dimensions that I laid out, since it helps to explain why parts of my dimensions make sense, and also aligns with this study’s findings.

Based on this interview study, elements that can help in the creation of educational based on story have been identified. The data suggests that children place an importance on characters, have a lack of understanding surrounding the whole structure of the game, and use the rules and game mechanics when no story is present to make inferences about the story and game. Suggestions for improving Tug-of-War have also been identified: (1) Introduce stronger characters; (2) Provide a better macrostructure for organizing all of the tug of war matches; and, (3) Present more clearly the microstructures of a stink bomb removing a fraction of players and winning points based on a disparity in the tug of war match to help improve the story aspects of the game. The findings also lead to a hypothesis that a game that is missing some key elements, or has game mechanics that children cannot infer the rules from, would produce less successful learning outcomes.

With a new framework, one of the goals in the next study was to investigate whether or not the framework and findings carry any significance. Hence it was important to test out the new framework and the findings by testing new variations of the game and its story. Making tweaks based on previous literature and findings should allow students to rely more heavily on the game’s story to understand the rules. It would also mean that students would be less likely to generate different outcomes than those that they have currently built.

Section V – The Experimental Study

From interviews to experiments: investigating the impact that story and particularly characters have on a game. In the previous interview study of elementary school-aged children, three main findings about stories and video games were reported. These were that the characters are of principal importance, the game mechanics seem to drive the story when none is present, and Tug-of-War needed refinements to enhance its story and connectedness. Based on this last finding, I made changes to the game before conducting a final study. Fine tuning was needed to make the game and story more tightly coupled at both the micro- and macrostructure levels, providing a better “cause and effect sequence” for the actions children would take in the game.

One way to tightly couple structures was by introducing more animations between the card-playing and fractions-answering screens. These new animations would help the user better understand how playing a card like a stink bomb would lead to them answering a fractions question. In the previous version, as soon as the user would play a card, it would immediately jump to the calculation screen. After the changes, when a student plays a card, the relevant manipulatives first run off the game screen before launching into the calculation screen. (See the area around the letter B in Figure 25).

Figure 25: Manipulatives moving off-screen before being shown the question

In addition to the aforementioned animation, another animation was introduced to transition between the calculation screen and the card screen. This new animation, which I call the post-calculation animation, detailed what happened between students answering the question and how it affected the game. Let me take Figure 25, which illustrates the action of playing as an example. In the upper-left corner of Figure 25 (represented by the letter A), one notices that the user is placing a 1/2 stink bomb on the Johnson Family. The result of placing this 1/2 stink bomb on the Johnson Family means that 1/2 of the eight manipulatives (four) would now be stinky. To illustrate that fact, the post-calculation animation was introduced to transition users back from their calculation (see Figure 26). That animation started by having the stink bomb graphic swirl around the screen (Figure 26: left). The animation then transitioned to showing four manipulatives transform into stinky characters. Once the characters were stinky, the stinky characters would then run around the screen fidgeting rapidly, before running back into their place on the game panel screen (Figure 26: right). In previous versions, the manipulatives would simply appear stinky on the game panel screen. Because students in the interviews had not understood that link, this animation made what was happening in the game more apparent. This was further enhanced by

98 having students select the manipulatives that were going to become stinky once they correctly figured out the appropriate number.

Figure 26: Animations detailing that 1/2 of 8 (left) are now stinky (right) This type of animation was also created for the air freshener and ice cream truck cards. In the ice cream truck case, the original mechanics of the game were slowed down by playing an animation that demonstrates to the users what effect the ice cream truck card had in the game. Finally, to address the problem that students had with connecting the end of the round with the tug of war, an animation was added that has the manipulatives move into position before starting the tug of war. This end of round animation serves as a link or transition between the game mechanics and the other animations that children were seeing.

With these changes in place, the two other findings from the interview study were considered – the importance that children seem to place with characters and the fact that game mechanics seemed to drive the children’s understandings for the game. While students seem to place an importance on characters, students also relied on game mechanics. Since both of these aspects seem to be important to children, the question of whether there was any value in having both was important and merited research to answer.

This dissertation began with a discussion of the perils of educational video game creation. One of the big hurdles in this creation is simply the resources needed to create a game. If researchers can generate better knowledge as to where to devote resources, that could help in creating more successful games in the future. The

99 interview study found two separate elements that seem to be important for children in educational video games. The duality of the importance of characters and game mechanics in games warrants the question of whether both are necessary. It was important to capitalize on the differences between these two findings, which led to a last experiment that aimed to investigate the role that story has in educational video games by having students play three different versions of the same game – a story less, abstract version, a story version without characters, and a story version with characters.

In order to inquire about story, the original Tug-of-War game was modified into two additional versions. One version, the Abstract version, is completely devoid of any story structure. The other version, the Characters version, places more prominence on the characters in the game. The differences between these three versions are discussed below. If one takes the literature on story into consideration, then one may believe that the story conditions would be able to capitalize both on helping the user learn as well as engaging more with the material. However, the interview data suggested a potential different outcome. It could be that traditional game mechanics are powerful enough for students to drive the rest of the learning and memorization in the game.

There are researchers who propose that the abstract is better, citing articles that argue against using idealized images and for using lines and shapes for student work (Goldstone & Wilensky, 2008), in learning applications such as agent-based modeling (Wilensky, 2002). There are also arguments against using a version of the game where the characters are more salient. The argument against characters may be that the characters become so salient that they distract the players from the content itself (Son & Goldstone, 2009). These complex nuances play out in games and need to be further discussed and researched. The goal of the experiment described below was to understand and detail the roles that story plays in games and how they impact and interplay with games. In order to explore this impact, a protocol was employed that assigned children to play one of the three different versions of the game.

100

The Three Versions of the game In the rest of this chapter I refer to versions of the game with the following labels:

 Abstract version – strips out all notions of story, leaving just the game with abstract general terms  Characters version – places more prominence on the characters by having the manipulatives resemble the people  Original version – mirrors closest to what students in earlier studies played.

In the Characters version of the game, I wanted to make children’s inclination of focusing on characters more prominent for them. That is, rather than having generic characters as the manipulatives, the students now got to work with the characters that are presented in the graphics of the cards, so that when they have to answer a question, they use the characters that are present in the cards as manipulatives. These characters also are present when the children have to answer questions. Figures Figure 27 and Figure 28 detail the differences present in the different conditions on the two most critical screens, which are the game screen and the calculation screen.

101

Characters Original

Abstract

Figure 27: The three conditions on the card playing screen: the Characters condition (top-left), the Original condition (top-right) and the Abstract game condition (bottom).

This Characters version is displayed in the top-left of both

Figure 27 and

Figure 28. The only crucial difference is that the manipulatives in this condition were made so that they would look exactly like characters that they represented in the cards. For example, rather than seeing eight general figurines which are used to represent everyone, each figurine shown in the game would correspond to one of the members of the Johnson Family shown on the card. These figurines would also be shown during the calculation and would also be shown as stinky or fresh. Now that each character is salient, when players select characters to become “stinky”, those specific characters would also be stinky on the other screen. The other conditions also give the students the opportunity to select which manipulative that they want to

102 become stinky; however, one would think that choice would become more trivial since all of the manipulatives look alike.

Characters Original

Abstract

Figure 28: The three conditions after having provided help on answering ¾ of four. The Characters condition (top-right), the Original condition (top-left), and the Abstract game condition (bottom)

In the Abstract condition, which is shown at the bottom of

Figure 27 and

Figure 28, one notices that the game has been completely stripped of any mention of the tug of war. Because the interview study indicated that game mechanics were sufficient enough for some students to use to encode the game’s rules, all of the Tug-of-War story images were replaced with abstract images. When replacing the images and text, careful consideration was given so that the software would not be written any differently in the Abstract condition. Only the images and text displayed to the user were changed. The people changed to dots, both in images and language; cards like the stink bomb were changed to be called “Reduce” cards. All of the new

103 animations were still kept in the game, except different images were used to explain what happened. For example, in the stink bomb animation, where 1/2 of eight became stinky, the Abstract condition text would state that 1/2 of eight are now reduced, and the four dots selected would transform to a different color and then move around the screen. In the case of the tug of war match, the Abstract condition still showed the end match, but instead of showing a tug of war match where students fell as they pulled the rope, the Abstract condition displayed dots that incrementally disappeared. All of the story elements were replaced by abstract representations so as to make the game playable without any insinuation of story.

The Larger Experimental Study Once all of the revisions and versions of the game were made, the study took place. Students were first exposed to one of three versions of the game. Afterwards, they were compared on how much they had learned and how much fun they had. The method and results of that study are presented below.

Participants: 80 fourth grade students signed up to participate. Three students switched schools during the study, leaving 77. All of these students attended one elementary school in the San Francisco Bay Area. The elementary school was in the same school district as sites for the earlier studies. There were no students who had been participants in the previous studies. They also made no mention to researchers that they had seen or played the game previously, which they would have only heard via word of mouth. Over 90% of the students in the study were classified as English language learners. These students belonged to three 4th grade classrooms in the same school, although one of the classrooms was a mixed fourth- and fifth-grade classroom. On a previous standardized benchmark test that the students took, only 27% of students tested well enough to be classified as having mastered the mathematics content taught so far at their school, with 20% struggling to learn the material. The remaining 53% were assessed as making progress towards mastery.

104

Method Rather than assigning each classroom to a game condition, all 77 students were randomly assigned to a condition. This random assignment was stratified by previous performance on a standardized test taken earlier in the year, and controlled for their performance so that no condition had a significant difference in standardized test scores. To accommodate wishes expressed by parents about their children not being videotaped, some of the students were switched out of conditions with other students who had equivalent standardized test scores.

Once students were assigned to a condition, they were then ranked by their previous performance and randomly assigned by rank to a playing group, which I call a pod. This was done to ensure that all of the pods were of mixed academic ability. Once in a pod, students were then placed in mixed-ability pairs since mixed-ability pairs have been shown to help with student learning in math board games (Guberman & Saxe, 2000). The only indicator that was used for these mixed-ability groups was a standardized test score that the students had taken earlier in the year. Each condition played in one of the three classrooms, which meant that some students would play in the classroom where they had instruction. To avoid having any effects based on being in the same rooms or having the same partners, students would also switch classrooms every two weeks over the five sessions. This ensured that all students played in their classrooms at least once. When they would switch classrooms, students were also given new partners and new play groups, in another random assignment, based on their standardized test scores. Subjects switched partners and pairs to limit any particular subjects from having a bad (or good) partner throughout the study. In instances where the random assignment left a student with the same partner they had earlier, partners were then manually reassigned.

Before starting the study, I asked teachers to administer a pre-test assessment in their classrooms. Then, with six colleagues to assist in research, (two per classroom), sessions were held for one hour a week for five weeks. The students would start each session in their original classrooms. Then students were led to their

105 new classroom based on condition. Once in the new classroom, students were called in by their pods and told whom their partner was. Students then sat in their seats, and began playing. Due to technical issues, and the logistics related to shuffling all of the students, the amount of time played by the students would vary from 35-45 minutes during each session. The students would play in the standard two versus two format that was used in the earlier studies.

After each session, all of the classroom researchers met and discussed problems that they noticed children were experiencing with the software. I took their list of problems and introduced simple changes each week to the software to help assuage any concerns or problems that had arisen. Some changes made mainly improved the language in the game, such as shortening messages given to students. Other features made the administration of the game easier, such as preventing students from starting games with other groups. Each week students would then play with the latest versions of the software.

Students played with random cards during the first four sessions, which meant they played the game as intended. In the fifth session, rather than giving everyone a random set of cards, each pod was given a fixed deck of cards arranged in the same order on that last day. While the students thought the cards were random, they were manipulated to be the same for everyone. This was done in order to gather enough data to help answer a few questions about students’ gameplay. A few of the students noticed and told researchers that they had received the same set of cards as before when their games had to be restarted. However, when the researchers were asked if they felt the students knew the cards were not random, and rather the same for everyone, they answered “No”. In addition, on the fifth session researchers placed video cameras on two pods, one from the Characters condition and one from the Abstract condition and recorded their gameplay. Two cameras were used to record each pod from two different angles. Video from each pod is approximately 30-45 minutes in length.

106

One week after having finished the five sessions of playing the game, the students were given a post-test by their classroom teachers in their original classrooms. This post-test looked exactly like the pre-test, except it added a few items at the end which were meant to assess the amount of fun that children had with playing the game, and their experiences with the game.

Materials Both the new pre- and new post-tests were an expanded version of the tests used in earlier studies. This new test expanded on questions that relied more heavily on their understanding of part-whole relationships (such as “what is ½ of 8” and less on their understanding of chance, which is known to be a very difficult subject to learn (Garfield & Ahlgren, 1988). The full test, which was given both at pre and post without any feedback, can be found in the Appendix as Appendix B – 2012 Fractions Assessment. The questions that were devised in this test were also broken down so that they belonged in one of five categories. These categories were questions where students had to compare fractions to one another, questions involving decimals, complex word problems, and fractions questions. The fractions questions were further broken down into two categories: fractions they had most likely seen in the game and fractions they could not have encountered in the game. The maximum score that could be obtained on the test was 41. Most questions were given one point; however, large problems were divided into sub-problems, and each of those sub-problems was given a point.

As mentioned previously, the post-test added a modified version of the fun survey as well as questions about the game to the end of the measurement. The fun survey was modified during the study for two reasons. During work with the children, I noticed that there were a few students who were more Spanish-dominant in their language abilities than English dominant. Because of this, I decided to make the survey, as well as the additional post-test items, bilingual. In doing so, I decided to model the bilingual survey based on the California Healthy Kids Survey (California Department of Education (Safe and Healthy Kids Program Office) & WestEd (Health

107 and Human Development Department), 2012). This meant that while I kept the same five-point scale, I modified the survey to include more of the language items as they were stated in the survey. I also added additional questions about the story and playing the game. These additional questions and survey items have been added to the Appendix as Appendix C – 2012 Additional Post-Test Items. Due to student absenteeism, 73 out of 77 students completed both the pre and post-tests.

Results

Reporting data from paper and pencil measures Table 4 below shows the average gain in scores for each condition, with the standard deviation of those scores in parentheses. Average gain is reported as the difference in student scores from pre-test to post-test (e.g. post-test score – pre-test score). When collapsed across all conditions, students had an average gain score of 5.5, which indicates about a 13% improvement. Moreover, the 5.5 gain score from pre-test to post-test was significant when compared to zero, t(72) = 7.86, p < .001, d = 1.85, with a large effect size. Regardless of condition, student’s scores on average increased from pre-test to post-test.

Abstract Group Original Group Characters Group Gain Score 4.00 (5.35) 5.59 (6.74) 6.89 (5.87) Table 4: Average Gain Scores from Pre to Post by Condition, (Standard Deviation in Parenthesis)

A cursory examination of the gain scores in Table 4 shows a small increase in the directions that I expected. The Characters group had a higher average score than the Original group, who in turn had a higher average score than the Abstract condition. A one-way ANOVA with planned comparisons between the Abstract and Characters conditions and Abstract and Original conditions was performed to investigate whether or not those differences were significant by condition. The results from that ANOVA show that the small increase by condition is insignificant F(2, 70) = 1.479, p = .24. The planned comparison between the Original and Abstract conditions also was not significant t(45) = .9, p = .37. However, there was a marginal difference between the Character and Abstract conditions t(50) = 1.72, p < .09. The marginal difference

108 between the Abstract and Characters conditions led to analyzing test scores by type of test questions, which were devised a priori. To reiterate, the five types of questions on the assessment were: questions that involved comparisons, questions involving decimals, questions involving complex word problems, questions involving fractions students would have seen in the game, and questions involving fractions students would not have seen in the game. I refer to each of these types of questions as Comparisons, Decimals, Word Problem, Old Fractions and New Fractions respectively.

All questions were categorized into one of these five groupings, and a subscore was calculated for each of the five categories for each student. The averages of these subscores along with the standard deviations for each condition are reported in Table 5. The table’s first three subscores (Comparison, Decimal and World Problem questions) show almost no gains and little difference by condition. Nevertheless, there seem to be trends in the Old Fraction and New Fraction questions in line with trends seen on the overall post-test. A one-way MANOVA then was run to determine if these trends were significant by condition.

Abstract Group Original Group Characters Group Comparison gain 0.83 (1.66) 0.41 (1.71) 0.41 (1.80) Decimal gain 0.38 (0.77) 0.68 (1.13) 0.56 (1.12) Word Problem gain 0.63 (1.01) 0.68 (1.78) 0.78 (1.12) Old Fraction gain 1.58 (3.20) 2.32 (3.56) 2.70 (3.29) New Fraction gain 0.58 (2.10) 1.50 (2.74) 2.44 (2.39) Table 5: Sub-test gain scores by Condition (Standard Deviation in Parenthesis)

The MANOVA determined that the overall fit of group to these gain scores was not significant F(2, 70)=.90, p=.54.

The next step was to look at the univariate scores for Old Fractions and New Fractions. The univariate test demonstrates that Old Fraction gain subscore by condition was not significant F(2, 70) = .73, p = .49. Nevertheless, the New Fraction gain subscore does have a significant difference by condition F(2, 70) = 3.78, p < .03.

109

Performing a planned comparison on this gain demonstrates that while the difference between the Original and Abstract conditions was not significant, t(45) = -1.29, p = .20, there was a significant difference between the Abstract and Characters condition with a medium effect size t(50) = 2.75, p < .01, d = .78. The Characters condition had a higher average gain on the new fractions when compared to the Abstract condition.

Investigating Children’s levels of Fun In addition to learning, it was also hypothesized that the three conditions would yield different experiences by the students in their reported amounts of fun. To determine their fun experience, two of the five items on the post-survey were used. These two items were the statements “This game was fun” and “This game was boring”. These items most directly tied with a student’s sense of fun. The other three items were not used because software glitches and usability issues during the session confounded their utility as measures of fun. A score was built on the two items once the second item was reverse-coded and One-Way ANOVA was again performed. Results of the ANOVA suggest a significant difference between scores F(2, 68) = 4.72, p < .02. Comparing the Abstract condition with the Original condition shows no significant difference t(44) = -1.38, p=.17, nonetheless, there is a significant difference between the Characters and Abstract conditions t(48) = 3.07, p < .01, d = .89, with Characters condition (M = 6.46, SD = 1.48) having a higher average rating than both the Original (M = 5.50, SD = 1.99) and Abstract conditions (M = 4.65, SD = 2.62). The lower score in the Abstract condition suggests that students ranked their experience as less fun than students in the Characters condition, which conforms to my hypothesis.

Investigating the relationship between fun and learning across conditions In addition to investigating the levels of fun and learning, it was also important to investigate the association between fun and learning across all three conditions. Figure 29 displays the gains that individual students had on the overall fractions test by condition while Figure 30 displays the gains that students had on the New Fractions

110 subscore. The height of each digit on both graphs represents the gain that a particular student had from the pre and post assessments. The particular digit represented on the graph is that particular student’s fun score. Upon an initial examination of both graphs, the Characters condition exhibited a pattern of having higher fun scores associated with higher learning gains. Nonetheless, this pattern does not seem apparent in the Abstract or Original conditions.

Figure 29: Student Gain Scores by Condition. The height of each digit represents the gain score for a particular student, and the digit used represents the student's fun score rating

111

Figure 30: Student Gains on New Fractions by Condition. Each digit represents the actual gain of one student. The digit itself is that specific student's reported level of fun

Table 6 below gives the correlations between the gain score (or subscore) and students’ reported rating of fun. Positive correlations would provide evidence that there is a positive association between fun and learning. In a positive correlation, students who reported having more fun would be linked to stronger gains from pre-test to post-test. The correlation table does not display a positive correlation for either the Abstract or the Original conditions, but rather a negative one. Furthermore, the Abstract condition has a stronger correlation than the Original condition, but in the negative direction. Nonetheless, the Abstract condition exhibited no significant correlations in either the fraction gain scores z(22) = -.89, p = .37, or the sub-gain scores z(22) = -.68, p = .50, which means that there is not a likely association between the gain score and fun rating. Because the Original condition was closer to zero than the Abstract condition, no significance tests were performed.

112

Abstract Original Characters Correlation (r) of Overall Gain with Fun -.14 -.08 .45 Correlation (r) of New Fractions with Fun -.11 -.09 .34 Table 6: Correlations of Fun Ratings with Gain or New Fractions sub-gain scores by condition

While the Abstract and Original conditions did not exhibit any correlations, there does seem to be a positive correlation between students’ self-reported ratings of fun and the gains that they made from pre-test to post-test in the Characters condition. The Characters condition exhibits significant positive correlations between the fun ratings and gain scores z(25) = 2.92, p < .01 as well as the New Fractions sub-gain scores z(25) = 2.17, p < .03.

Figure 31: Plotting Learning Gains by the Amount of Fun that a subject had. To describe the relationship between the fun total and overall gain scores, a graph was drawn placing each measure on an axis (See Figure 31). The letters in the graph represent the conditions that each student was in – A for Abstract, O for

113

Original, and C for Characters. In addition to reporting each individual’s relationship between their fun rating and gain scores, Figure 31 also provides a visual representation to the aforementioned correlations by condition; the lines drawn in the graph are the regression lines used to predict each condition’s association. The fact that there exists a positive correlation in the Characters condition, but no correlation in the other two conditions warrants further review. To make sense of what was happening, the following sections detail analyses of in-game measures and the video captured during gameplay.

Investigating in-game measures In addition to measures taken outside of the game, there were also internal measures that were created and stored in the software. These measurements were logs detailing activity from each user, such as when a question was answered without first getting it wrong and whether or not a student asked for help. To determine if the play by students differed by condition, a MANOVA was performed to investigate any differences by condition on several measures that were determined to be of possible use in earlier studies, such as the number of times they asked for help, the number of questions solved, and the average amount of time students took to answer a question. Performing the MANOVA demonstrates an overall fit of the data to the general model F(2, 70) = 1.99, p < .01, which would indicate that the type of play students exhibited seemed to differ by condition. Looking at the univariate scores, however, one notices that there is only one variable that carries a significant difference, the average number of questions a student would answer correctly on their first attempt in the last session F(2, 70) = 4.72, p < .02. By the last session, the effects that students have in playing the game should be different, which may explain the differences reported in Table 7 below.

Abstract Original Characters Number of 1st Attempts Correct in Last Session 6.00 (2.80) 8.32 (4.34) 9.15 (3.95) Number of Incorrect Moves 8.13 (3.46) 7.86 (4.89) 7.59 (3.21) Percentage of Incorrect Moves to Correct Moves 24.7 (9.1) 24.4 (9.1) 20.4 (6.6)

114

Table 7: Select in-game results by condition

Performing a planned comparison between the Abstract and Original conditions yields a significant difference between them, t(45) = 2.10, p < .05, as well as when comparing the Abstract condition with the Characters condition t(50) = 3.00, p < .01.

One line of reasoning about the Characters condition or story conditions was that if the students understood the story and its rules, they would be less likely to make errors. If they were less likely to make errors, then the software would have logged less of those errors on average. Table 7 above reports the average amount of times that a student would incorrectly make an incorrect move across all sessions. The MANOVA’s results displayed no significant differences across the three conditions F(2,70) = .12, p =.88. However, thinking about the question critically, solely investigating the number of incorrect moves was not as appropriate a measure as the percentage of incorrect moves to total moves, as students who played more would naturally make more incorrect moves. Analyzing how this measure differed by condition yielded a larger F-statistic but demonstrated no significant difference F(2, 70) = 2.13, p = .13. Performing planned comparisons between the Original and Abstract conditions yielded no significant difference t(45) = .12, p = .90; however, there was a marginal difference between the Abstract and Characters conditions t(50) = 1.84, p < .07. There is a marginal trend towards students having less of a tendency for making illegal moves. Based on the earlier analysis that correlated student’s fun with learning, it may be that interest is driving students to make fewer errors. When we look specifically at whether such a measure correlates with the amount of reported interest, we do get correlations in the correct direction for both the Original (r = -.26, t(20) = -1.18, p = .25) and Characters conditions (r = -.17, t(24) = -.82, p = .42), though neither of these are significant, while the Abstract condition shows no correlation between the percentage of incorrect moves and the amount of interest that students have (r = 0). While the in-game measures have generated insight into the student’s behaviors in the game, the data logging techniques did not catch any of the social behaviors that the children exhibited while playing the game. The next section

115 reports on qualitative analysis generated from the video captured in the Abstract and Characters conditions.

Investigating video With perspective to the videos, I used an open coding system, watching each video twice. Codes such as joint problem solving, teasing, story linking, emotion, pressuring, and helping others were generated. After having watched the video, what was immediately striking were the amounts of references made to story elements in the Characters condition that were not being made in the Abstract condition. I investigated this hunch more closely. For example, in the Characters condition, there appeared to be a frequent amount of teasing in the pods as to the moves they were going to make. One such example illustrates this teasing.

In this videosession, Freddy and Ramon have been paired together and are playing Lalo and Carmen. All four players are in a conversation. Lalo and Carmen have just placed a point-five air freshener to save some of their players. Freddy starts by announcing the card that Lalo and Carmen placed to protect their players.

Freddy - Air Freshener point five. Whatever we don’t really care, we’re gonna stink them back Carmen - Not even Freddy - Yes we can Lalo - We still got another air freshener (Freddy and Lalo play a 1/2 stink bomb on their end) Ramon - Now pick the yellow one Carmen - Why do you use that, you’re not gonna do that watch, you guys are gonna play the ice cream truck. Lalo - The red one is still fresh

If we look more closely at the conversation, we see that after announcing or reading the card, Freddy expresses his indifference to the card, and then adds that they are going to “stink” them back. At this point, there is a pause for a few seconds before Carmen looks at Freddy and Ramon and then says “Not even”, disagreeing with Freddy’s statement. Freddy then reiterates the possibility, and Lalo then tells Freddy and Ramon that they still have another air freshener. My interpretation of the first four lines of conversation is that there is some disagreement about what Freddy and

116

Ramon are able to play and whether they could “stink them back”. After Freddy and Ramon play the 1/2 stink bomb, Ramon then suggests to Freddy that he should pick the yellow one to be stinky, which I conclude is in reference to picking one of two characters. The character card that this pair is referring to is called The Twins. On this “twins” card, there is a drawing of two identical twins: one wearing a red shirt and another wearing a yellow shirt. While the twins were not given any names, it would make sense that the students would use color as a way to reference the characters, which both Ramon and Lalo referenced. Using color as an identifier also occurred in the interview study, where children spoke about the green and purple monsters in the Number Eaters game. Aside from deciding whom to stink and who is fresh, the dialog suggests that Carmen was reasoning about what card she felt her opponents were going to play, providing an example of the students thinking ahead in the game. Nevertheless, what is important from a story standpoint is the reliance that the students have in using the language of the story (air fresheners, stinking, getting people back, determining who is stinky). All four players in this group made mention or referenced elements or language specific to the story conditions that were not a part of the Abstract condition in this short excerpt.

This dialog is one of over 30 instances where these kinds of story referencing speech acts were made. The Characters condition heavily relied on the language during their gameplay, and theoretically the Abstract condition could have followed the same structure. These students could have talked about how they would get dots, lose dots, and how they reduce/restore dots. The video for the Abstract condition provides a sample of group interactions and talk references. Nonetheless, most of the video recorded in the Abstract condition suggests a far different reality. Conversations in the Abstract condition were commonly arguments between the individual pairs of students about answers and who had control, something that was rare if at all present in the Characters condition. The recordings provided few instances of the Abstract pod using the language in their version of the game. Below are all of the transcripts from the video that could be argued as referencing story elements in the Abstract condition where Tito and Hector played against Angel and Gabriel:

117

Gabriel (1-9:20) – We’re tied right now look – three, six, nine, two, four, six, eight, nine – it’s nine on nine (reference to dots)

Gabriel (1-14:25) – Yay! (Giggles)…No – wait! Reduce is over theirs! No! (Reduce card) Tito/Hector (2-3:28) – Están felices! [They are happy!] (Dots)

Gabriel (2-3:48) – Oh wait – they have more…They won again…Why do they give them more dots than us all the time?

Gabriel (2-8:29) – Put two tokens…look look (Two tokens) Gabriel (2-10:20) – We won! We only got four points, oh no six points.

Over the 25 minutes of audio that I have for this group, the six statements made in this segment (and one additional example below) are the only statements arguably related to the referent language structure of the game. Other than the example below, I found no other mentions of people (dots), the stink bomb (reduce), air freshener (restore), or the ice cream truck (chance). While the Abstract pod only had seven references, the Characters condition had over 30 references to these same elements. In the Abstract condition, there is only one extended example where the pod could be referencing the story elements. This example occurs when a chance card is played.

Tito - I wanna shake the bag I'm good at it Hector - no no no, mira Hector [look…Hector] (referring to name on screen) Gabriel - It doesn't matter with three draws our dots are still gonna stay with three draws Other student - Oh but you guys can leave if you have two leaves Gabriel - No but we have three draws, so even with one stay we stay, the dots stay…Hurry up, We don't have to follow it Angel - We don't have all day Gabriel - I know we only have like 3 more minutes left because we end at 11:10 but we have to pack up early to go to the other room Angel - Go! Gabriel - Alright…leave, stay, stay, hehehehe, we got two stays, (sings) the dots stay, the dots stay

In this example, there is a little bit more buzz in the pod as they play the game. Before the start of this excerpt, the researchers inform the entire Abstract condition

118 that they don’t have much time left before class ends, as noted by Gabriel. Gabriel also has a story-related discussion about the rules of the game using the leave and stay language used in the Chance card. Gabriel uses logic to determine that they do not have to worry about the Chance card because of the amount of opportunities they have to pick a stay chip. Gabriel also shows some positive affect by singing the dots stay, reading the result displayed on the screen. This is one of the few instances where players in the Abstract condition use the game’s language to explain what is happening in the game.

Because the final day presented all pods with the same cards in the same order, an analogous scenario also occurred in the Characters condition. The dialog for this event follows:

Freddy – Yeah! We played two radios, so we have more chance to get them Ramon – It’s ours? Freddy – Yeah, they’re gonna shake the bag Ramon – Y cual es? [And which one is it?] Freddy – Shake it shake it, whatever, we still got three draws left, shakes the bag Imma pick them ok? (Lalo and Carmen finish shaking the bag, Freddy draws the chips) Freddy – The kids are safe! Yeah! We got them back” (Freddy and Ramon giggle) (2 minutes later) Freddy – Ves si nos sirvieron los radios [You see, the radios were useful to us]

Again there is a heavy reliance on the language of the game in the story – in this case the radio (extra token in the Abstract condition) – that helped Freddy explain the rules to Ramon. At both the beginning and end of this snippet, one notices that Freddy is suggesting to Ramon how the radio works and why it is useful. When Freddy mentions that the kids are safe, he is reading a message that is displayed on the screen, however, his excitement and vocalization of getting people back suggests that he understood the consequences of the card. His final line about how the radios were useful also suggests his understanding of how the game and story in that condition work together, and how such an understanding could pass between the students. The

119

Characters condition has throughout the video recorded many instances of the children using the language in the game. By using the story’s language in their conversations and teasing, the students stayed engaged and gave themselves more opportunities to understand the game and its content.

It is unclear why the Abstract pod seemed to use the language from the game considerably less. If I speculate, there are several other possibilities. For example, it could be that the language did not make as much sense to them, the children might had simply been tired with the game, or they may not had been engaged in the first place. Because the research literature on story has demonstrated that story is more likely to lead to better comprehension, it could just be that the players in the story condition were able to rely more on the micro and macrostructures to help understand the game and its rules. Other than the examples cited there seems to be few examples of the Abstract condition using the game’s language – even when it would seem appropriate. For example, Hector, while talking with a researcher, talks about the chance card but never mentions it by name,

Hector: (2-8:09) “He (Tito) should put that one because I like that one.”

When the researcher informs Hector that he should suggest to Tito what card Tito should play. Hector then goes on to tell Tito: “Pon ese”, which translates to “put this one”. In this small interaction, Hector had three opportunities to refer to the card as a Chance card, but instead chooses to reference it by using a general descriptor. I do not know if Hector’s English ability prevented him from using detailed language in this example. If this were the case then the other students would use the language more frequently, but the other students in the pod did not use the language with much frequency either.

The Characters pod video demonstrated both constant teasing and frequent use of the language tied to the story. This was exhibited in several of the statements that students made during the gameplay. In one situation, Freddy teases Lalo and Carmen about a recent move in their gameplay, saying, “You freshened them all and now

120 they’re stinky” and then giggling. This statement both demonstrates how the students were using the language as well as some of the mild teasing or taunting that was happening in reference to game actions. In another situation, after a round was over and Lalo and Carmen won the round, Lalo tells Freddy, “You said you were gonna crush us” to which Freddy quickly replied, “Well yeah, you got the basketball team”. In this interaction, Freddy is using the game’s language (basketball team) to explain why they could not crush their opponents like he had proclaimed earlier. This also suggests some knowledge of the basketball team card as a powerful card, which was demonstrated at another point in the video when Freddy states, “ooh the basketball team”. That type of excitement for powerful cards was not observed at all in the video from the Abstract pod. In fact, there was rarely any sign of emotion other than in- fighting and the example given above for the Abstract pod.

Videos from the two pods provide evidence of the children in the Characters condition being more engaged, more on task, and on the whole having a better grasp of the rules. While there are a variety of reasons for these differences, I speculate that the teasing between groups would keep the references to the story fresh in students’ minds. The engagement, and the use of language to tease, may possibly explain why a positive correlation between fun and learning existed in the Characters condition. The engagement that students could have with the story and the use of the language to tease their opponents, could have had led to a desire to learn more about the game and kept them more involved than they would have been in both the Abstract condition. The first dialog presented in the video analysis sub-section might also help explain why the Characters condition was stronger than the Original condition. In that dialog, there were two separate segments of conversation that exhibited explicit references to which characters were made stinky and which were made fresh. Other examples of this type of dialog were also present in the Characters condition, where time was spent by the students discussing and debating who they would stink. One could argue such discussions would not be as salient in the Original condition, since all of the characters look exactly the same.

121

Discussion One of the questions that I aimed to answer with this experiment was whether or not story is a necessary feature in developing effective educational games. While one may believe that story is necessary based on the literature, the results from the interview study indicated that story might not be necessary. Initial results from the first experimental study provided support for the latter point, demonstrating that students increased from pre-test to post-test regardless of condition. This may lead to conclusions that embedding a story in an educational game may not be necessary for helping students learn more. However, further exploration of the results from the experimental study indicate a more complex scenario for the role that story may play in educational video games.

While the initial results from the final study could be used to argue that students can learn from a game without story, one should also recognize the marginal differences that were present overall between the Abstract and the Characters conditions. In particular, the Characters condition displayed marginal trends in having larger gains from pre-test to post-test when compared to the Abstract condition. These results, combined with the high gains irregardless of condition, lay the foundation for a general viewpoint about the role of stories in educational games: a story may not be necessary for creating a successful educational video game, but its presence may help in increasing student’s gains from pre to post.

The role of a story becomes clearer with respect to learning when one looks specifically at the types of questions asked on the tests. Once the pre and post-tests were broken down by the type of fraction question, the results indicated that most of the types of questions did not differ by condition. Nevertheless, the results did indicate that one question category did differ by condition – new fractions questions. Students in the Characters condition had significantly higher gains when answering new fractions questions when compared to students in the Abstract condition. This means that students in the Characters condition had higher gains from pre-test to post- test when answering fraction questions they had not seen in the game, like “What is

122

2/5 of 15?”. Therefore, not only does the story with characters marginally allow students to learn more, those students were better able to answer fractions questions they had not seen before. Because students in the Characters condition were able to answer questions they had not seen before, one may conclude that the Characters condition was better preparing students to think about fractions in the future. This finding coincides with earlier research with commercial video games and preparation for future learning (Arena, 2012). While the research presented in Arena’s work establishes that video games can be used as preparation for future learning in a domains such as history, I would argue that the results from this experimental study both support Arena’s research and further reveal the role that story has as being a potential critical aspect for that preparation. While game play may be strong enough to drive learning in a specific domain, having story may help students be better prepared to learn new material related to that domain in the future.

In addition to fractions learning, I was interested in measuring the amount of fun students had across conditions. The results reported in the last study highlight this fact without much nuance from the learning standpoint. Students who were in the Characters condition had significantly higher ratings of fun than students in the Abstract condition. This suggests that story not only seems to prepare students to answer complex fractions in the future, it also helps make an educational game more fun. Fun is important because it could lead both to more positive affect and more engagement and motivation (Plass et al., 2010; Read, MacFarlane, & Casey, 2002; Slater & Rouner, 2002) in the fractions context. Because of the evidence of story being able to provide benefits for fun and learning, it would make sense that, from a theory standpoint, story is an island on the educational voyage that has impacts in both areas.

One last finding of interest was the high correlation present between the amount of fun that the students reported, and the amount that a student’s score increased from pre-test to post-test. While such a correlation displayed no correlation in the Abstract and Original conditions, the Characters condition exhibited a strong

123 correlation between fun and learning. Therefore, the results reveal a strong positive association between the amount of fun a Characters condition student had and the amount that they learned. One possible speculation for this comes from previous research done on transportation (Green & Brock, 2000). That research highlighted the role that the amount that a student was absorbed in a narrative would affect that amount that they were persuaded by such a narrative. Green & Brock’s research may be one way to explain the high correlation – students who enjoyed and were more absorbed by the story were more persuaded and thus motivated to learn the material.

The results from the video data also support the idea that students who enjoyed the game in the Characters condition would be more likely to learn. Transcripts and video exhibit the amount of chatter about the story that certain students made, like Freddy, who was discussed earlier. Freddy displayed some utterances and phrases that related elements of the story to the gameplay as the group was playing. The amount of talk that Freddy displayed linking the story to the game was high. Not surprisingly, Freddy also had a high gain score from pre to post and rated the game with the highest possible fun score. Because of the amount of dialog that Freddy used in the game about the story, I would argue that Freddy was absorbed in the story, which could then lead him to have more fun with the game, and be persuaded to think about and focus on the game’s pedagogical content. If a child is having fun, then he or she may be more likely to learn the material. When children play educational games, they can choose whether or not to pay attention to the educational message. If they become absorbed in the story which leads them to have more fun, research on story contends that they would be more open to learning the material or having a more open view on the game itself (Koenig & Zorn, 2002; Sunwolf & Frey, 2001). I like the idea of playing with this idea and tying it to Green & Brock’s work that details the amount that one is willing to be absorbed dictates how much they will be persuaded (2000). The difference between Green & Brock’s study and these findings is that rather than be persuaded by the story’s message; I argue that being absorbed in the story persuades students to learn.

124

This discussion contributes to the understanding of the roles that story has to play in educational games. The Tug-of-War story did seem to make students enjoy the game more. Having that story, however, does not seem completely necessary as an aid to children’s learning. Nonetheless, it does seem to have some pedagogical benefits. Having a story with characters seems to have had an effect on students’ performance on fractions questions they had not seen before. The amount of fun students had in playing the game also correlated well with how much students learned in the story-laden Characters condition, which was not apparent in the other conditions. The results also showed that children who played in the story with characters condition got marginal learning benefits overall. Having this fuller understanding of the role that story has in games can help the educational community think about how many resources might be devoted to the creation of a story and its incorporation into a game. If a designer does a good job in creating a powerful story that most people enjoy, that designer may have an easier time engaging students to work with the material. It may better prepare students for future learning, as well as providing them more entertainment.

New Questions While the last study generated the findings discussed, the results also generated new questions for investigation. For example, the fractions test results by type of question (see Table 5) indicated that the game in its current state is not increasing student’s scores on comparisons, word problems or decimal problems from pre to post. However Table 5’s results (page 109) also displayed trends in the two other types of questions (New Fractions and Old Fractions) which show differences from pre to post and possible differences by condition in the direction one would expect. The difference between pre-test and post-test on students’ New Fractions scores turned out significant by condition while the Old Fraction scores did not. Future research could devise more Old Fractions questions that could help explain the trends by condition in the results. Having a larger corpus of fractions questions could help us to further divide by type of question. For example, one question category that shows promise from the data are items that are formulated to look exactly like what they

125 would see in the game (What is X/Y of Z?), which I call basic questions. Currently, the Old Fractions bank of questions includes both basic questions and other questions that mirror other activities that students perform in the game. Since there are only six basic fractions questions on the assessment, having additional basic fractions questions in the assessment might allow stronger claims to be made as to what the game is and is not helping students do by condition.

Another question that is raised by the results would be why the correlation between fun and learning was only found in the Characters condition. Both the Original and Abstract conditions had either no correlation or what approached a negative correlation between the fun ratings and gains from pre to post. While I have proposed a hypothesis for why the Characters condition is related, it is equally important to try and figure out how subjects in the two other conditions related fun with learning, if they did at all. The Abstract condition videos suggest that students struggled more with understanding the game, answering the questions, and general boredom. It may be that their ratings for how much fun they had were related to these other factors. On the other hand, it could be that their ratings were based on other factors, such as their feelings about the partners they were given. Thus, it is important to search for other factors other than fun that may be associated with student’s scores from pre to post in conditions that did not contain as many story elements. Understanding relationships for why some students learned more than others in the Original and Abstract conditions could also help contribute to the knowledge of creating excellent serious games, which do not always strive for having a compelling story.

While the video data detailed and informed ways in which the Abstract and Characters conditions differed, little is known about the social interactions occurring in the Original condition. Another question to answer is to look more closely at how children played and talked in the Original condition, and how that compared to the Abstract condition and the Characters condition. Such an analysis could help us understand if the changes are really helping students and encouraging them to engage

126 differently, which at this point may be likely. By videotaping, one could also figure out whether students mirrored the Characters condition and used language to get them interested in the story, or whether they exhibited a different type of behavior while playing the game.

Study Limitations While the results of the study provided interesting discussion and generated new questions, the study had its limitations. The most basic limitation was the lack of a control condition. Some may argue that not having a control condition prevents the study from stating that all three versions of the game helped the students learn. Since all three groups of students had gains from pre-test to post-test, having a control group would have allowed for stronger claims. One of the biggest hurdles to having a control group was the limitation on the number of students in the study. The number of subjects was not large enough that a separate fourth condition could have been used. Implementing a crossover design similar to what was done in earlier studies could have worked as it would not require an increase in subjects. Nonetheless, the crossover design was not feasible because of the restricted set of resources (time, rooms, people, and curriculum) that would have been necessary to implement such a design. While there was no true control condition in this last study, previous studies using earlier versions of the game did have control conditions and the results from those studies suggest that students who played the game did learn more fractions material, which could help assuage concerned readers. The highly scrutinizing reader may still have reservations with such a claim as the transition from earlier experimental studies to the latest experimental study was not without its changes. Because the latest experimental study used an updated version of the game, the sample sizes were larger and the researchers were blind to hypotheses, the lack of control must be considered as a limitation.

The second limitation in conditions was the difficulty in making sure the Original condition matched the other two conditions, so that stronger claims could be made between the Original condition and Abstract condition or the Original condition

127 and Characters condition. In general if resources were limited across conditions, the Original condition was the first to be slashed. For example, the Original condition did not have any videotapes so students who did not have parental consent for video were swapped out with students who had similar standardized scores. Moreover, the Original condition suffered two network outages on the third and fourth sessions that reduced playing time by 30 to 40 minutes overall when compared to the other two conditions. Because the previous studies used versions of the game that looked most like the Original condition, I felt again that it was important to make sure the two more fringe conditions looked as close to each other as possible. Despite all of these setbacks, it is still surprising to me how much the Original condition fell in between the other two conditions in terms of learning and fun. It is unclear if the reduced playing time would have any effect. Having a better matched Original condition may have allowed for better analyses of the Original condition’s impact.

The third limitation comes from a technical standpoint. It is quite a challenge to keep an increasingly complex software program stable. A major obstacle in custom-made educational video games is the potential for having bugs. This potential for bugs increased dramatically as it got closer to the time of the study, as the amount of features and fixes rose steadily and the amount of time that could be devoted to solving such issues diminished steadily. This caused most pairs of users to suffer at least one bug or restart during their five weeks of playtime. Certain pods or pairs seem to suffer more restarts than other pods, which could have affected how much they learned. Nevertheless other than the specific issues with the Original condition, I did not notice any differences in restarts by condition.

Finally, the last limitation is that there could have been stronger linkage between the manipulations by condition and the addition of the story. In the previous discussion on the interpretation of results and its impact on story, I argued that the inclusion of story was the reason that the Characters condition displayed benefits. One thing that was missing was manipulation checks for story between the three conditions. Because of the lack of resources, participants were not pulled aside and

128 interviewed to gain insight into their perceptions of the game’s story, as they had been in the interview study. Efforts to gather such data via the post-test measures were largely unsuccessful. For example in the interview study, we would ask the children, “What is this game about?” and would then continue on by getting some type of synopsis of the game. However, when the post-test questionnaire asked the children to write down an answer to this question, most children, regardless of condition wrote down, “This is a game about fractions” and then proceeded to the next question. Not having the ability to have students discuss their thoughts on the story led to the reliance on the interview study results, limiting the experimental study’s findings.

Next Steps The limitations discussed in the previous section raise an important and more general point. Not only is creating educational video games a difficult process, but so is conducting research around the effectiveness of the elements of video games. Nonetheless, these studies have provided insight into how the inclusion of story and concentration on specific aspects of stories can impact an educational game. There are three directions that research can pursue based on these results: 1) investigate the role that story plays more deeply in further experiments, and 2) use the collected gameplay data to do a better job in predicting student behavior, 3) devise studies that investigate the role that the other islands (choice and collaboration) play in educational video games.

While the results herein have started to make a case for how story can impact learning, much more needs to be done to understand the ways that story can impact learning in educational games. One clear result from the limitations is to understand and investigate more closely the impact that characters play in games. While both the interview and last experimental study reveal the prominence that characters may have in stories, there is insufficient evidence to generate claims about what is necessary to make a great story. Similar to the finding that the type of story, and how engaged one is in that story could matter (Green & Brock, 2000), more can be done to generate different types of stories to investigate this claim further. Possible future studies could

129 be conducted in which the Original and Characters conditions used in the last study are modified. Changes could also be made according to the latest story model adopted (Kintsch & Van Dijk, 1978) that could help Tug-of-War have improved microstructures and macrostructures. One such area that may benefit from such a structure would be to improve the link between the characters in the game and player’s role in manipulating those game’s characters. If a macrostructure is introduced where students have a better understanding of how a specific character they manipulate has an effect on the entire story and that character’s reason for wanting to win the game, that macrostructure may make such a story even more of a vital part of the game.

Part of what has been included in the development of this game is logging of the actions that students take within the game. This is an important milestone for research because it can supplement what can be learned from the pre- and post- measurements of student learning. One future direction that could be to help build a model and analyze how the decisions in the game could help predict the test scores that the students were given outside the game. This type of analysis could help build on recent research surrounding the future of assessment in educational software (Schwartz & Arena, 2009; Shute, 2011) as it could help provide an additional independent model of how Tug-of-War could assess or predict players’ knowledge via the behaviors and answers that users provide in the game. It could also serve as a complementary model to other models of student knowledge present in other fraction games (Andersen, 2012). Notwithstanding, one way that Tug-of-War is different from other recent games is that it engaged a social component that is equally critical to the development of the game and outcomes. The video of the in-game actions and subsequent out of game interactions can be holding important implications for designing future games. This data along with follow up studies, could serve as a primer in another possible future direction – understanding how the collaboration and choice islands mentioned this dissertation framework affect educational games.

130

Figure 32: The conceptualization framework proposed early on

This research could help understand the validity of the conceptualization framework proposed at the beginning of the study (see Figure 32). That conceptualization could be further solidified by examining two other areas – Choice Island and Collaboration Island. Particularly, I am most excited about exploring collaboration and the educational and/or entertainment impact it can have. My current hunch is that the element of collaboration as has been documented previously in technology could have a tremendous impact in educational games. While there is this movement towards using computers to individualize education (Corbett, Koedinger, & Anderson, 1997; Koedinger, Anderson, Hadley, & Mark, 1997), I believe that the collaboration literature has already demonstrated huge impacts in the promise that collaboration has in helping people learn both via technology (Koschmann, 2012; Okita, Bailenson, & Schwartz, 2007) and via games (Nasir, 2005). An important first step in this matter is to establish and investigate the role that collaboration and competition against others in educational games have on learning and the associated fun. Understanding the ways that story, collaboration and choice could affect both fun

131 and learning is an important next step in helping the games community navigate the sirens of educational game design.

Conclusion Creating educational video games are not for the faint of heart. Video games by themselves are expensive and difficult to create, and struggling to balance learning objectives with entertainment and software goals makes it even more difficult. The creation and design process can lead educational game creators and designers to an array of possible failures, mistakes and missteps. This is no more apparent than in the design and making of Tug-of-War. Tug-of-War’s iterations and experimental studies demonstrated the sheer difficulty and failures associated with the educational game design process. After having modified and created a game that students found fun and could learn from, I have proposed a framework that can help the educational games community better think about how to successfully design and create such educational games by making them aware of the sirens of fun and learning.

After having navigated the Tug-of-War game through this space and around the sirens of fun and learning, I also propose three areas (or islands) that are worth further investigation – choice, collaboration and fantasy/story. This dissertation focuses on the latter (fantasy/story), first reviewing the literature on story and noticing the impacts that story has been documented in having on comprehension and on persuasion, which I argue is linked to learning. In addition to reviewing the literature, I also conducted an interview study with children to search for the critical parts of story. After finding the importance that the characters component has on story, I then discuss the results of the last experimental study, which uses three different versions of the same game to investigate more closely the role that story (and characters) have on learning. The results showed that while story is not necessary to help children learn educational content, it may help. Adding story with characters also made the educational game a more enjoyable experience for children who play it. While creating games is certainly at times an insurmountable process, there is a great joy in hearing children’s excitement in playing educational games, and knowing at the same

132 time that it is helping those students learn concepts they may struggle with. Creating artifacts that children enjoy and can help them learn is sufficient enough of a reason to continue through the many failures and missteps of this process. By conducting more research such as this, the educational games community can create more viable and easier paths for creating more didactic and enjoyable games.

133

Bibliography Allen, M., & Preiss, R. W. (1997). Comparing the persuasiveness of narrative and

statistical evidence using meta‐analysis. Communication Research Reports,

14(2), 125–131. doi:10.1080/08824099709388654

Altarriba, J., & Santiago-Rivera, A. L. (1994). Current perspectives on using linguistic

and cultural factors in counseling the Hispanic client. Professional

Psychology: Research and Practice, 25(4), 388–397. doi:10.1037/0735-

7028.25.4.388

Andersen, E. (2012). Optimizing adaptivity in educational games. In Proceedings of

the International Conference on the Foundations of Digital Games (pp. 279–

281). New York, NY, USA: ACM. doi:10.1145/2282338.2282398

Anderson, N., Lankshear, C., Timms, C., & Courtney, L. (2008). “Because it”s boring,

irrelevant and I don’t like computers’: Why high school girls avoid

professionally-oriented ICT subjects. Computers & Education, 50(4), 1304–

1318. doi:10.1016/j.compedu.2006.12.003

Arena, D. (2012, June 5). Commercial Video Games as Preparation for Future

Learning.

Aronson, E., Blaney, N., Stephan, C., Sikes, J., & Snapp, M. (1978). The jigsaw

classroom. Oxford, England: Sage.

Ball, D. L., & (US), E. R. I. C. (1990). Halves, pieces, and twoths: Constructing

representational contexts in teaching fractions. National Center for Research

on Teacher Education, Michigan State University.

134

Barab, S. A., Sadler, T. D., Heiselt, C., Hickey, D., & Zuiker, S. (2007). Relating

Narrative, Inquiry, and Inscriptions: Supporting Consequential Play. Journal of

Science Education and Technology, 16(1), 59–82.

Baroody, A. J., & Ginsburg, H. P. (1990). Chapter 4: Children’s Mathematical

Learning: A Cognitive View. Journal for Research in Mathematics Education.

Monograph, 4, 51–210. doi:10.2307/749912

Barron, B. (2003). When smart groups fail. Journal of the Learning Sciences, 12(3),

307–359.

Bartol, G. M., & Richardson, L. (1998). Using Literature to Create Cultural

Competence. Journal of Nursing Scholarship, 30(1), 75–79.

doi:10.1111/j.1547-5069.1998.tb01240.x

Behr, M. J., Wachsmuth, I., Post, T. R., & Lesh, R. (1984). Order and Equivalence of

Rational Numbers: A Clinical Teaching Experiment. Journal for Research in

Mathematics Education, 15(5), 323–341. doi:10.2307/748423

Black, J. B., & Bower, G. H. (1980). Story understanding as problem-solving. Poetics,

9(1–3), 223–250. doi:10.1016/0304-422X(80)90021-2

Bower, G. H. (1978). Experiments on story comprehension and recall∗. Discourse

Processes, 1(3), 211–231. doi:10.1080/01638537809544437

Bowman, R. F. (1982). A Pac-Man theory of motivation. Tactical implications for

classroom instruction. Educational Technology, 22(9), 14–17.

Bransford, J., National Research Council (U.S.). Committee on Developments in the

Science of Learning, & National Research Council (U.S.). Committee on

135

Learning Research and Educational Practice. (2000). How People Learn:

Brain, Mind, Experience, and School. National Academies Press.

Brooks, F. P. (1975). The mythical man-month (Vol. 79). Reading, Mass: Addison-

Wesley.

Bryant, J., & Miron, D. (2003). Excitation-Transfer Theory and Three-Factor Theory

of Emotion. Communication and emotion: essays in honor of Dolf Zillmann,

31.

Buckingham, D., & Sefton-Green, J. (2003). Gotta Catch’em all: Structure, Agency

and Pedagogy in Children’s Media Culture. Media, Culture & Society, 25(3),

379.

California Department of Education (Safe and Healthy Kids Program Office), &

WestEd (Health and Human Development Department). (2012). California

Healthy Kids Survey. Retrieved from http://chks.wested.org/resources/chks-

elem-bilingual-1213.pdf

Carnagey, N. L., & Anderson, C. A. (2005). The Effects of Reward and Punishment in

Violent Video Games on Aggressive Affect, Cognition, and Behavior.

Psychological science, 16(11), 882.

Castronova, E., Ross, T., Bell, M., Falk, M., Cornell, R., & Haselton, M. (2008).

Constructing Arden: Life Inside the Machine. Multimedia, IEEE, 15(1), 4–8.

Celano, M., Hazzard, A., Webb, C., & McCall, C. (1996). Treatment of traumagenic

beliefs among sexually abused girls and their mothers: An evaluation study.

Journal of Abnormal Child Psychology, 24(1), 1–17.

136

Chen, J. (2007). Flow in games (and everything else). Communications of the ACM,

50(4), 31–34.

Cohen, E. G. (1994). Restructuring the Classroom: Conditions for Productive Small

Groups. Review of Educational Research, 64(1), 1–35.

doi:10.3102/00346543064001001

Confrey, J., Maloney, A., Nguyen, K., Mojica, G., & Myers, M. (2009).

Equipartitioning/splitting as a foundation of rational number reasoning using

learning trajectories. Proceedings of the 33rd Conference of the International

Group for the Psychology of Mathematics Education, 1(1998), 1–8.

Confrey, J., & Smith, E. (1995). Splitting, Covariation, and Their Role in the

Development of Exponential Functions. Journal for Research in Mathematics

Education, 26(1), 66–86. doi:10.2307/749228

Corbett, A. T., Koedinger, K. R., & Anderson, J. R. (1997). Intelligent tutoring

systems. Handbook of human-computer interaction, 849–874.

Cordova, D., & Lepper, M. (1996). Intrinsic Motivation and the Process of Learning:

Beneficial Effects of Contextualization, Personalization, and Choice. Journal

of Educational Psychology, 88(4), 715.

Costantino, G., Malgady, R. G., & Rogler, L. H. (1986). Cuento therapy: A culturally

sensitive modality for Puerto Rican children. Journal of Consulting and

Clinical Psychology, 54(5), 639–645. doi:10.1037/0022-006X.54.5.639

Csikszentmihalyi, M. (1990). Flow : the psychology of optimal experience. New York:

Harper & Row.

137

Davis, R. B. (1984). Learning Mathematics: The Cognitive Science Approach to

Mathematics Education. Greenwood Publishing Group.

Elliott, E. S., & Dweck, C. S. (1988). Goals: an approach to motivation and

achievement. Journal of personality and social psychology, 54(1), 5.

Epstein, D. (2005, October 26). Joystick Nation. Inside Higher Ed. Retrieved from

http://www.insidehighered.com/news/2005/10/26/games

Esmonde, I., Blair, K. P., Goldman, S., Martin, L., Jimenez, O., & Pea, R. (2013).

Math I Am: What We Learn from Stories That People Tell About Math in

Their Lives. In B. Bevan, P. Bell, R. Stevens, & A. Razfar (Eds.), LOST

Opportunities (Vol. 23, pp. 7–27). Springer Netherlands. Retrieved from

http://www.springerlink.com/content/j0852m156u1725qv/abstract/

Fazio, R. H., & Zanna, M. P. (1981). Direct experience and attitude-behavior

consistency. Advances in experimental social psychology, 14, 161–202.

Fisher, W. R. (1985). The narrative paradigm: An elaboration. Communication

Monographs, 52(4), 347–367. doi:10.1080/03637758509376117

Fullerton, T., Swain, C., & Hoffman, S. (2004). Game Design Workshop: Designing,

prototyping, and playtesting games. Focal Press.

Gardner, R. A. (1971). Mutual storytelling: a technique in child psychotherapy. Acta

Paedopsychiatrica, 38(9), 253–262.

Garfield, J., & Ahlgren, A. (1988). Difficulties in Learning Basic Concepts in

Probability and Statistics: Implications for Research. Journal for Research in

Mathematics Education, 19(1), 44–63. doi:10.2307/749110

138

Gee, J. P. (2004). What Video Games Have to Teach Us About Learning and Literacy.

Palgrave Macmillan.

Goldman, S., Pea, R., & Maldonado, H. (2004). Emerging social engineering in the

wireless classroom. In Proceedings of the 6th international conference on

Learning sciences (pp. 222–229). International Society of the Learning

Sciences. Retrieved from http://dl.acm.org/citation.cfm?id=1149126.1149152

Goldstone, R. L., & Wilensky, U. (2008). Promoting Transfer by Grounding Complex

Systems Principles. Journal of the Learning Sciences, 17(4), 465–516.

doi:10.1080/10508400802394898

Green, M.C. (2004). Storytelling in teaching. APS Observer, 17(4), 37–39.

Green, Melanie C., & Brock, T. C. (2000). The role of transportation in the

persuasiveness of public narratives. Journal of Personality and Social

Psychology;Journal of Personality and Social Psychology, 79(5), 701–721.

doi:10.1037/0022-3514.79.5.701

Griffin, S. A., Case, R., Siegler, R. S., & McGilly, K. (1994). Rightstart: Providing the

central conceptual prerequisites for first formal learning of arithmetic to

students at risk for school failure. In Classroom lessons: Integrating cognitive

theory and classroom practice (p. 25––49).

Guberman, S. R., & Saxe, G. B. (2000). Mathematical Problems and Goals in

Children’s Play of an Educational Game. Mind, Culture, and Activity, 7(3),

201–216.

Habgood, M. P. J., Ainsworth, S. E., & Benford, S. (2005). Endogenous fantasy and

learning in digital games. Simulation & Gaming, 36(4), 483.

139

Habgood, M. P. Jacob, & Ainsworth, S. E. (2011). Motivating Children to Learn

Effectively: Exploring the Value of Intrinsic Integration in Educational Games.

Journal of the Learning Sciences, 20(2), 169–206.

doi:10.1080/10508406.2010.508029

Harter, S. (1981). A new self-report scale of intrinsic versus extrinsic orientation in the

classroom: Motivational and informational components. Developmental

Psychology, 17, 300.

Heath, C., Heath, D., & Kahlenberg, C. (2007). Made to stick: Why some ideas survive

and others die. Random House New York.

Hodgen, J., Küchemann, D., Brown, M., & Coe, R. (2010). Lower secondary school

students’ knowledge of fractions. Research in Mathematics Education, 12(1),

75–76.

Holson, L. M. (2004, April 10). In Land of Oscars, Rich New Category Is Best Video

Game - New York Times. New York Times. Retrieved from

http://www.nytimes.com/2004/04/10/business/in-land-of-oscars-rich-new-

category-is-best-video-game.html?pagewanted=all&src=pm

Honey, M., & Hilton, M. (2011). Learning Science: Computer Games, Simulations,

and Education. National Academies Press.

Hooper, S., & Hannafin, M. J. (1988). Cooperative CBI: The effects of heterogeneous

versus homogeneous grouping on the learning of progressively complex

concepts. Journal of Educational Computing Research, 4(4), 413–424.

doi:10.2190/T26C-3FTH-RNYP-TV30

140

Hovland, C. I., Janis, I. L., & Kelley, H. H. (1953). Communication and persuasion;

psychological studies of opinion change. New Haven, CT, US: Yale University

Press.

Hunicke, R., & Chapman, V. (2004). AI for Dynamic Difficulty Adjustment in

Games. In Challenges in Game Artificial Intelligence AAAI Workshop (p. 91–

96).

Hunicke, R., LeBlanc, M., & Zubek, R. (2004). MDA: A formal approach to game

design and game research. In Proceedings of the AAAI-04 Workshop on

Challenges in Game AI (pp. 1–5).

Ito, M. (2006). Intertextual Enterprises: Writing Alternative Places and Meanings in

the Media Mixed Networks of Yugioh. Encountering the Extraterrestrial:

Anthropology in Outerspaces.

Ito, M. (2007). Education vs. Entertainment: A Cultural History of Children’s

Software. In K. Salen (Ed.), The Ecology of Games: Connecting Youth, Games,

and Learning (pp. 89–116).

Iyengar, S. S., & Lepper, M. (1999). Rethinking the value of choice: A cultural

perspective on intrinsic motivation. Journal of Personality and Social

Psychology, 76(3), 349.

Iyengar, S. S., & Lepper, M. (2000). When Choice Is Demotivating: Can One Desire

Too Much Of A Good Thing? Journal of Personality and Social Psychology,

79(6), 995.

Jimenez, O., Arena, D. A., & Acholonu, U. (2011). Tug-of-War: a Card Game for

Pulling Students to Fractions Fluency. In C. Steinkuehler, C. Martin, & A.

141

Ochsner (Eds.), Proceedings of the Games, Learning, & Society Conference

7.0. Presented at the Games, Learning & Society 7.0, Madison, WI: ETC Press.

Kerslake, D. (1986). Fractions: Children’s strategies and errors: A report of the

Strategies and Errors in Secondary Mathematics Project.

Kintsch, W., & Van Dijk, T. A. (1978). Toward a model of text comprehension and

production. Psychological review, 85(5), 363.

Kirkwood, W. G. (1983). Storytelling And Self-Confrontation: Parables as Teaching

Stories. Quarterly Journal of Speech, 69(1), 58–74.

Kirkwood, W., & Gold, J. (1983). The Philosopher As Teacher: Using Teaching

Stories to Explore Philosophical Themes In The Classroom. Metaphilosophy,

14(3-4), 341–352.

Kirriemuir, J., & McFarlane, A. (2004). Literature review in games and learning: a

report for NESTA Futurelab. NESTA Futurelab series report 8, 5.

Koedinger, K. R., Anderson, J. R., Hadley, W. H., & Mark, M. A. (1997). Intelligent

Tutoring Goes To School in the Big City. International Journal of Artificial

Intelligence in Education (IJAIED), 8, 30–43.

Koenig, J. M., & Zorn, C. R. (2002). Using Storytelling as an Approach to Teaching

and Learning With Diverse Students. Journal of Nursing Education, 41(9),

393–399.

Koschmann, T. (2012). CSCL: Theory and practice of an emerging paradigm.

Routledge.

Koster, R. (2005). A Theory Of Fun In Game Design. Scottsdale, AZ: Paraglyph Press.

142

Lamon, S. J. (1994). Ratio and Proportion: Cognitive Foundations in Unitizing and

Norming. In J. Confrey (Ed.), Development of Multiplicative Reasoning in the

Learning of Mathematics (pp. 90–121). Ithaca, NY, USA: State University of

New York Press. Retrieved from

http://site.ebrary.com/lib/stanford/docDetail.action?docID=10022521

Langer, E. J. (1975). The illusion of control. Journal of Personality and Social

Psychology, 32(2), 311.

Lee, E., & Leets, L. (2002). Persuasive Storytelling by Hate Groups Online. American

Behavioral Scientist, 45(6), 927 –957. doi:10.1177/0002764202045006003

Mack, N. K. (1990). Learning fractions with understanding: Building on informal

knowledge. Journal for research in mathematics education, 21(1), 16–32.

Mackey, B. (2010, November 8). Launch Failure: The Most Explosively Awful

Console Launch Titles from 1UP.com. 1Up.com. Retrieved May 29, 2012,

from http://www.1up.com/features/launch-failure

Malone, T. W. (1981). Towards a Theory of Intrinsic Motivation. Cognitive Science,

4, 333.

Malone, T. W., & Lepper, M. (1987). Making Learning fun; A taxonomy of intrinsic

motivations for learning. In R. E. Snow & M. J. Farr (Eds.), (p. 223).

Hillsdale, NJ: Lawrence Erlbaum Associates.

Mandler, J. M., & Johnson, N. S. (1977). Remembrance of things parsed: Story

structure and recall. Cognitive Psychology, 9(1), 111–151. doi:10.1016/0010-

0285(77)90006-8

143

Martin, T., & Schwartz, D. L. (2005). Physically distributed learning: Adapting and

reinterpreting physical environments in the development of fraction concepts.

Cognitive Science, 29(4), 587–625.

McDaniel, M. A., Hines, R. J., Waddill, P. J., & Einstein, G. O. (1994). What makes

folk tales unique: content familiarity, causal structure, scripts, or

superstructures? Journal of Experimental Psychology: Learning, Memory, and

Cognition, 20(1), 169.

Mills, J., & Crowley, R. (2001). Therapeutic Metaphors for Children and the Child

Within (1st ed.). Routledge.

Mischel, W., Shoda, Y., & Rodriguez, M. (1989). Delay of gratification in children.

Science, 244(4907), 933 –938. doi:10.1126/science.2658056

Naone, E. (2005, July 12). Virtual Labor Lost: The failure of a highly anticipated,

multiplayer game shows the limits of academic virtual worlds. Technology

Review Published by MIT. Retrieved from

http://www.technologyreview.com/web/19817/page1/

Nariman, H. N. (1993). Soap Operas for Social Change: Toward a Methodology for

Entertainment-Education Television. Praeger Publishers.

Nasir, N. S. (2005). Individual Cognitive Structuring and the Sociocultural Context:

Strategy Shifts in the Game of Dominoes. Journal of the Learning Sciences,

14(1), 5. doi:10.1207/s15327809jls1401_2

National Governors Association Center for Best Practices, Council of Chief State

School Officers. (2010). Common Core State Standards Mathematics. National

144

Governors Association Center for Best Practices, Council of Chief State

School Officers, Washington D.C.

Nelson, B., Ketelhut, D. J., Clarke, J., Bowman, C., & Dede, C. (2005). Design-based

research strategies for developing a scientific inquiry curriculum in a multi-

user virtual environment. Educational Technology, 45(1), 21–27.

Nunes, T. (2006). Fractions: difficult but crucial in mathematics learning. Teaching

and Learning Research Programme (TLRP) Research Briefing. Retrieved from

www.tlrp.org/pub/documents/no13_nunes.pdf

Okita, S. Y., Bailenson, J., & Schwartz, D. L. (2007). The mere belief of social

interaction improves learning. In Proceedings of the Twenty-ninth Meeting of

the Cognitive Science Society.

Okita, S. Y., Bailenson, J., & Schwartz, D. L. (2008). Mere belief in social action

improves complex learning. In Proceedings of the 8th international conference

on International conference for the learning sciences-Volume 2 (pp. 132–139).

Orton, R. E., Post, T. R., Behr, M. J., Cramer, K., Harel, G., & Lesh, R. (1995).

Logical and Psychological Aspects of Rational Number Pedagogical

Reasoning. Hiroshima Journal of Mathematics Education, 3, 63–75.

Paris, S. G., & Turner, J. C. (1994). Situated motivation. In P. R. Pintrich, D. R.

Brown, & C. E. Weinstein (Eds.), Student motivation, cognition, and learning:

Essays in honor of Wilbert J. McKeachie (pp. 213–237). Hillsdale, NJ,

England: Lawrence Erlbaum Associates, Inc.

145

Parker, L. E., & Lepper, M. (1992). Effects of fantasy contexts on children’s learning

and motivation: making learning more fun. Journal of Personality and Social

Psychology, 62(4), 625.

Parlett, D. (1991). A History of Card Games. Oxford University Press, USA.

Payne, J. (1976). Review of research on fractions. In R. Lesh & D. Bradford (Eds.),

Number and measurement: Papers from a research workshop (pp. 145–187).

Columbus, Ohio: Information Resource Center.

Pellegrini, A. D., & Galda, L. (1982). The Effects of Thematic-Fantasy Play Training

on the Development of Children’s Story Comprehension. American

Educational Research Journal, 19(3), 443 –452.

doi:10.3102/00028312019003443

Petit, M. M., Laird, R. E., & Marsden, E. L. (2010). A focus on fractions: Bringing

research to the classroom. Routledge.

Petty, R. E., & Cacioppo, J. T. (1986). The Elaboration Likelihood Model of

Persuasion. In L. Berkowitz (Ed.), Advances in Experimental Social

Psychology (pp. 123–205). Academic Press.

Piaget, J. (1952). The Origins of Intelligence in Children. New York, NY, USA:

International Universities Press.

Piaget, J. (1964). Part I: Cognitive development in children: Piaget development and

learning. Journal of Research in Science Teaching, 2(3), 176–186.

doi:10.1002/tea.3660020306

146

Plass, J. L., Underwood, J., Homer, B. D., Goldman, R., Frye, J., Hendee, C., …

Perlin, K. (2010). Defining and Measuring Engagement in Educational Games.

Presented at the Games Learning and Society Conference, Madison, WI.

Putnam, R. T., Lampert, M., & Peterson, P. L. (1990). Alternative Perspectives on

Knowing Mathematics in Elementary Schools. Review of Research in

Education, 16, 57–150. doi:10.2307/1167351

Read, J., MacFarlane, S., & Casey, C. (2002). Endurability, engagement and

expectations: Measuring children’s fun. In Interaction Design and Children

(pp. 189–198).

Reeves, B., & Read, J. L. (2009). Total engagement: using games and virtual worlds

to change the way people work and businesses compete. Harvard Business

School Press.

Regan, D. T., & Fazio, R. (1977). On the consistency between attitudes and behavior:

Look to the method of attitude formation. Journal of Experimental Social

Psychology, 13(1), 28–45.

Reinard, J. C. (1988). The Empirical Study of the Persuasive Effects of Evidence The

Status After Fifty Years of Research. Human Communication Research, 15(1),

3–59. doi:10.1111/j.1468-2958.1988.tb00170.x

Reynolds, G. (2008). Presentation zen: Simple ideas on presentation design and

delivery. New Riders.

Rick, J., Francois, P., Fields, B., Fleck, R., Yuill, N., & Carr, A. (2010). Lo-fi

prototyping to design interactive-tabletop applications for children. In

147

Proceedings of the 9th International Conference on Interaction Design and

Children (pp. 138–146).

Roschelle, J. (1987). The Envisioning Machine: Facilitating students’

reconceptualization of motion. Third International Conference on AI and

Education, Pittsburgh, PA.

Roschelle, J., & Teasley, S. D. (1994). The construction of shared knowledge in

collaborative problem solving. NATO ASI SERIES F COMPUTER AND

SYSTEMS SCIENCES, 128, 69–69.

Rumelhart, E. (1975). Notes on a schema for stories. In Representation and

Understanding: Studies in Cognitive Science. Academic Press Inc.

Saltz, E., Dixon, D., & Johnson, J. (1977). Training Disadvantaged Preschoolers on

Various Fantasy Activities: Effects on Cognitive Functioning and Impulse

Control. Child Development, 48(2), 367–380. doi:10.2307/1128629

Sansone, C., Weir, C., Harpster, L., & Morgan, C. (1992). Once a boring task always a

boring task? Interest as a self-regulatory mechanism. Journal of Personality

and Social Psychology, 63(3), 379.

Sarama, J., & Clements, D. (2002). Building Blocks For Young Children’s

Mathematical Development. Journal of Educational Computing Research,

27(1), 93–110.

Schank, R. C., & Abelson, R. P. (1975). Scripts, plans, and knowledge. In

Proceedings of the 4th international joint conference on Artificial intelligence-

Volume 1 (pp. 151–157).

148

Schwartz, D. L., & Arena, D. (2009). Choice-based assessments for the digital age.

MacArthur 21st Century Learning and Assessment Project.

Sefelin, R., Tscheligi, M., & Giller, V. (2003). Paper prototyping-what is it good for?:

a comparison of paper-and computer-based low-fidelity prototyping. In

CHI’03 extended abstracts on Human factors in computing systems (pp. 778–

779).

Sessler, A., Webb, M., & Herter, B. (2009, December 23). Discussion: The Rise of

Co-Op Gaming. G4tv.com. Retrieved from

http://www.g4tv.com/videos/43319/discussion-the-rise-of-co-op-gaming/

Shaffer, D. W. (2006). Epistemic frames for epistemic games. Computers &

Education, 46(3), 223–234.

Sherif, C. W., Sherif, M., & Nebergall, R. E. (1981). Attitude and attitude change: The

social judgment-involvement approach. Westport, Connecticut: Greenwood

Press.

Shute, V. J. (2011). Stealth assessment in computer-based games to support learning.

Computer games and instruction, 55(2), 503–524.

Simmons, A. (2006). The story factor. Perseus Books.

Singhal, A., Obregon, R., & Rogers, E. M. (1995). Reconstructing the story of

Simplemente Maria, the most popular telenovela in Latin America of all time.

International Communication Gazette, 54(1), 1 –15.

doi:10.1177/001654929505400101

149

Singhal, A., & Rogers, E. M. (2002). A Theoretical Agenda for Entertainment—

Education. Communication Theory, 12(2), 117–135. doi:10.1111/j.1468-

2885.2002.tb00262.x

Slater, M. D., & Rouner, D. (2002). Entertainment—Education and Elaboration

Likelihood: Understanding the Processing of Narrative Persuasion.

Communication Theory, 12(2), 173–191. doi:10.1111/j.1468-

2885.2002.tb00265.x

Slavin, R. E. (1980). Cooperative Learning. Review of Educational Research, 50(2),

315–342. doi:10.3102/00346543050002315

Smith, G. G., & Celano, M. (2000). Revenge of the mutant cockroach: Culturally

adapted storytelling in the treatment of a low-income African American boy.

Cultural Diversity and Ethnic Minority Psychology;Cultural Diversity and

Ethnic Minority Psychology, 6(2), 220–227. doi:10.1037/1099-9809.6.2.220

Son, J. Y., & Goldstone, R. L. (2009). Contextualization in Perspective. Cognition and

Instruction, 27(1), 51–89. doi:10.1080/07370000802584539

Squire, K., & Klopfer, E. (2007). Augmented Reality Simulations on Handheld

Computers. The Journal of the Learning Sciences, 16(3), 371–413.

Staffin Metz, S. (2007). Attracting the engineers of 2020 today. In Ronald J. Burke &

Mary C. Mattis (Eds.), Women and Minorities in Science, Technology,

Engineering and Mathematics (p. 184). Northampton, MA: Edward Elgar

Publishing. Retrieved from

http://adsabs.harvard.edu/abs/2007wmst.book..184S

150

Steinkuehler, C. (2006). Massively Multiplayer Online Video Gaming as Participation

in a Discourse. Mind, Culture, and Activity, 13(1), 38–52.

Stevens, R., Satwicz, T., & McCarthy, L. (2007). In-Game, In-Room, In-World:

Reconnecting Video Game Play to the Rest of Kids’ Lives. In K. Salen (Ed.),

The Ecology of Games: Connecting Youth, Games, and Learning (pp. 41–66).

Strange, J. J., & Leung, C. C. (1999). How Anecdotal Accounts in News and in

Fiction Can Influence Judgments of a Social Problem’s Urgency, Causes, and

Cures. Personality and Social Psychology Bulletin, 25(4), 436 –449.

doi:10.1177/0146167299025004004

Strickland, L. H., & Grote, F. W. (1967). Temporal presentation of winning symbols

and slot-machine playing. Journal of Experimental Psychology, 74(1), 10.

Sunwolf. (1999). The pedagogical and persuasive effects of Native American lesson

stories, Sufi wisdom tales, and African dilemma tales. Howard Journal of

Communication, 10(1), 47–71.

Sunwolf & Frey, L. (2001). Storytelling: The power of narrative communication and

interpretation. The new handbook of language and social psychology, 15, 119–

135.

Taylor, S. E., & Thompson, S. C. (1982). Stalking the elusive “vividness” effect.

Psychological Review, 89(2), 155–181. doi:10.1037/0033-295X.89.2.155

Thorndyke, P. W. (1977). Cognitive structures in comprehension and memory of

narrative discourse. Cognitive Psychology, 9(1), 77–110. doi:10.1016/0010-

0285(77)90005-6

151

Truby, J. (2007). The Anatomy of Story: 22 Steps to Becoming a Master Storyteller

(1st ed.). Faber & Faber.

Vasquez, V. (2003). What Pokemon Can Teach Us about Learning and Literacy.

Language Arts, 81(2), 118–25.

Watts, W. A. (1967). Relative persistence of opinion change induced by active

compared to passive participation. Journal of Personality and Social

Psychology, 5(1), 4–15. doi:10.1037/h0021196

Weiner, B. (1985). An attributional theory of achievement motivation and emotion.

Psychological Review, 92(4), 548–573. doi:10.1037/0033-295X.92.4.548

Wilensky, U. (2002). Modeling nature’s emergent patterns with multi-agent

languages. In Proceedings of EuroLogo (p. 1–6). Retrieved from

http://ccl.northwestern.edu/papers/MEE/

Williamson, P., & Silvern, S. (1991). Thematic-fantasy play and story comprehension.

Play and early literacy development, 69–90.

Wilson, P. H., Myers, M., Edgington, C., & Confrey, J. (2012). Fair Shares, Matey, or

Walk the Plank. Teaching Children Mathematics, 18(8), 482–489.

doi:10.5951/teacchilmath.18.8.0482

Winterbottom, A., Bekker, H. L., Conner, M., & Mooney, A. (2008). Does narrative

information bias individual’s decision making? A systematic review. Social

Science & Medicine, 67(12), 2079–2088. doi:10.1016/j.socscimed.2008.09.037

Wood, R. T. A., Griffiths, M. D., Chappell, D., & Davies, M. N. O. (2004). The

Structural Characteristics of Video Games: A Psycho-Structural Analysis.

CyberPsychology & Behavior, 7(1), 1–10. doi:10.1089/109493104322820057

152

Yueh, J., & Alessi, S. M. (1988). The effect of reward structure and group ability

composition on cooperative computer-assisted instruction. Journal of

Computer-Based Instruction, 15(1), 18–22.

153

Appendix A – Early Tug-of-War Fractions Assessment

154

155

156

157

158

Appendix B – 2012 Fractions Assessment

159

160

161

162

163

Appendix C – 2012 Additional Post-Test Items

164

165

166

Appendix D – Protocol for Interview Study

Part 1 - Introduce yourself “Ok, so I’m going to ask you a lot of questions, and don’t feel like you’re not answering well enough, because I’m asking you more, it’s just that I want to understand as much as I can and I want to get you to talk more, ok?”

Part 2 - Cartoon part of interview “So first things first, what grade are you going into?”

“Do you have a favorite cartoon show? (Have them elaborate on this, by saying things like…”) if they don’t have a favorite cartoon show, ask, them for a favorite movie or TV show, if they give you jersey shore or something else, just ask for another one until they give you a show that is based on a story.

“Hmm…I don’t know about that show, but it sounds cool! Can you tell me what it’s about?”

“What normally happens on the show?”

Then ask them for a specific example. You can ask them if they have a memorable episode (though they might think the word “episode” is a TV show)

“Wow that sounds really cool! Can you tell me more about it?

Part 3 - Game part of interview (Do the whole game with screenshots and then the other based on what they played first)

“Which game did you play first?”

Do you remember the game? (If they don’t offer to have them play either game for 5 minutes)

“So what’s the game about?”

167

“How do you play this game?”

“How do you win at the game?

“What’s the main thing you’re doing in the game?” (If they give you like to learn or something, try and say “Ok, what else other than learning, think of the game itself”)

“How do you advance (get further) in the game?

“Sometimes a game has a story and sometimes it doesn’t”

“Does the game have a story?”

If “yes”

“Can you tell me what the story is?”

“What are the most important parts of the story?”

If “no”

What does the game need to make it a story?

Can you think of a good story for the game to have?

Then present screenshots with the questions. Keep following up all the questions with, “so?”

“So why does it matter that ______?” (Insert whatever they just said)

“Who are the main folks in the game? What’s their responsibility/what do they do in the game?”

168

“What do you have to do with the game? What’s your role? How do you hurt or help these folks?”

Part 4 – Example Slides with Questions

169

“What is “What aboutis to happen here?” “What’s fractionthis on cardthis doing here?” 1.

170

What does the mean¼ here? How would the game be different was if ½?it

“

2. How does help it you to do the math here?

171

how come that’s important? Or so how comematters? that

Once Ithe answer calculate for this, what happens next? Keep probing with

3. “So

172

doing here?”

e e top

on th

number

“What is “What aboutis to happen here?” “What’s this

4. What’s the 2 doing here? How wouldbe differentit was if 3?it How does help it you to do the math here?

173

hat hat happens next?? Keep

5. nowSo have I pressed the spacebar and the calculated answer, w probing with so how come that’s important or so how comematters? that

174