View metadata, citation and similar papers at core.ac.uk brought to you by CORE

provided by Cork Open Research Archive

Title Gamification as behavioral psychology Author(s) Linehan, Conor; Kirman, Ben; Roche, Bryan Editor(s) Walz, Steffen P. Deterding, Sebastian

Publication date 2015-01 Original citation Linehan, C., Kirman, B. and Roche, B. (2015) 'Gamification as behavioral psychology' in Walz, S.P. and Deterding, S. (eds.) The Gameful World: Approaches, Issues, Applications. Cambridge, MA, USA : MIT Press, pp. 81-105.

Type of publication Book chapter

Link to publisher's https://mitpress.mit.edu/books/gameful-world version Access to the full text of the published version may require a subscription.

Rights © 2014, Massachusetts Institute of Technology. All rights reserved. No part of this chapter may be reproduced in any form by any electronic or mechanical means (including photocopying, recording, or information storage and retrieval) without permission in writing from the publisher.

Item downloaded http://hdl.handle.net/10468/3462 from Downloaded on 2017-02-12T06:33:32Z PROPERTY OF THE MIT PRESS FOR PROOFREADING, INDEXING, AND PROMOTIONAL PURPOSES ONLY

3 GAMIFICATION AS BEHAVIORAL PSYCHOLOGY

Conor Linehan, Ben Kirman, and Bryan Roche

Those who advocate the benefits of a gamified or behavior, and introduce the field of behavioral psy- gameful world often advance a vision of the future in chology as an approach to understanding observed which all life and all work becomes increasingly behavior in gamified products. Behavior analysis is a playful, game-like, and rewarding; a future in which natural science branch of psychology and has been the world’s problems can be fixed by mass collabora- successful in developing principles and procedures tive game-like activities, which simultaneously stim- for engaging users in a wide range of training pro- ulate and delight the participants, while also grams and engendering behavior change, usually in providing useful services to science, charities, and an educational context (see Cooper, Heron, and industry (McGonagle 2011; Schell 2010b). Given the Heward 2006). Of the many fields within psychology, current popularity of game playing as a pastime and behavior analysis has devoted itself to precision in the success of many early examples of gamification the understanding of, and perhaps more importantly (e.g., von Ahn and Dabbish 2004; Khatib et al. 2011), the control of, human behavior. A consideration of it is difficult to not get caught up in this excitement. the principles generated by behavioral psychologists Notably, however, this excitement about gamifica- might be useful in explaining how specific game tion does not appear to have been generated by any design elements motivate and maintain user engage- specific new scientific or technological breakthrough. ment, and knowledge of the principles and processes Indeed, there seems to be very little novel—theoret- defined by behavioral psychologists can readily help ically or practically, from a sociological, psychologi- in the design of more useful and engaging gamified cal, or design perspective—about the gameful design experiences. Given the tremendous strength of of products, services, and activities (Deterding et al. empirical grounding, behavioral psychology is a 2011). proven, valid, useful, and interesting lens through We argue that in order fully to appreciate gamifi- which we can investigate gamification. cation as a design tool, it is necessary to understand To communicate the contribution of this chapter the spectrum of relevant social and psychological as clearly as possible, it is important to first provide processes acting on both the designer and consumer a definition for the phenomenon of gamification. O of such products, services, and activities. In this Deterding et al. (2011) suggest that the unique phe- chapter, we focus on one level of analysis, observed nomenon of interest when discussing gamification is

Walz—The Gameful World PROPERTY OF THE MIT PRESS PROPERTY OF THE MIT PRESS FOR PROOFREADING, INDEXING, AND PROMOTIONAL PURPOSES ONLY FOR PROOFREADING, INDEXING, AND PROMOTIONAL PURPOSES ONLY

82 CONOR LINEHAN, BEN KIRMAN, AND BRYAN ROCHE

“the use of game design elements in non-game con- design patterns) to the abstract (game design texts.” They identify that game design elements are methods). We specifically emphasize how the effects “elements that are found in most (but not necessarily of characteristic game design elements (i.e., points, all) games, readily associated with games, and found badges, leaderboards, time constraints, clear goals, to play a significant role in game play.” Examples of challenge) can be explained through principles of game design elements are provided, which vary in behavior investigated and understood by behavioral terms of abstraction from the concrete (interface psychologists for decades (see Skinner 1974).

A Gameful Life

Arguably, one of the catalysts for the current interest used to help people become better motivated, pro- in gamification was a keynote speech by Jesse Schell ductive, and healthy. Fascinatingly, the argument (2010b). In his talk, he outlined a future where game put forward by Skinner is strikingly similar to that mechanics are totally intertwined with our daily advanced by McGonigal and Schell. lives. Players receive game rewards for brushing It seems that in order to design the type of world their teeth, using public transport, eating certain envisioned by McGonigal and Schell, what is needed branded foods, and so on. In a similar vein, Jane is a deeper understanding not only of games and play McGonigal (2011) argues that through careful use of but also of the processes through which it is possible game design elements, people can become motivated to incentivize people to behave in an appropriate or to solve real-world problems, do more work, and productive manner. We need to understand how to better manage their health (e.g., SuperBetter.com). measure, understand, predict, and control people’s The core idea is that through modifying the environ- behavior. These are exactly the questions that behav- ment and giving suitably motivating rewards, the ioral psychologists have tried to answer through behavior of players can be changed for their own their research. The remaining sections of this chapter benefit (or that of their corporate masters). will introduce and discuss behaviorism as a philo- Notably, in 1948 the behavioral psychologist B. F. sophical approach to understanding the gamified Skinner released a science fiction novel called Walden world. This philosophy is the foundation for the prac- Two, which tells the tale of a utopian community tice of behavior analysis, and some interesting points whose members live together bound by a strict set of of overlap between this philosophy and the assump- rules defining how tasks are completed and rewards tions underlying gamification will be considered. granted in such a way to encourage positive behavior Subsequently, the very effective behavior control change and maximize motivation. For example, principles developed by behavioral psychologists will working less desirable jobs earns more “labour- be reviewed. Many of these can be readily imple- credits,” which means those workers get more free mented in gamified products and services and are time for leisure. Walden Two acts as an argument for already being applied by researchers in the field of O how the principles of behavioral psychology can be persuasive technology (Fogg 2002). We will provide

Walz—The Gameful World PROPERTY OF THE MIT PRESS PROPERTY OF THE MIT PRESS FOR PROOFREADING, INDEXING, AND PROMOTIONAL PURPOSES ONLY FOR PROOFREADING, INDEXING, AND PROMOTIONAL PURPOSES ONLY

GAMIFICATION AS BEHAVIORAL PSYCHOLOGY 83

an analysis of game playing from a behavioral psy- some broad concerns and criticisms often associated chology perspective and will conclude by offering with the behavioral approach.

The History and Philosophy of Behavioral Science

Behaviorism is an approach to psychology that that the individual has cash in his or her pocket, and attempts to understand all behavior, and all psycho- given that the individual has not eaten for some time, logical events, in terms of the interactions of an he or she is likely to put some money in the appropri- organism in and with its environment (Hayes 1993). ate vending machine slot. Put another way, the per- The work of a behavioral psychologist lies in investi- son’s history of reinforcement is coming into contact gating which specific features of the environment with the current environment, and it is this history lead to particular behaviors of interest and in under- combined with the previously established functions standing how to replicate and control those behav- of the various stimuli present (the vending machine, iors through control of the environment (Catania money) that explains the behavior (i.e., predicts and 1998). This approach is steadfast in its adherence to controls it). Of course, explanations of specific behav- environmental explanations of behavior because iors are usually more complex than this, but this these lend themselves most readily to the develop- example serves merely to outline the form that ment of means of controlling behavior (i.e., by behavioral explanations typically take. manipulating the environment appropriately). Importantly, a behavior analyst would not explain Behaviorists eschew explanations of behavior in the behavior of buying candy from a vending machine terms of free will or cognitive activity (e.g., decisions, in terms of the hunger or the intention of the indi- intentions, etc.) because (a) these processes cannot vidual, as we so often do in commonsense reasoning be easily manipulated for the purposes of behavior and in softer branches of psychology (as well as much control, (b) they usually constitute hypothesized of cognitive psychology). That is, it is not acceptably rather than observable processes, and (c) as aspects scientifically rigorous to use, in explanation of of human activity, they must themselves be explained behavior, a hypothesized internal and private state in terms of organism-environment interactions. For (i.e., hunger), the only proof for which is the very instance, we might explain the behavior of interact- behavior it is supposed to explain (i.e., the candy ing with a vending machine in terms of a history of purchase). More specifically, we can control the successful acquisition of candy bars upon the inser- history and state of food deprivation of the individ- tion of cash, as well as in terms of the current physi- ual, but we cannot directly control his or her level of ologic state of the individual engaged in the behavior hunger. We can also control the individual’s history (e.g., the person is food deprived). That is, given the of reinforcement and so increase the individual’s effi- individual’s history of being naturally rewarded (i.e., ciency and frequency of using vending machines, but reinforcement) by the delivery of food for inserting we cannot directly control his or her intentions to do money correctly into a vending machine, and given so. In summary, behavioral psychology is utterly O

Walz—The Gameful World PROPERTY OF THE MIT PRESS PROPERTY OF THE MIT PRESS FOR PROOFREADING, INDEXING, AND PROMOTIONAL PURPOSES ONLY FOR PROOFREADING, INDEXING, AND PROMOTIONAL PURPOSES ONLY

84 CONOR LINEHAN, BEN KIRMAN, AND BRYAN ROCHE

non-esoteric and is supremely pragmatic in its as a behavioral psychologist, except that in the case approach. To this extent, it dovetails well with the of technology, these constraints are technologically purposes of an engineer whose goal is to increase rather than philosophically imposed. Therefore, the user engagement with a product or system, rather tools developed by behavioral psychologists to than merely understand it in hypothetical terms. understand and control behavior are very relevant Notably, the constraints that have led behavioral for anyone using technology to monitor and change psychologists to adopt their unique approach are human behavior. remarkably similar to those operating on any In summary, the insistence on observation in technologically mediated system that attempts to attempting to change or maintain the actual behav- modify human behavior, such as is often the goal of ior of individuals is what is unique and useful about gamified technologies. Specifically, technology is behavioral psychology. We would suggest that it is good at objective measurement, analyzing patterns, also useful to take this approach when designing a and determining solutions based on executable func- game, gamified service, or, indeed, persuasive tech- tions. It is not good at intuitively inferring states of nology (Fogg 2002). Focusing on the actual observed mind, thoughts, feelings, and emotions and effecting behavior of a person, rather than some presumed behavior change outcomes by nonempirical means. inner state or intentions or some other common- Essentially, a computer that is attempting to modify sense-influenced model of behavior, will lead the behavior, such as improving the frequency of reading engineer closer to finding means of maintaining the or exercise, is operating under the same constraints engagement of that person with his or her task.

B. F. Skinner and Radical Behaviorism

The basic science of behavior analysis is heavily tive recording device (Skinner 1959) while the animal indebted to the work of B. F. Skinner (i.e., Skinner interacted with its environment. Numerous such 1953, 1959, 1974) and his contemporaries, who exper- studies were conducted, leading to considerable imentally studied the behavior of animals such as success in defining the now well-understood “prin- rats and pigeons. A typical study by Skinner or his ciples” of behavior, such as operant conditioning. contemporaries involved an animal being placed into Operant conditioning could be described as a set a specially designed box containing a lever and a food of circumstances in which the “consequences of dispenser (often referred to by popular media as a behaviour may ‘feed back’ into the organism, and, “Skinner box”). The experimenter set up a contin- when they do so, they may change the probability gency whereby the delivery of rewards or punish- that the behaviour which produced them will occur ments was dependent upon either a fixed amount of again” (Skinner 1953, 59). For example, a rat may time or some specific response (such as a lever press) engage in many different behaviors while trapped in produced by the animal. An experimenter kept a a cage. If one of these behaviors, such as pressing a O record of the behavior of the animal using a cumula- lever, is followed by a favorable consequence, such as

Walz—The Gameful World PROPERTY OF THE MIT PRESS PROPERTY OF THE MIT PRESS FOR PROOFREADING, INDEXING, AND PROMOTIONAL PURPOSES ONLY FOR PROOFREADING, INDEXING, AND PROMOTIONAL PURPOSES ONLY

GAMIFICATION AS BEHAVIORAL PSYCHOLOGY 85

the delivery of food, the probability of this behavior The game Farmville (Zynga 2009) provides an example occurring in the future will have been altered (in this of how this process can be implemented. In this case, we may expect that lever-pressing will be more game, crops must be harvested within a certain likely in the future). The consequences of behavior period or else they die. The reward for visiting and have a direct and measurable impact upon the likeli- tending to your farm is the removal of impending hood of that behavior occurring again. crop death, a consequence of your negligence. Skinner specified the operant behavioral unit as a Negative punishment describes a situation in which three-term contingency consisting of an antecedent, a the removal or termination of a stimulus as a conse- response, and a consequence. He suggested that this quence of an instance of behavior makes that behav- concept could be used to describe all behaviors of ior less likely to occur. Positive punishment describes organisms and that understanding the determinants a situation in which the presentation or addition of of behavior simply required understanding the spe- a stimulus as a consequence of an instance of behav- cific antecedents and consequences operating on ior makes that behavior less likely to occur in that that organism in that context. In an operant response, context in the future. These aversive contingencies both an antecedent, such as an environmental (situations that people will work to avoid) are used context or stimulus, and a consequence combine to less often in gamified applications because of the fear produce behavior (Skinner 1953, 65). Importantly, that they will lead to disengagement with the product. this definition allowed for a systematic program of Specifically, if the consequence of eating a chocolate research examining various antecedent and conse- cake will be a disapproving message from a phone quent conditions as determinants of behavior. This application, the easiest way of avoiding that feedback program of research led to technical, mathematical is to stop using the application rather than to change definitions of terms such as positive and negative rein- your eating behavior (Kirman et al. 2010). However, forcement, punishment, and avoidance. because games often use aversive consequences as Positive reinforcement describes a situation in which key mechanics in game play, we should expect to see the presentation of a stimulus as a consequence of an the prevalence of such aversive contingencies instance of behavior makes that behavior more likely increasing in gamified products (Foster et al. 2011). to occur in that context in the future. There are To apply reinforcement techniques successfully, it countless examples of positive reinforcement contin- is crucial to understand the difference between a gencies implemented in gamified applications. For “reward” and a reinforcer. A reward is any stimulus example, the rewarding of points, badges, leveling given to a user on the assumption that it will increase up, and access to new features as a consequence of the likelihood of the consequent behavior being appropriate behavior are all examples of this process. repeated in the future. A reinforcer is any stimulus Negative reinforcement describes a situation in that has been observed to increase the likelihood of the which the removal or termination of an existing rewarded behavior being repeated in the future. Cru- stimulus (or existing aversive condition) as a conse- cially, the technical definitions provided earlier were quence of an instance of behavior makes that behav- defined based on careful observation and analysis of ior more likely to occur in that context in the future. the single subjects (i.e., one animal, one person). O

Walz—The Gameful World PROPERTY OF THE MIT PRESS PROPERTY OF THE MIT PRESS FOR PROOFREADING, INDEXING, AND PROMOTIONAL PURPOSES ONLY FOR PROOFREADING, INDEXING, AND PROMOTIONAL PURPOSES ONLY

86 CONOR LINEHAN, BEN KIRMAN, AND BRYAN ROCHE

They do not refer to assumptions or to typical effects The general principles of behavior are at work in across a group of participants. The concept of rein- many gamified products, even where product devel- forcement, therefore, is not theoretical, and the opers are not fully aware of the fact. Indeed, those parameters of the various processes involved are well game design characteristics used to “gamify” prod- understood empirically. Essentially, if we have not ucts can often be described using the concept of rein- carefully observed and measured behavior, we cannot forcement alone. In many cases, there is no need to describe the specific consequences that one should refer to game play at all as an explanation of what a provide as a reinforcer, punisher, and so forth, in “gamified” service does to engage users. For example, order to alter the rate and probability of that behav- there are many popular exercise applications, ior in the future. However, if we have carefully mea- such as Nike+ (http://nikeplus.nike.com/plus) and sured and recorded behavior, as the creators of many Fitocracy (fitocracy.com), which use simple positive persuasive gamified products do (e.g., healthmonth. reinforcement contingencies to encourage exercise com, Nike+), reinforcement techniques can be very through the awarding of points, badges, and progres- effectively applied in the control of behavior rates. sion through levels in exchange for observed activity, Importantly, behavioral psychology is a form of such as completion and regularity of runs and other selectionist analysis, entirely coherent with evolu- workout sessions. In these instances, badges are pre- tionary biology (see Hayes and Long 2013). Evolution- sented as the consequence of observed behavior on ary theory describes how behaviors and traits are the assumption that the “earning” of that badge will selected by the environment across generations. make exercise more likely in the future. While these Behavioral psychology, in contrast, explains how features are commonly seen in games, they have behaviors and traits are selected by the environment little to do with the concept of play. Other systems within the life span of the organism. In both cases, use forms of positive reinforcement to encourage the analytic unit (the species or the behavior) is healthier eating (https://foodzy.com/) and language selected by the consequences of its occurrence. learning (https://www.duolingo.com).

Scheduling Feedback

The use of consequences, whether real (e.g., points Through experimental investigation of operant redeemable for credit) or virtual (e.g., points with conditioning, behavioral psychologists discovered social value in terms of comparing oneself favor- that there are significant temporal and contextual ably with others), is a crucial and central aspect of components that affect how the environment is any program of behavior maintenance. But it is responded to by an organism (Ferster et al. 1957). For not simply a matter of providing encouraging feed- example, the effect that any one stimulus, presented back or points. The real science of behavior analy- as a consequence of behavior (i.e., feedback), will sis lies in the scheduling of these behavior have on subsequent behavior is determined primar- O consequences. ily by the history of that organism encountering that

Walz—The Gameful World PROPERTY OF THE MIT PRESS PROPERTY OF THE MIT PRESS FOR PROOFREADING, INDEXING, AND PROMOTIONAL PURPOSES ONLY FOR PROOFREADING, INDEXING, AND PROMOTIONAL PURPOSES ONLY

GAMIFICATION AS BEHAVIORAL PSYCHOLOGY 87

stimulus previously, rather than any inherent feature VR of the stimulus itself (an exception to this are those stimuli that humans have unconditioned, genetically determined responses to, such as painful stimuli). Essentially, the power of any stimulus to function as FR a reward or punishment changes over time through VI experience (i.e., learning). Skinner and his colleagues

specifically investigated how that process occurred ve number of responses and how to manipulate these factors in order to FI predict accurately and control subsequent behavior.

Because the use of game design elements in non- Cumulati game contexts is typically carried out to encourage, Time provoke, or maintain specific behaviors, these pro- Figure 3.1 cesses are important to understand in the design of Illustration of prototypical behavior observed across four gamified products and services. different schedules of reinforcement. FI, fixed interval; VI, Behavioral psychologists use the term schedule variable interval; FR, fixed ratio; VR, variable ratio. of reinforcement to describe important contextual aspects that define the organism’s experience of schedules are similar to FI schedules, with the excep- reinforcement. Specifically, two variables were iden- tion that the time for which reinforcement is unavail- tified as significant: the interval, or amount of time able oscillates around a mean, rather than being that has passed since the last instance of reinforce- predictable. This schedule results in a steady but ment, and the ratio, or the amount of work that it relatively low rate of response. Fixed ratio (FR) sched- takes to earn a reinforcer (Ferster et al. 1957). ules deliver reinforcement after every nth response. Researchers found that varying either of these had For example, FR5 schedules provide reinforcement significant impact on behavior (figure 3.1). Four dif- consistently after every fifth response. This schedule ferent configurations produce different patterns of produces a high, steady rate of responding with a responses in animals engaging with a lever that can brief pause after the delivery of the reinforcer. Vari- be pressed to earn food pellets. able ratio (VR) schedules are similar to FR schedules, A fixed interval (FI) is a schedule in which only the with the exception that rather than being predict- first response after a specified amount of time has able, the number of responses required for reinforce- elapsed is rewarded, while premature responses are ment oscillates around a mean. This type of schedule not reinforced at all. This schedule results in a pattern creates a high and steady rate of responding and is of behavior in which most behavior occurs in the typically the most economical; a lot of work can be minutes before reinforcement is expected and behav- generated by few instances of reinforcement. ior rates reduce rapidly immediately afterward, until Variable ratios have been the source of much the end of the interval. Overall behavioral engage- research and controversy. Because the work pro- O ment under FI schedules is low. Variable interval (VI) duced in response to a VR schedule is so out of

Walz—The Gameful World PROPERTY OF THE MIT PRESS PROPERTY OF THE MIT PRESS FOR PROOFREADING, INDEXING, AND PROMOTIONAL PURPOSES ONLY FOR PROOFREADING, INDEXING, AND PROMOTIONAL PURPOSES ONLY

88 CONOR LINEHAN, BEN KIRMAN, AND BRYAN ROCHE

proportion to the rewards offered, implementing this skills, and leveling up. As the player progresses and schedule in a game, gamified service, or work envi- spends more time playing the game, the number of ronment can be seen as exploitative. Indeed, work actions needed to produce a reinforcer is increased. practices in which pay can be varied at will by The technique of schedule leaning is evident in employers (e.g., piecework) are very much frowned online social network games (Deterding et al. 2010) upon if not illegal in some jurisdictions. VR schedules and particularly in massively multiplayer online have been advanced as an explanation for addiction role-playing games, such as World of Warcraft to gambling, as both demonstrate evidence of unre- (Blizzard Entertainment 2004). For instance, let us alistic expectations of reinforcement for the actions consider the archetypal example of the popular role- taken (Haw 2008; King, Delfabbro, and Griffiths 2010). playing game Dungeons and Dragons (Gygax and Indeed, Karlsen (2011) has extended this analysis to Arneson 2000). In this game, players gain “experi- explain addiction in massively multiplayer online ence points” (XPs) through play. After gaining a games. Needless to say, the implementation of VR certain number of XPs, a character moves up a “level” schedules in gamified services, while an extremely and gains additional strengths and abilities. However, effective strategy for motivating engagement, will the number of XPs required to level up increases with certainly draw criticism on grounds of exploitation. each level that is completed, progressively increasing The research conducted on schedules of reinforce- the time taken to complete each subsequent level ment demonstrates that it is not usually optimal to (the completion of levels and leveling up is presented offer a reward or punishment after every action that as a reward in such games; table 3.1). Given the prev- a user takes. Rather, in order to sustain behavior over alence of this technique in games across media, we a period of time, it is necessary to manipulate either should consider it as a characteristic game design the number of responses required or the time elapsed element that has itself been selected by its conse- before reinforcement is delivered (Ferster et al. quences for game and other product developers, but 1957). Different schedules are appropriate in differ- ent contexts, depending on the type of behavior one Table 3.1 wishes to engender in the user. Indeed, the literature Learning schedule of reinforcement in terms of character (see Catania 1998; Cooper, Heron, and Heward 2006) levels in Dungeons and Dragons (third edition) suggests that if people have a consistent history of being reinforced for their efforts, the workload Character Level Experience Points (XPs) Required required to reach those same rewards can be 1 0 increased gradually over time without losing the 2 1,000 motivational effects of those rewards. Behavioral 3 3,000 psychologists refer to this technique of progressively 4 6,000 spacing out the delivery of reinforcement as schedule leaning. This technique is also observed in computer 5 10,000 games, in which the first few tasks that a player com- Source: Gygax and Arneson (2000). O pletes are often reinforced through new items, new

Walz—The Gameful World PROPERTY OF THE MIT PRESS PROPERTY OF THE MIT PRESS FOR PROOFREADING, INDEXING, AND PROMOTIONAL PURPOSES ONLY FOR PROOFREADING, INDEXING, AND PROMOTIONAL PURPOSES ONLY

GAMIFICATION AS BEHAVIORAL PSYCHOLOGY 89

in the absence of a dedicated behavioral analysis to surprising that concerns have been raised about the guide such developments. application of these techniques to non-game contexts Notably, commentators (e.g., Bartle 2011) have (i.e., Bogost 2011a). It is important to remember, criticized the use of techniques such as variable rein- however, that these techniques have been used suc- forcement ratios and schedule leaning in games, sug- cessfully in special education for decades and have gesting that they are in fact so effective when used helped to transform the lives of countless individuals properly that they are exploitative of users. Critics suffering with developmental delay and other behav- suggest that for some players, the schedules even ioral problems (see Cooper et al. 2006; Rehfeldt and remain effective long after the player has ceased Barnes-Holmes 2009). Token economies (a specially having fun or “playing.” The player is then seen as designed context where appropriate behavior is engaging in a repetitive, monotonous, menial task, rewarded through earning of tokens that can be saved analogous to a low-wage job, rather than a fulfilling, up and exchanged for preferred items) provide a par- challenging experience. Essentially, game research- ticularly clear example of how topographically game- ers suggest that the use of these game design ele- like behavioral interventions can have profoundly ments equates to lazy or uninspired game design, and positive effects on behavior in even the most chal- that good games should maintain engagement and lenging environments (Corrigan 1995). Regardless of motivation through the provision of inherently whether you consider these game design elements interesting experiences (Bartle 2011). as useful tools or potentially exploitative practices, Given that game researchers are increasingly knowledge of the effectiveness of these techniques, as uncomfortable with the use of these game design ele- well as the controversies around their use, is essential ments within self-contained games, it should not be for the designers of gameful experiences.

Evaluating the Effectiveness of Feedback

To apply schedules of reinforcement successfully, it whether the feedback they offer produces changes in is crucial that we remain mindful of the difference the target behavior (i.e., learning), and consequences between a reward and a reinforcer. As stated earlier, are systematically modified “online” in order to a reward is any stimulus (points, badges, etc.) given achieve the desired behavior rate. to a user on the assumption that it will increase the Especially in applied contexts (see Cooper et al. likelihood of the rewarded behavior being repeated 2006), many different types of rewards are offered, in the future. However, simply providing people with and the psychologist must analyze data to under- rewards is of little value unless there is a check to see stand better whether consistent patterns of behavior whether subsequent behavior has changed as a con- are observed after each reward is presented. This sequence (if not, then the reward was not a good process is necessary as there are very few (if any) reinforcer). In both experimental and applied set- stimuli that function as a reinforcer or punisher for tings, behavioral psychologists continually test all people at all times. For example, some people find O

Walz—The Gameful World PROPERTY OF THE MIT PRESS PROPERTY OF THE MIT PRESS FOR PROOFREADING, INDEXING, AND PROMOTIONAL PURPOSES ONLY FOR PROOFREADING, INDEXING, AND PROMOTIONAL PURPOSES ONLY

90 CONOR LINEHAN, BEN KIRMAN, AND BRYAN ROCHE

listening to classical music to be the highlight of their The clear implication of an analysis of reinforce- week, while many find it boring. Delicacies such as ment effectiveness for gamified products is that caviar, kokoretsi (organ meat), oysters, and Marmite there should be different types of rewards available are often seen as repulsive to different palates to users, and the application should include some (Kirman et al. 2010). Thus, it is necessary to evaluate simple way of evaluating which rewards are most the impact of different rewards in order to evaluate reinforcing for each user. Behavioral psychologists whether those rewards are ones that the person, as have developed precise methods for doing exactly an individual, is motivated to obtain. If the targeted that, and it appears that these techniques may behavior does increase as a consequence of the deliv- be ideal for use in technology such as computer ery of a particular reward, then that stimulus can be games or gamified applications. For example, Her- classified as a reinforcer in that context. The identi- rnstein’s matching law (Herrnstein 1961) is a math- fied reinforcer can then be used in the future as a ematical way of determining which contingencies consequence of behavior that the psychologist wants an organism finds most rewarding when multiple to reinforce. The same process is applicable in the options are available. Herrnstein found, in experi- identification of punishers. In the context of a game, ments with animals, that the amount of time and for example, a behavior analyst would offer a wide work that was devoted to each of the options was range of rewards for effective behavior, initially consistent with the rate at which that work was emitted at low rates on a rich schedule (e.g., an FR1), rewarded. Essentially, the matching law is a math- and then “lean” the schedule in tandem with a nar- ematical way of determining which contingencies rowing of the range of rewards being provided, all that individual organism found most rewarding. the while removing those rewards that do not func- Understanding the matching law can help game tion as reinforcers. designers create uniquely adaptive and engaging The above procedure is clearly applicable to both games or gamified products and services, as Her- stand-alone games and non-game contexts alike. rnstein’s algorithm allows us to monitor the rela- Taking an example from computer games, Ultima tive attractiveness of each of the various reinforcers Online (Origin Systems 1997) provides many possible on offer with a given game. Through continually types of behavior for the user to engage with, from monitoring a player’s behavior, a system can auto- crafting to exploring and fighting, all of which matically calculate which rewards are eliciting the provide the possibility of advancing within the game. most work from that person (i.e., which are most The completion of a masterwork piece of armor by a reinforcing). player who enjoys crafting is rewarded by the game In an application such as foursquare (foursquare. in the same way as defeating a dragon (i.e., through com), for example, through monitoring user behav- gained skill points). As a result, the different histories ior, the application could easily identify that a given of individual players are catered to, and a wide user is twice as likely to check in at further locations variety of different individuals can experience the that day after receiving a large check-in bonus than same reinforcing consequences from the same game, after receiving a badge. Using the matching law, the O even though their behaviors differed markedly. application could deduce that check-in bonuses are

Walz—The Gameful World PROPERTY OF THE MIT PRESS PROPERTY OF THE MIT PRESS FOR PROOFREADING, INDEXING, AND PROMOTIONAL PURPOSES ONLY FOR PROOFREADING, INDEXING, AND PROMOTIONAL PURPOSES ONLY

GAMIFICATION AS BEHAVIORAL PSYCHOLOGY 91

twice as reinforcing for that user as badges. Thus, strength of each of the available game rewards for using this simple strategy, it is possible for a gamified each individual player and make adjustments to its application to evaluate dynamically the reinforcing own reward system to exploit these data.

Beyond the Skinner Box

The Skinner box apparatus is one of the most fre- What is genuinely useful is the insistence that the quently misrepresented aspects of discussions on observation of behavior is key in understanding, pre- games and gamification. The Skinner box was a dicting, and controlling it. Notably, Bogost (2011b) simple apparatus designed to observe and measure wrote: the behavior of animals (we should not expect it to Game mechanics are the operational parts of provide entertainment for humans). However, this games that produce an experience of interest, does not mean that the behavioral processes that enlightenment, terror, fascination, hope, or any behavioral psychologists discovered through this number of other sensations. Points and levels and apparatus are not generalizable to more complex the like are mere gestures that provide structure and measure progress within such a system. behavior. For example, a reinforcer was defined experimentally as any consequence that improves If the contribution of behavioral psychology to our the likelihood of a behavior being repeated. This can understanding of gamification is that it demonstrates be anything from a drop of sugar solution to a more how to provide structure and context for behavior, complex consequence such as the resolution of a par- to observe, measure, and incentivize progress (i.e., ticularly surprising or opaque narrative arc, a par- learning), then it provides useful explanatory power ticularly “juicy” cut scene, or other in-game event. for the phenomenon of gamification.

Understanding Game Playing

There are many ways of defining and analyzing game The behavior of computer game playing was first playing—in terms of physiology (Nacke, Grimshaw subjected to a basic behavior analysis in book-length and Lindley 2010), social behavior (Kirman and Lawson format by Loftus and Loftus (1983). The authors pro- 2009), immersion or flow (Ijsselsteijn et al. 2007), uses posed a technical account of the type of game playing and gratifications (Sherry et al. 2006), and many other seen in early 1980s video games, using basic experi- methodologies. The current analysis does not reject mentally defined behavioral psychology principles or overlook those definitions. Rather, an alternative is such as operant conditioning, extinction (the reduc- advanced as a means of explaining how the often- tion in behavior rates induced by the removal of complex behavior observed in game playing can be reinforcement), and schedules of reinforcement understood in the context of the experimental find- (described earlier). The authors drew a direct com- ings of behavioral psychology research. parison between a person playing the popular arcade O

Walz—The Gameful World PROPERTY OF THE MIT PRESS PROPERTY OF THE MIT PRESS FOR PROOFREADING, INDEXING, AND PROMOTIONAL PURPOSES ONLY FOR PROOFREADING, INDEXING, AND PROMOTIONAL PURPOSES ONLY

92 CONOR LINEHAN, BEN KIRMAN, AND BRYAN ROCHE

game Pac-Man (Namco 1980) and a rat in one of B. F. more probable, usually by changing the environ- Skinner’s classic experiments (a comparison repeated ment. (Skinner 1974, 123) much since). Loftus and Loftus’s account is both rig- Skinner suggested that, contrary to appearances, the orous and interesting and appeared valid at the time, behavior that solves a novel problem is not a brand when success at contemporary games was based pri- new behavior or insight, but is simply a novel marily on reaction times and reflexes. Of course, arrangement of already established behaviors (i.e., modern games provide more complex and interest- “taking steps to make that response more probable”; ing challenges to players than those analyzed by Skinner 1974, 123). For instance, a child who has been Loftus and Loftus. However, similar low-level behav- taught to pull a chain on the ceiling to flush a toilet, ioral processes may provide some insight into players’ and also to climb on a step to reach objects, may one engagement with some aspects of more complex day climb on a step to pull a chain that is out of reach. games. For example, the variable difficulty levels This appears to be a form of insight, but it might be available in most modern games may be seen as a better described non-mentalistically as response method of adapting the schedules of reinforcement chaining—the mere coming together of previously inherent in a game in order to produce the most established behavioral units. Skinner contends that game-playing behavior in the user. such a process could be applied to understanding Modern games often involve problem solving in many types of real-world problem solving, and we addition to, or in place of, fluid stereotyped responses suggest that this includes those observed in many to a limited number of stimuli. For example, popular forms of game play, although more complex forms of (4X) strategy games such as Civilization V (Firaxis problem solving are now understood to be possible Games 2010) and Eclipse (Tahkokallio 2011) require a (see later). player not only to fight battles with multiple units of Notably, Gingold (2005), in explaining the appeal different characteristics but also to build economies, of the game Wario Ware (Nintendo 2003), appeals to military bases, towns, cities, and empires. Many valid a form of response chaining, without naming it such. strategies can be adopted for pursuing such goals. B. Specifically, Wario Ware consists of a large number F. Skinner attempted to explain precisely this type of of simple minigames that last approximately five problem solving using the principles of behavioral seconds each, grouped according to theme. The psychology. In his book Science and Human Behavior, player must quickly learn the rules of each minigame Skinner (1953) describes how even the complex to progress. At the end of each level, the skills learned behavior of problem solving could be explained in in the preceding minigames must be combined in terms of basic behavioral principles. Consider the fol- order to pass a more complex game (i.e., chaining of lowing passage: previously learned simple behaviors). Gingold pro- A person has a problem when some condition will poses that the process of gradually learning simple be reinforcing but he lacks a response that will behaviors and combining these as the game pro- produce it. … solving a problem is, however, more gresses explains the appeal of the game. Indeed, this than emitting the response which is the solution; explanation could also apply to the fascinating and O it is a matter of taking steps to make that response

Walz—The Gameful World PROPERTY OF THE MIT PRESS PROPERTY OF THE MIT PRESS FOR PROOFREADING, INDEXING, AND PROMOTIONAL PURPOSES ONLY FOR PROOFREADING, INDEXING, AND PROMOTIONAL PURPOSES ONLY

GAMIFICATION AS BEHAVIORAL PSYCHOLOGY 93

hugely popular modern puzzle games in the Portal simple requirement that responses be chained into (Valve Corporation 2007) series. The structure of long behavioral units provides some explanatory these and may other puzzle games, such as World of power for the engaging structural properties of Goo (2D Boy 2008), seems optimized to take advan- simple games. However, as we will now see, game tage of the human capacity for problem solving in complexity may also extend to include more recently terms of response chaining, which, in the field of analyzed forms of problem solving than mere game design, has been referred to as scaffolding. The response chaining.

Complexity and Challenge in Games

Understanding, investigating, and manipulating the trary stimulus “C” in the presence of stimulus “B” challenge or complexity presented by games is an (where these stimuli were things like randomly area in which behavioral psychology may be particu- chosen Chinese characters), then a number of larly useful. Specifically, appropriate challenge and untrained responses emerged, including choosing complexity are often proposed as an explanation of “C” in the presence of “A” and “A” in the presence of why a given computer game succeeds in maintaining “C.” It has now been shown conclusively that humans player attention and enjoyment across the period of can derive novel stimulus relations between various game play (i.e., Koster 2005). However, there is no indirectly related objects or occurrences in the envi- technical definition of game complexity offered in ronment, and this phenomenon cannot be accounted the literature. Without such a definition, how can we for in terms of mere response chaining. In addition, know which forms of complexity are most reinforc- this single psychological ability is considered to be a ing for an individual or along which parameters to foundational unit for all human reasoning and logical alter such complexity? In one recent paper, a behav- thought (see Dymond and Roche, 2013). ioral definition of complexity was offered in terms of The concept of derived relations makes both the a concept called derived relational responding (Linehan, understanding of and systematic manipulation of Roche, and Stewart 2010, Linehan 2008). Before we complexity in games more amenable. Specifically, consider the utility of this new definition, we must there are some types of derived relations that have first consider what we mean by the term derived rela- been demonstrated as more complex than others. For tional responding. example, nodal distance (see Fields et al. 1997, 1990; The simplest example of derived relational Arntzen and Holth 1997, 2000) is a means for analyz- responding is a psychological phenomenon called ing the closeness of a relationship between related stimulus equivalence (Sidman 1971; see also Sidman stimuli. Responding appropriately to directly related 1994, 2000). In his research, Murray Sidman showed stimuli is an observably less complex task than that once people had been explicitly taught to choose responding to stimuli that are related through a an arbitrary stimulus “B” in the presence of an arbi- series of nodes. Similarly, responding appropriately trary stimulus “A” and also to choose a third arbi- to stimuli that are the opposite of each other, bigger O

Walz—The Gameful World PROPERTY OF THE MIT PRESS PROPERTY OF THE MIT PRESS FOR PROOFREADING, INDEXING, AND PROMOTIONAL PURPOSES ONLY FOR PROOFREADING, INDEXING, AND PROMOTIONAL PURPOSES ONLY

94 CONOR LINEHAN, BEN KIRMAN, AND BRYAN ROCHE

or smaller than each other, or different to each other atically in a linear and stepwise manner across levels. is a measurably more complex task than responding It also provides a paradigm within which to under- appropriately to stimuli that are the same as each stand the level of challenge presented by currently other (see Hayes, Barnes-Holmes, and Roche [2001] popular games. For an example of an experimental for a book-length analysis and discussion). Adopting game built entirely on the concept of derived rela- the technical nomenclature of relational complexity tions and nodal distance, see Linehan et al. (2010). offers game developers an empirical means of creat- For an example of an online educational program ing appropriate challenge levels in a non-haphazard that uses the relational complexity game element way, based on empirically understood psychological and draws explicitly on reinforcement procedures processes, and of manipulating complexity system- and schedules of reinforcement, see case study 3.1.

Case Study 3.1 RaiseYourIQ

Summary Gameful Design Elements

RaiseYourIQ is a suite of online cognitive training tools • SMART uses explicit reinforcement through audio developed by behavior analysts to improve general cog- and visual feedback, points and badges, and optional nitive functioning. It falls under the general rubric of a updating of current point status on social media brain training system but is offered more as a clinical/ sites, but on a well worked out schedule that is educational tool than primarily as a form of entertain- “leaned” during test stages and during higher levels. ment. At their own convenience, users practice (twenty • To optimize learning rates, points and badges are to thirty minutes several times per week) at a series of awarded for revising previously completed levels, mental challenges, which take the form of deriving but on a diminishing rate (i.e., systematic leaning of relations of increasing complexity across levels of the schedule). training. Each task involves nonsense words, and levels • E-mail reminders are sent to users if a hiatus in of the training consist of blocks of tasks of similar training is observed. relational complexity. Extensive training at such • Progress is tracked constantly in terms of speed and tasks is understood to have wide intellectual benefits. accuracy so that levels may be skipped if challenge The main product offered by RaiseYourIQ is called is too low or stages regressed if challenge is too high. SMART (strengthening mental abilities with relational This optimizes challenge level and therefore engage- training). ment and learning. • The use of multiple response consequences increases Facts and Figures the likelihood of behavior coming under control of one of these (i.e., a reinforcer). As a recent startup, RaiseYourIQ has only been online • A virtual professor provides helpful encouragement since October 2012 but currently has several thousand O on a well worked out schedule that is as lean as registered users. possible.

Walz—The Gameful World PROPERTY OF THE MIT PRESS PROPERTY OF THE MIT PRESS FOR PROOFREADING, INDEXING, AND PROMOTIONAL PURPOSES ONLY FOR PROOFREADING, INDEXING, AND PROMOTIONAL PURPOSES ONLY

GAMIFICATION AS BEHAVIORAL PSYCHOLOGY 95

Case Study 3.1 (continued)

Box Figure 3.1 A screenshot from SMART at RaiseYourIQ.com. The task shown is a one-node complexity task involving two types of relation (same and opposite), with responses consequated initially on a FR1 schedule by a wide range of rewards, including audio and visual feedback as well as points and badges.

Issues Related Cases

Difficulties identifying a sufficiently broad range of An increasing number of products claiming to improve reinforcers for a wide range of users online have been general mental ability are available online. Many of noted. Innovative solutions are being sought to broaden these are simply games that should in principle help this range. stimulate brain activity and neurogenesis (growth of brain cell connections). These other cases use common- Outcomes sense game elements and are devised by game develop- ers rather than psychologists, even where the core Increases in general intelligence have been reported in purpose of the game was inspired by psychological published studies (e.g., Cassidy, Roche & Hayes 2011). theory, such as the concept of neurogenesis. The market The creators have also claimed anecdotal evidence of leader in this regard is Lumosity (lumosity.com). improvements in reading and vocabulary as well as documented maintenance of IQ increases across four Further Information years (Roche, Cassidy, and Stewart in press). This is unique among brain training products but not surpris- http://RaiseYourIQ.com ing given the uniqueness of the relational training O approach and its foundation in behavior analysis.

Walz—The Gameful World PROPERTY OF THE MIT PRESS PROPERTY OF THE MIT PRESS FOR PROOFREADING, INDEXING, AND PROMOTIONAL PURPOSES ONLY FOR PROOFREADING, INDEXING, AND PROMOTIONAL PURPOSES ONLY

96 CONOR LINEHAN, BEN KIRMAN, AND BRYAN ROCHE

Applied Behavior Analysis and Behavior Modification

At the beginning of this chapter, we pointed out that (though not necessarily) delivered in an intensive the use of game design elements as a means of engen- one-to-one manner. dering engagement in non-game contexts often ABA programs are designed on the assumption involves implementation of processes such as highly that learning is maximized when high-performance structured behavior measurement, algorithmic anal- targets are set and teaching is focused on the indi- ysis of behavior, feedback loops, and reward mecha- vidual. Indeed, unlike in traditional education, the nisms. This is especially apparent where the intention passing criterion in behavioral education is not 40 of the application is to change explicitly the behavior percent, but typically somewhere around 90 percent. of the user (e.g., healthmonth.com). It is also the If the learner does not reach this stringent passing focus of much work carried out in the field of persua- criterion, he or she is required to repeat the program sive technology design (Fogg 2002). Notably, these until the criterion is reached. This process will be processes (measurement of behavior, analysis, and familiar to any player familiar with use of “boss feedback) are also the fundamental building blocks fight” mechanics as a way of testing learned in-game of behavioral interventions—also referred to as skills. behavior modification, or, more recently, applied Indeed, ABA programs have structures that resem- behavior analysis (Cooper et al. 2006). Applied behav- ble characteristic elements of computer games in ioral psychologists have conducted a wealth of many striking ways (for an in-depth discussion, see research on the optimal means for implementing Linehan et al. 2011). For example, highly engaging these processes in order to motivate engagement and games usually share with ABA interventions clearly behavioral change. Thus, some knowledge of this specified and measurable goals (such as to complete field of research may be useful for those designers a section of game or level up the character), require attempting to gamify their products or services. a great deal of repetition of skills in order to reach Applied behavior analysis (ABA) is an umbrella that goal (fighting numerous similar enemies), are term for a range of behavioral interventions that often conducted under time constraints, have clearly build upon the principles discovered by experimental specified rewards for reaching the specified goal behavioral psychology. These have been used to (stronger player/more weapons/access to new treat a huge variety of behavioral problems from levels), and provide consistent feedback from the developmental delays to autistic spectrum disorders game state on how successfully the player is perform- (McEachin, Smith, and Lovaas 1993). They are, by ing. In addition, successful games pay a great deal of definition, evidence-based, individualized interven- attention to the rate in which complexity is increased tions. The behavior of each participant is observed, over the course of game levels and to the balance and measured, and analyzed, and treatment is driven by pacing of player advancement through these levels. evidence of whether improvements are observed or These issues of rates, balance, and pacing appear to not, and under what conditions those improvements parallel precisely the process that the behavior O were brought about. The interventions are typically analyst undertakes in designing an intervention.

Walz—The Gameful World PROPERTY OF THE MIT PRESS PROPERTY OF THE MIT PRESS FOR PROOFREADING, INDEXING, AND PROMOTIONAL PURPOSES ONLY FOR PROOFREADING, INDEXING, AND PROMOTIONAL PURPOSES ONLY

GAMIFICATION AS BEHAVIORAL PSYCHOLOGY 97

Define target Analyze behavior performance

Measure Define goals performance

Define rewards and Present reward schedule feedback

Figure 3.2 Diagrammatic illustration of the key processes involved in any behavioral intervention.

Besides the structural similarities between char- (Saville et al. 2006) to secondary school (Olympia et acteristic game design elements and the processes al. 1994), primary school (Lindsley 1971, 1992a, used for behavior modification, the other reason why 1992b), driver education programs (Bell et al. 1991), this field should be of interest to the designers of and challenging populations (Christopherson and gamified products is that there has been a great deal Mortweet 2001). Behavioral teaching methodologies of empirical support for the effectiveness of ABA pro- have been particularly successful as early interven- grams. Indeed, they have been extremely successful tions for children diagnosed with autistic spectrum wherever implemented, from university modules disorders (Lovaas 1987).

ABA Processes as Game Design Elements

In this section, we will take a step-by-step look at Selecting and Defining Target Behaviors some of the processes essential to any form of ABA, The most basic and important step of any interven- briefly explain some issues surrounding the imple- tion is to define a target behavior clearly. This must mentation of those processes, and point out how they be a clearly, objectively observable behavior that it is can be useful in the design of gamified products and possible to measure via the technology on which the services (figure 3.2). Obviously, we do not have space system is implemented. Whether or not participants here adequately to summarize decades’ worth of have reached a behavioral goal should be judged by work by thousands of researchers. For those who observation of that behavior, not by their answers to wish for a more detailed account, we recommend a questionnaire or other such self-report measure. Cooper et al. (2006).

O

Walz—The Gameful World PROPERTY OF THE MIT PRESS PROPERTY OF THE MIT PRESS FOR PROOFREADING, INDEXING, AND PROMOTIONAL PURPOSES ONLY FOR PROOFREADING, INDEXING, AND PROMOTIONAL PURPOSES ONLY

98 CONOR LINEHAN, BEN KIRMAN, AND BRYAN ROCHE

ABA programs, like games, break long-term goals tise. Time constraints are also characteristic game (such as running a marathon) into short-term com- design elements, and, as such, we can expect them to ponent tasks (the exercises expected each day over be used in many gamified services in the future. the course of training). Participants must demon- strate success at all of these short-term goals as they Recording Data advance through the program before requiring per- formance of the more complex skill (asking someone Closely related to the process of measuring behavior to run a marathon without having completed suffi- is that of recording those measurements in a manner cient training is not likely to have a successful that is amenable to analysis. Because the dependent outcome). Thus, the designer must clearly define not measure of all behavioral interventions is the change only the ultimate goal of the program but also the in behavior over time, applied behavioral psycholo- series of steps that learners must reach on their way gists typically use line charts to record and represent to that goal. In this way, a hierarchy of observable data. These charts are called celeration charts, as they behavior measurements is created, in which the most are designed to represent accelerating and decelerat- basic concepts and processes are taught first, and ing frequencies of target behaviors. In a gamified knowledge and performance are built methodically. service, these data points must be recorded in a way that is easy for the game application to read and Measuring Behavior analyze. Just how the data are presented is open to the creativity of the designer. Measurement refers to the process of assigning It is also essential to decide on what specifically numerical values to observed behavior. This must be must be recorded. Behavior analysts aim to record done in a coherent and meaningful manner so that every single instance of a target behavior and to plot the system can analyze that behavior and provide these on celeration charts. For example, in a spelling feedback. Notably, ABA programs typically measure exercise, the position of each letter in a word is not only accuracy (whether a target has been met or checked and marked whether it is correct or not. In not) but also temporal aspects of performance (how a gamified healthy eating application (i.e., https:// long it took the person to reach that goal). Behavior foodzy.com), it is essential to record every meal, analysts have found that measures that include tem- snack, and drink consumed in order to understand poral components, known as fluency measures, are a fully a user’s dietary behavior. more accurate method for judging the efficiency of behavior than simple measures of accuracy. For Analyzing Behavior Change example, knowing that someone has completed a five-mile run provides us with a lot less information The key metric used by behavior analysts in monitor- about their expertise than if we also know whether ing the success of learners is the change in their behav- the run lasted twenty minutes or an hour. ior over time. Essentially, once a learning outcome has Behavioral psychologists have also found that been defined, the behavior analyst continually mea- O imposing strict time constraints on behavior is a sures the learner performing that behavior and exam- useful method for ensuring the learner attains exper- ines whether or not the learner is approaching that

Walz—The Gameful World PROPERTY OF THE MIT PRESS PROPERTY OF THE MIT PRESS FOR PROOFREADING, INDEXING, AND PROMOTIONAL PURPOSES ONLY FOR PROOFREADING, INDEXING, AND PROMOTIONAL PURPOSES ONLY

GAMIFICATION AS BEHAVIORAL PSYCHOLOGY 99

outcome. Using celeration charts, it is easy for the the effectiveness of feedback. Both engaging games behavior analyst to understand the trajectory of and successful ABA programs use these basic pro- behavior and to take appropriate action to ensure that cesses in combination to ensure that the game is able appropriate behaviors are promoted and maintained, to provide consistent, appropriate, and specific feed- while inappropriate behaviors are modified or extin- back to the player and to guide the player toward guished. If games and game-inspired applications performing at a high skill level. It appears that adopt- are to automate this process of analysis successfully, ing the following approach is useful in (a) offering a they must similarly focus on identifying behavioral variety of rewards for appropriate performance, (b) trajectory. Specifically, trajectories­ explain crucial offering persistent negative consequences for poor temporal and contextual aspects of behavior that are performance, which the player will work to avoid, not available when analyzing behavior in terms of and (c) directly presenting aversive consequences means, individually or in groups. Luckily, analyzing when the user does something that the service pro- change in behavior is relatively simple, once the pre- vider does not want him or her to do. Of course, ceding steps of defining, measuring, and recording whether any stimulus serves as a reinforcer or an behavior have been carried out in a methodical fashion. aversive stimulus for any individual should be defined through careful observation of that individual’s Presenting Feedback behavior. Care should also be taken to personalize the schedule on which feedback is presented. Design- Throughout the chapter, we have discussed feedback ers who understand and use these processes will have in great detail in terms of operant conditioning, a better chance of promoting and maintaining scheduling of feedback, and ongoing evaluation of engagement with their gamified services.

Criticisms of Behavioral Psychology

It is almost a cliché that behavior analysis is not cur- now legendary attacks on Skinner’s position. The rently as popular as it once was as an approach to Skinnerian approach did not seem adequate to the psychology, because it apparently failed to provide task of describing behavior other than reflexive or an adequate account of complex human behavior and directly trained operant behavior. in particular an account of language and cognition. Skinner did indeed concentrate most of his attention Complex Language and Cognition on animal research and never conducted a single experiment on humans. However, his 1957 text Verbal As it happens, the critics were correct. Humans were Behavior was an attempt to show how the basic prin- more complex than animals, and it was Skinnerians ciples of behavior discovered using animal popula- who first came up with the evidence (see Galizio 1979). tions would apply in the human case. It is fair to say, In particular, it turned out that animals and humans sometimes behave differently under schedules of however, that it was relatively unsuccessful at that O task. Critics like Noam Chomsky (1959) engaged in reinforcement, and the reason had to do with the

Walz—The Gameful World PROPERTY OF THE MIT PRESS PROPERTY OF THE MIT PRESS FOR PROOFREADING, INDEXING, AND PROMOTIONAL PURPOSES ONLY FOR PROOFREADING, INDEXING, AND PROMOTIONAL PURPOSES ONLY

100 CONOR LINEHAN, BEN KIRMAN, AND BRYAN ROCHE

ability to follow verbal rules, which sometimes aid the case of complex linguistic activities such as game schedule learning but sometimes interfere with it playing, is in how it explains activities that seem (O’Hora and Barnes-Holmes 2001). Later on, it emerged motivated by intrinsic or private rewards, rather that only humans appear to be able to derive relations than extrinsic, observable ones. Because intrinsic (see the earlier section “Complexity and Challenge in motivations are not observable, their explanation Games”) between stimuli, irrespective of the level of would seem to lie outside the explanatory power of training supplied to do so (see Hayes et al. [2001] for behavior analysis. Aside from the possibility that an in-depth review and discussion). This represented there may be observable physiologic responses (e.g., a qualitative as well as quantitative difference in the adrenaline) that can explain some of the appeal of complexity of animal and human behavior. Moreover, such activities, the modern behavioral explanation it turned out that the ability to derive relations under- suggests that coherence is an important reinforcer lies all forms of human cognitive ability and may even for humans. Coherence and sense-making serve as be definitive of human consciousness itself. In the continually available reinforcers for further respond- meantime, new approaches to just about every aspect ing (Hayes et al. 2001). Humans appear to be highly of psychology have been provided by behavior ana- motivated to achieve coherence and make sense in lysts. The approach to therapy has been transformed, every context, even in the context of solving puzzles and the analysis of creativity, cognition, language, or playing games (see Barnes-Holmes et al. 2010). spirituality, personality, and intelligence have all This is yet another shift in modern behavior analysis been re-energized (see Dymond and Roche 2013 for a that is not familiar to the average psychologist. The book-length treatment of this issue). In effect, the move toward the explanation of behavior in terms of current analysis is provided within that context and in self-sustaining reinforcement loops is a major step the full knowledge that human behavioral repertoires toward explaining many forms of complex behavior, are more complex than those of animals. While a including game playing. detailed discussion of these differences is beyond the scope of the current chapter, it is worth pointing out Questions of Values and Control that an awareness of these differences has made the analysis provided in this chapter possible. More spe- Behavioral psychology often provokes unease due to cifically, our analysis of game complexity in terms its pragmatic focus on understanding and controlling of derived relations relies on the newly discovered behavior. Specifically, the goal of behavioral psychol- analytical unit of the derived stimulus relations, which ogy is to understand the processes through which is suitable only for human applications and is one that any desired change in any observed behavior can be Skinner did not live to see. brought about. In the context of designing a society, such as that imagined by Skinner (1948), this raises Intrinsic Motivation questions regarding who is designing that society, what values are inherent in that design, and who is Another criticism that has been leveled at behavioral judging what constitutes appropriate and inappro- O psychology, and one which is especially relevant in priate behavior (i.e., the behaviors that should be

Walz—The Gameful World PROPERTY OF THE MIT PRESS PROPERTY OF THE MIT PRESS FOR PROOFREADING, INDEXING, AND PROMOTIONAL PURPOSES ONLY FOR PROOFREADING, INDEXING, AND PROMOTIONAL PURPOSES ONLY

GAMIFICATION AS BEHAVIORAL PSYCHOLOGY 101

reinforced or punished). Because the gamified world perhaps we should be careful to hold these game envisioned by McGonigal (2011) and Schell (2010b) is designers to account in much the same way that one in which a designer decides on these exact issues, behavioral psychologists have been.

Conclusion

Gamification, the process of using game design ele- characteristic game design elements. Starting from ments in non-game contexts, has rapidly emerged as its origin in the early twentieth century, we have a massively popular tool in the development of online described behaviorism and its underlying philosophy services and applications. Seized by entrepreneurs as well as the (frequently misunderstood) core prin- and businesses as a way of increasing engagement ciples of operant conditioning, feedback schedules, with products, existing game designers and scholars and evaluation, relating these directly to techniques have, unsurprisingly, been vocal about what they per- used in real games and gamified experiences. Build- ceive as a desecration of their craft. However, both ing on this, we have explored the realities of com- camps fail to understand the true powers of games as plexity in games and the tried and tested approaches tools for learning, within the context of decades of of ABA in effecting behavior change in real-world research into the realities of behavioral psychology. contexts. Finally, we discussed the key components Specifically, all games, and all gamified products and of successful ABA programs in terms of game design. services, follow strict patterns of highly structured This includes the key processes of defining target behavior management, feedback loops, and reward behaviors, measuring and recording behavioral data, mechanisms in order to effect changes in player analyzing behavior change, and presenting appropri- behavior. Just as one can beat the boss in battle by ate personalized feedback. applying skills learned through the game or change Through a more thorough understanding of the one’s lifestyle through participation in a gamified principles of behavioral psychology, game designers experience, game design elements have a predictable and gamification professionals can better under- and measurable effect on one’s behavior. stand the processes at work when a player is engaged In this chapter, we have reviewed the field of with his or her game, and the potential effects on the behavioral psychology and described how behavioral player’s behavior. With the tools of ABA, designers processes are commonly implemented in both stand- have the ability to create measurably better-gamified alone games and gamification, through the use of experiences for the benefit of their players.

References

Arntzen, Erik, and Per Holth. 1997. Probability of Arntzen, Erik, and Per Holth. 2000. Equivalence stimulus equivalence as a function of training design. outcome in single subjects as a function of training Psychological Record 47:309–320. structure. Psychological Record 50:603–628. O

Walz—The Gameful World PROPERTY OF THE MIT PRESS PROPERTY OF THE MIT PRESS FOR PROOFREADING, INDEXING, AND PROMOTIONAL PURPOSES ONLY FOR PROOFREADING, INDEXING, AND PROMOTIONAL PURPOSES ONLY

102 CONOR LINEHAN, BEN KIRMAN, AND BRYAN ROCHE

Barnes-Holmes, Dermot, Yvonne Barnes-Holmes, Ian Cooper, John O., Timothy E. Heron, and William L. Stewart, and Shawn Boles. 2010. A sketch of the Heward. 2006. Applied Behavior Analysis. 2nd ed. Engle- implicit relational assessment procedure (IRAP) and wood Cliffs, NJ: Prentice Hall. the relational elaboration and coherence (REC) Corrigan, Patrick W. 1995. Use of token economy with model. Psychological Record 60:527–542. seriously mentally ill patients: Criticisms and mis- Bartle, Richard. 2011. Gamification: Too much of a conceptions. Psychiatric Services 46:1258–1263. good thing? Presented at: Digital Shoreditch. Avail- Deterding, Sebastian, Staffan Björk, Aki Järvinen. Ben able at: http://www.mud.co.uk/richard/Shoreditch Kirman, Julian Kücklich, Jaane Paavilainen, Valen- .pdf. Accessed February 17, 2013. tino Rao, and Jan Schmidt. 2010. Social Game Studies: Bell, Kenneth E., K. Richard Young, Charles L. Salz- A Workshop Report. Hamburg: Hans Bredow Institute berg, and Richard P. West. 1991. High school driver for Media Research. education using peer tutors, direct instruction, and Deterding, Sebastian, Dan Dixon, Rilla Khaled, and precision teaching. Journal of Applied Behavior Analysis Lennart Nacke. 2011. From game design elements to 24:45–51. gamefulness: Defining “gamification.” In Proceed- Bogost, Ian. 2011a. Exploitationware. Gamasutra, May ings of the 15th International Academic MindTrek 3. Available at: http://www.gamasutra.com/view/ Conference, 9–15. New York: ACM. feature/6366/persuasive_games_exploitationware Dymond, Simon, and Bryan Roche, eds. 2013. Advances .php. Accessed January 14, 2013. in Relational Frame Theory & Contextual Behavioral Bogost, Ian. 2011b. Gamification is bullshit. Atlantic 9 Science: Research & Application. Oakland, CA: New (August). Available at: http://www.theatlantic.com/ Harbinger. technology/archive/2011/08/gamification-is Ferster, Charles B., B. F. Skinner, Carl D. Cheney, -bullshit/243338/. Accessed January 14, 2013. W. H. Morse, and P. B. Dews. 1957. Schedules of Rein- Cassidy, Sarah, Bryan Roche, and Stephen C. Hayes. forcement. New York: Appleton-Century-Crofts. 2011. A relational frame training intervention to Fields, Lanny, Barbara J. Adams, Thom Verhave, and raise intelligence quotients: A pilot study. Psychologi- Sandra Newman. 1990. The effects of nodality on the cal Record 61:173–198. formation of equivalence classes. Journal of the Experi- Catania, Charles A. 1998. Learning. 4th ed. Cornwall- mental Analysis of Behavior 53:345–358. on-Hudson, NY: Sloan Publishing. Fields, Lanny, Kenneth Reeve, Devorah Rosen, Anto- Chomsky, Noam. 1959. Review of Skinner’s Verbal nios Varelas, and Barbara Adams. 1997. Using the Behavior. Language 35:26–58. simultaneous protocol to study equivalence class formation: The facilitating effects of nodal number Christophersen, Edward R., and Susan L. Mortweet. and size of previously established equivalence 2001. Treatments That Work: Empirically Supported classes. Journal of the Experimental Analysis of Behavior Strategies for Managing Child Behavior Problems. Wash- O 67:367–389. ington, DC: American Psychological Association.

Walz—The Gameful World PROPERTY OF THE MIT PRESS PROPERTY OF THE MIT PRESS FOR PROOFREADING, INDEXING, AND PROMOTIONAL PURPOSES ONLY FOR PROOFREADING, INDEXING, AND PROMOTIONAL PURPOSES ONLY

GAMIFICATION AS BEHAVIORAL PSYCHOLOGY 103

Fogg, B. J. 2002. Persuasive Technology: Using Computers Herrnstein, R. J. 1961. Relative and absolute strength to Change What We Think and Do. San Francisco: Morgan of response as a function of frequency of reinforce- Kaufmann Publishers. ment. Journal of the Experimental Analysis of Behavior 4:267–272. Foster, Derek, Conor Linehan, Shaun Lawson, and Ben Kirman. 2011. Power ballads: Deploying aversive Ijsselsteijn, Wijnand A., Yvonne de Kort, Karolien energy feedback in social media. In ACM CHI Extended Poels, Andrius Jurgelionis, and Francesco Belotti. Abstracts on Human Factors in Computing Systems, 2221– 2007. Characterising and measuring user experi- 2226. New York: ACM. ences. In Proceedings of International Conference on Advances in Computer Entertainment Technology, Galizio, M. 1979. Contingency-shaped and rule-gov- ACM Press, Salzburg, Austria, 2007. erned behavior: Instructional control of human loss avoidance. Journal of the Experimental Analysis of Behav- Karlsen, Faltin. 2011. Entrapment and Near Miss: A ior 31:53–70. comparative analysis of psycho-structural elements in gambling games and massively multiplayer online Gingold, C. 2005. What warioware can teach us about role-playing games. International Journal of Mental game design. International Journal of Computer Game Health and Addiction 9:193–207. Research 5 (1). Available at: http://www.gamestudies .org/0501/gingold/. Accessed May 8, 2014. King, Daniel, Paul Delfabbro, and Mark Griffiths. 2010. Video game structural characteristics: A new Haw, John. 2008. Random-ratio schedules of rein- psychological taxonomy. International Journal of forcement: The role of early wins and unreinforced Mental Health and Addiction 8:90–106. trials. Journal of Gambling Issues 21:56–67. Kirman, Ben, and Shaun Lawson. 2009. Hardcore Clas- Hayes, Stephen C. 1993. Why environmentally sification: Identifying Play Styles in Social Games Using based analyses are necessary in behavior analysis. Network Analysis, In Proceedings of Entertainment Journal of the Experimental Analysis of Behavior 60 Computing ICEC 2009, 246–251. Berlin: Springer. (2):461–463. Kirman, Ben, Conor Linehan, Shaun Lawson, Derek Hayes, Stephen C., and D. M. Long. 2013. Contextual Foster, and Mark Doughty. 2010. There’s a monster behavioral science, evolution, and scientific episte- in my kitchen: Using aversive feedback to motivate mology. In Advances in Relational Frame Theory & Con- behaviour change. In Proceedings of ACM CHI Extended textual Behavioral Science: Research & Application, ed. Abstracts on Human Factors in Computing Systems, 2685– Simon Dymond and Bryan Roche. Oakland, CA: New 2694. New York: ACM. Harbinger. Koster, Raph. 2005. A Theory of Fun for Game Design. Hayes, Stephen C., Dermot Barnes-Holmes, and Bryan Scottsdale, AZ: Paraglyph Press. Roche, eds. 2001. Relational Frame Theory: A Post- Skinnerian Account of Human Language and Cognition. Khatib, Firas, Seth Cooper, Michael D. Tyka, Kefan Xu, New York: Plenum Press. Ilya Makedon, Zoran Popovic, and David Baker. 2011. Algorithm discovery by protein folding game players. O

Walz—The Gameful World PROPERTY OF THE MIT PRESS PROPERTY OF THE MIT PRESS FOR PROOFREADING, INDEXING, AND PROMOTIONAL PURPOSES ONLY FOR PROOFREADING, INDEXING, AND PROMOTIONAL PURPOSES ONLY

104 CONOR LINEHAN, BEN KIRMAN, AND BRYAN ROCHE

Proceedings of the National Academy of Sciences of the McEachin, John J., Tristram Smith, and O. Ivar Lovaas. United States of America 108:18949–18953. 1993. Long-term outcome for children with autism who received early intensive behavioral treatment. Lindsley, Ogden R. 1971. From Skinner to precision American Journal of Mental Retardation 97: 359–359. teaching: The child knows best. In Let’s Try Doing Something Else Kind of Thing, ed. J. B. Jordan and L. S. McGonigal, Jane. 2011. Reality Is Broken: Why Games Robbins, 1–11. Arlington, VA: Council for Exceptional Make Us Better and How They Can Change the World. Children. London: Penguin.

Lindsley, Ogden R. 1992a. Precision teaching: Discov- Nacke, Lennart E., Mark N. Grimshaw, and Craig A. eries and effects. Journal of Applied Behavior Analysis Lindley. 2010. More than a feeling: Measurement of 25:51–57. sonic user experience and psychophysiology in a first-person shooter game. Interacting with Computers Lindsley, Ogden R. 1992b. Why aren’t effective teach- 22 (5):336–343. ing tools widely adopted? Journal of Applied Behavior Analysis 25:21–26. O’Hora, Denis, and Dermot Barnes-Holmes. 2001. Stepping up to the challenge of complex human Linehan, Conor. 2008. A Behavioural Analysis of Com- behavior: A response to Ribes-Inesta’s response. puter Game Playing Competence, Experience and Related Behavior and Philosophy 29:59–60. Physiological Processes. Doctoral dissertation, National University of Ireland, Maynooth. Olympia, Daniel E., Susan M. Sheridan, William R. Jenson, and Debra Andrews. 1994. Using student- Linehan, Conor, Bryan Roche, and Ian Stewart. 2010. managed interventions to increase homework com- A derived relations analysis of computer gaming pletion and accuracy. Journal of Applied Behavior complexity. European Journal of Behaviour Analysis Analysis 27:85–99. 11 (1):69–78. Rehfeldt, Ruth A., and Yvonne Barnes-Holmes, eds. Linehan, Conor, Ben Kirman, Shaun Lawson, and 2009. Derived Relational Responding Applications for Gail Chan. 2011. Practical, appropriate, empirically- Learners with Autism and Other Developmental Disabili- validated guidelines for designing educational games. ties: A Progressive Guide to Change. Oakland, CA: New In Proceedings of ACM CHI 2011, 1979–1988. New York: Harbinger. ACM. Roche, Bryan, Sarah Cassidy, and Ian Stewart. In Loftus, Geoffrey R., and Elizabeth F. Loftus. 1983. Mind press. Nurturing genius: Realizing a foundational aim at Play: The Psychology of Video Games. New York, NY: of psychology. In Cultivating Well-Being: Treatment Basic Books. Innovations in Positive Psychology, Acceptance and Com- Lovaas, O. Ivar. 1987. Behavioral treatment and mitment Therapy, and Beyond, ed. T. Kashdan and normal educational and intellectual functioning J. Ciarrochi. Oakland, CA: New Harbinger. in young autistic children. Journal of Consulting and Saville, Bryan K., Tracy E. Zinn, Nancy A. Neef, Clinical Psychology 55:3–9. O Renee Van Norman, and Summer J. Ferreri. 2006. A

Walz—The Gameful World PROPERTY OF THE MIT PRESS PROPERTY OF THE MIT PRESS FOR PROOFREADING, INDEXING, AND PROMOTIONAL PURPOSES ONLY FOR PROOFREADING, INDEXING, AND PROMOTIONAL PURPOSES ONLY

GAMIFICATION AS BEHAVIORAL PSYCHOLOGY 105

comparison of interteaching and lecture in the Sidman, Murray. 1994. Equivalence Relations and Behav- college classroom. Journal of Applied Behavior Analysis ior: A Research Story. Boston: Authors Cooperative, Inc. 39:49–61. Sidman, Murray. 2000. Equivalence relations and the Schell, Jesse. 2010a. Design outside the box. Presented reinforcement contingency. Journal of the Experimen- at: DICE 2010. Available at: http://www.g4tv.com/ tal Analysis of Behavior 74 (1):127–146. videos/44277/dice-2010-design-outside-the-box Skinner, B. F. 1948. Walden Two. Indianapolis: Hackett -presentation/. Accessed January 14, 2013. Publishing Company. Schell, Jesse. 2010b. Visions of the gamepocalypse. Skinner, B. F. 1953. Science and Human Behavior. New Presented at: Long Now Foundation, San Fran­­ York: Free Press. cisco, CA, July 27. Available at: http://longnow.org/ seminars/02010/jul/27/visions-gamepocalypse/. Skinner, B. F. 1957. Verbal Behaviour. New York: Accessed January 14, 2013. Appleton-Century-Crofts. Sherry, John L., Kristen Lucas, Bradley S. Greenburg, Skinner, B. F. 1959. Cumulative Record. New York: and K. Lachlan. 2006. Video game uses and gratifica- Appleton-Century-Crofts. tions as predictors of use and game preference. In Skinner, B. F. 1974. About Behaviorism. New York: Playing Video Games: Motives, Responses, and Conse- Random House. quences, ed. Peter Vorderer and Jennings Bryant, 213–224. Mahwah, NJ: Lawrence Erlbaum. von Ahn, Luis, and Laura Dabbish. 2004. Labeling images with a computer game. In Proceedings of ACM Sidman, Murray. 1971. Reading and auditory-visual CHI 2004, 319–326. New York: ACM. equivalences. Journal of Speech and Hearing Research 14:5–13.

Gameography

2D Boy. 2008. World of Goo. PC. 2D Boy. Origin Systems. 1997. Ultima Online. PC. Electronic Arts. Blizzard Entertainment. 2004. World of Warcraft. PC. Blizzard Entertainment. Relational Frame Training Ltd. 2012. SMART. RaiseYourIQ. Firaxis Games. 2010. Civilization V. PC. 2K Games. Tahkokallio, T. 2011. Eclipse. Lautapelit. Gygax, G., and Arneson, D. 2000. Dungeons and Dragons (3rd ed). Wizards of the Coast. Valve Corporation. 2007. Portal. PC. Valve Corporation Namco. 1980. Pac-Man. Arcade. Namco Midway. Zynga. 2009. Farmville. Facebook. Zynga Inc. Nintendo. 2003. Wario Ware. Gameboy Advance. O Nintendo.

Walz—The Gameful World PROPERTY OF THE MIT PRESS FOR PROOFREADING, INDEXING, AND PROMOTIONAL PURPOSES ONLY

O

Walz—The Gameful World