NEWSLETTER Building Better Trainers Through Education Nov/Dec 2001

The Science of by , PhD, and Robert E. Bailey

Editor’s note: We are pleased to dedicate this issue to an overview of —its grounding in science, its fundamental principles, and how behavior is analyzed by scientists. In two articles—this cover story and a second piece on the ABCs of behavior—behaviorists Marian and Bob Bailey summarize operant conditioning in a way that should be helpful to novice and experienced trainers alike. They whet our Joan Maxwell appetites for all there is to know, and all that is yet to be learned. Member Profile, p.13 We extend our deepest thanks to the Baileys for their contributions. Be sure to watch for INSIDE their articles in future issues, too, where they’ll talk about common myths and the application of operant conditioning in . For a biographical sketch of the ABCs of Behavior ...... 8 Baileys, please see “Meet the Baileys” on page 11. APDT Members in the News...... 4 Editor’s Corner ...... 2 es, Virginia, there is a science animals could learn through imitation, trial and error, or observa- Executive Director’s Message...... 5 behind modern animal training. It’s called operant tion. By 1910 Thorndike had devel- Gimme Shelter ...... 19 Y Y oped a law of —the law conditioning (OC)—more precisely Meet the Baileys...... 11 referred to as behavior analysis— of effect—that attempted to explain Member Profile ...... 13 and includes a method of behavior behavior in terms of stimulus- News from the UK...... 21 modification and therapy based on response and satisfaction/discom- OC Definitions at a Glance ...... 10 laboratory science. fort associations. The On Behavior ...... 15 Scientists who study law of effect stated that behaviors that Pick of the Letter...... 23 behavior have been “... the roots lead to satisfying President’s Message ...... 3 undertaking extensive science- of this science outcomes were more Reviewer’s Corner ...... 12 based research and likely to be repeated, Upcoming Events...... 23 gathering volumes of go back to while behaviors that Vital Signs...... 17 data on behavior lead to undesired outcomes were less analysis since the the turn of likely to be repeated. The Association of early 1930s. Psychology profes- Pet Dog Trainers, Inc. Although B.F. the century.” 17000 Commerce Pkwy., sor and researcher Suite C Skinner is the John Broadus Mt. Laurel, nj 08054 acknowledged Watson basically bypassed voice: 800 -pet-dogs originator of operant conditioning, fax: 856-439-0525 Thorndike’s law of effect. Instead, the roots of this science go back to Watson seized the Russians’ (Pavlov e-mail: [email protected] the turn of the century. Early web site: www.apdt.com and Bekhterev) work on the classical researchers such as Edward conditioning paradigm as the Executive Director Thorndike, , and John scientific cornerstone of his Lynn Becker Broadus Watson contributed to the “.” (We are most President evolution of our present-day familiar with Allan Bauman understanding and application of as the type of learning made famous Vice President behavior analysis. Pia Silvani by Ivan Pavlov’s stimulus-response Secretary FROM THORNDIKE TO SKINNER (S-R) experiments with dogs.) With Teoti Anderson (1900 TO 1940) the 1919 publication of his major Treasurer book, Psychology from the Standpoint The experiments of Sue Pearson of a Behaviorist, Watson became the addressed the Founder founder of the American school of Ian Dunbar mechanical problem-solving ability behaviorism. of cats and dogs and whether or not continued on page 6 The Science of Animal Training continued from page 1

During the 1920s, Watson became more involved with practical applications of psychology (in advertising and marketing) and did not extend his early work on behav- iorism. However, the “behavioral genie” was out of its bottle and, during the 1930s, others pursued animal behavior and its conse- quences. Chief among studying animal behavior at this time were the neobehaviorists, the “new” behaviorists, who adopted Watson’s main principles of objectivity and the study of overt behavior while developing their own variations on Watson’s theme. Most also tried to (l-r) B.F. Skinner, Bob Bailey, Marian Bailey (circa 1987) adapt Watson’s model of classical or Pavlovian conditioning to the more important, everyday behavior of animals and humans. Behaviorism, and mostly Skinner’s species. This fact should be noted version—operant conditioning— by those who dismiss OC as merely While B.F. Skinner has become the dominated psychology until the stimulus-response or as a simple most famous and influential of the 1970s and ‘80s. In the 1980s, the so- extension of respondent neobehaviorists, others developed a called “cognitive revolution” began conditioning. following. Tolman, Hull, and others to color psychologists’ thinking and also studied animal behavior. For a OPERANT CONDITIONING teaching, but not their practical number of reasons, behaviorism, led ACCORDING TO SKINNER achievements. Today, practical by the neobehaviorists, gained applications of so-called “cognitive Let’s take a look at the original basic strength through the 1930s and psychology” look suspiciously like principles of operant conditioning 1940s, perhaps paralleling the operant conditioning with different as laid out by Skinner’s landmark general rise of science, and became wrappings. book, The Behavior of Organisms. the dominant “system” of Operant conditioning concerns the In 1938 B.F. Skinner published The psychology. Skinner’s view was changing of motor actions and Behavior of Organisms: An finally to prevail because he put action patterns. These are fancy Experimental Analysis of Behavior, aside the Pavlovian paradigm as the names for things that we do in our arguably the most influential work center of his system, developing his normal working day—our so-called own scientific description of how of the century on animal behavior. Skinner’s approach focused on voluntary movements. Keep in behavior changes. His system also mind that this focus on voluntary worked in the real world, a everyday behavior instead of involuntary physiological reflexes. movements is a big switch from significant contrast to most other earlier research that focused on “psychologies” of the time. Thus, he moved away dramatically from the behaviorism models of involuntary physiological reflexes. The point of all of this is that John Watson and Ivan Pavlov that Skinner shrewdly reasoned that behaviorism, and the neobehav- had emphasized the psychology of most everyday behavior—such as iorists’ view of behavior, is not stimulus-response, and embraced studying, mowing the lawn, or monolithic. Operant conditioning, the theory of natural selection of going shopping or to a movie—does which we are discussing here, is not successful responses through their not fit the Pavlovian S-R model. the only behavioral psychology. consequences to humans and other There is no single, outstanding

6 NOV/DEC 2001 stimulus to which one can point that strength or happen less often. The selectively shape just about any elicits, automatically, any of those process of reducing behavior due to behavior possible for a given behaviors. Rather, the responses that lack of is called animal. Operant conditioning is a make up our everyday behavior extinction. science that has given us the tools to grow stronger and survive as the study and predict behavior and, Applying an aversive stimulus result of our operation on the ultimately, to influence, if not com- () when administered by environment—hence, the designa- pletely control, a given behavior. a human or another animal will also tion “operant” for these responses. decrease behavior. Actually, the very Applied behavior analysis is not In other words, reflexes may allow definition of whether a stimulus is exactly synonymous with applied us to control our blood pressure aversive or punishing rests on the operant conditioning, anymore than automatically, but learning operant decline of behavior. If applying the behavior analysis is not exactly the behaviors allows us to cope with stimulus results in the decline of a same as operant conditioning. our surroundings. behavior, that stimulus is classed as Operant conditioning, as a science BEHAVIORAL CONSEQUENCES an aversive, or a punisher. It is also or as a technology, is the study, Skinner recognized that an operant true that if a stimulus increases the prediction, and control of operant behavior could have three possible rate of behavior, that stimulus is a behavior. Respondent conditioning consequences: reinforcer. Thus, the nature of a has to do with respondent (largely stimulus, reinforcing or aversive, is reflex) behaviors, not operant • good (reinforcing) defined by the behavior of the behaviors. Behavior analysis • bad (aversive) animal and not by a decision of the includes both operant and trainer. respondent behaviors and their • neutral, or none (extinction) interactions. We, the Baileys, view This analysis of potential conse- behavior as more or less a quences is one of the beauties of continuum between what has been operant conditioning, offering a “Operant conditioning thought by many to be totally simplicity and clarity available in no separate camps, operant and other behavioral methodology. is a science that has respondent. For this discussion, Things can get better, get worse, or however, we do not wish to split stay the same—all depending on the given us the tools to hairs and will accept OC as synony- circumstances. mous with behavior analysis. According to Skinner, behavioral study and predict When we teach animal trainers, our consequences are measured by the position is that OC is “simple but rate of response; that is, how often a behavior and, not easy.” The animal decides what response occurs per unit of time. is reinforcing and what is punishing. Skinner said that a reinforcer is any ultimately, to Not all reinforcers are under the object or activity that causes a direct control of the trainer (there response to increase in strength (rate influence, if not are internal as well as external or probability of response). In other words, the occurrence of reinforced completely control, events leading to reinforcement). behavior increases (or sometimes The animal decides the meaning of maintains its rate), while the occur- a given behavior.” cues. The animal is always right. rence of non-reinforced behavior The trainer’s challenge is to observe decreases. the animal, communicate with the animal, and respond to the animal. REDUCING BEHAVIOR Yet, all the while, the trainer must be There are two ways of reducing OC IS SIMPLE in substantial control of the animal’s behavior: ignoring it and punishing Are you grasping the point that OC behavior in order to ply his or her it. If a behavior is not reinforced in is made up of simple concepts, craft. This is the reason for our any way, it will decrease in fre- practices, or principles? You might “simple but not easy” view of OC, quency. Thus, if nothing occurs argue, “It can’t be that simple.” But and probably why many find OC inside or outside the animal that it is. By combining these practices difficult to apply. reinforces a behavior, the probability and some others, you can increase is that the behavior will reduce in or decrease most behavior and continued on page 11

NOV/DEC 2001 7 The ABCs of Behavior by Marian Breland Bailey, PhD, and Robert E. Bailey

he “ABCs of Behavior”— can become salient in one of two him in the past. But quite a different meaning the antecedents, ways: situation occurs when the dog has T behavior, and consequences— • Natural salience: where the dog spent time in a pound, an animal T shelter, or an abusive home. If you is a shorthand expression used by comes with a genetic tendency to behavior analysts since the 1960s. pay special attention to certain give a cue or command to sit and Before behavior analysts, or trainers, stimuli (e.g., a German shorthair the dog responds, you can probably begin to build a new behavior or pointer being stimulated by the assume that this behavior was part change an existing one, they care- rustling of a pheasant in a field). of the dog’s previous training his- fully examine the content of each of tory. Likewise, if the dog is fearful of these elements. This helps to deter- • Learned or conditioned salience: specific objects such as sticks or mine what may be causing a behav- where the dog is exposed to a walking canes, there is a good ior, as well as how to change it. stimulus that is paired with chance that the dog has encountered primary or strong secondary A IS FOR ANTECEDENT these objects in an unpleasant reinforcers or aversives (e.g., the framework. “Antecedent” refers to anything that sound of a kitchen can opener precedes the occurrence of the that precedes the dog’s meal, the The more you know about the dog behavior. In the dog training world, word “No!” shouted by an you are training, the greater your we are most familiar with owner as his dog moves toward advantage in determining how to antecedents that are stimuli—such a ham sandwich on the kitchen change behavior. as an odor, a high-pitched sound, or counter). ESTABLISHING OPERATIONS a trainer’s verbal cue—in short, any Regardless of how stimuli acquire Yet another form of antecedent is an change in physical energy that the their salience, they can become the “establishing operation,” also dog can perceive or react to. most important variable in a known as a “setting factor.” The Stimuli present themselves to the training scenario. For shaping or general definition of an establishing dog in different forms. Let’s talk modifying a behavior, the trainer operation is a condition that about two of those forms: should have control over these changes the value of a reinforcer background/context stimuli and salient stimuli. If a stimulus in the and, thus, increases the likelihood salient stimuli. whole array of background stimuli that certain responses will increase BACKGROUND/CONTEXT STIMULI is more salient than those being or decrease. For example, if a dog presented by the trainer, that trainer deprived of food becomes more This form is the entire array of is in trouble because she is losing hungry than usual, the value of the stimuli—the context—that greets the stimulus control over her animal. food will increase; the dog will per- dog as he encounters a situation in form behaviors that have resulted in which behavior is going to occur. For example, a trainer may discover food in the past. The dog also will These stimuli might include sounds, to her chagrin that an incidental arm work harder to get the food. odors, and visual cues, or any com- movement causes the dog to emit a bination of vision, hearing, touch, response prematurely. In behavior Other establishing operations may temperature change, , head analysis parlance, that arm move- include availability of water, movement, and balance. It includes ment is a context stimulus; however, changes in climate, hormonal the trainer and any other individ- it is more commonly referred to by changes, and various health con- uals present, and their movements. trainers as a “secondary cue.” Con- ditions (e.g., a dog with digestive Of this total array of stimuli, most text stimuli are always present, so tract problems might typically show may be totally irrelevant or mean- trainers should take precautions to no interest in food). ingless to the dog at a given time. stand or sit very still and supply no B IS FOR BEHAVIOR In some situations, however, the dog extraneous stimuli while training. may react to one or more important When we talk about a dog’s PREVIOUS CONDITIONING behavior that occurs in response to stimuli, which we call the salient HISTORY stimuli. antecedents, we’re referring to the Another important antecedent of behavior’s function—what the SALIENT STIMULI behavior is the dog’s history. If you behavior does for the dog or, in The word “salient” means have raised a pet yourself, you more other words, the consequence of the outstanding or prominent. Stimuli or less know what has happened to behavior. We are also talking about

8 NOV/DEC 2001 all the responses and sub-responses treat given by a trainer for a or occurrences (e.g., an event that that take place in the instance of properly executed response. causes tissue damage or an intolera- behavior occurring. Positive reinforcement involves the ble odor). Secondary aversives, like For example, a dog trainer may presentation of a good consequence secondary reinforcers, become what have a problem with his dog not when the response is performed they are by association with the “real being able to clear a high hurdle. (e.g., you say “Come,” the dog thing” (e.g., the word “No!” paired The trainer may analyze a behavior comes, and you offer a treat). It with a slap). called, perhaps, “hurdle jumping.” increases the likelihood that the Aversives in general, and punish- Clearly, such a behavior consists of a behavior will occur in the future. ment in particular, may have bad number of responses such as getting consequences for the dog and into the starting position, running trainer. They can produce uncontrol- toward the hurdle, gathering limbs “Extinction ... lable fear, not only of the trainer, but into the jumping mode, springing the entire training situation. Aver- over the hurdle, and landing. sives can suppress virtually all represents the surest, behavior. They may also encourage The trainer’s goal is to evaluate each aggressive responses. More accept- response individually and try to most permanent, able alternatives, such as reinforce- identify a specific sub-response ment, should always be considered that’s causing the problem. For and most humane before using aversives. example, he may videotape the jump and notice that one of the way of eliminating EXTINCTION dog’s legs is poorly angled for best Extinction refers to the gradual propulsion of the body into the leap. a behavior.” weakening of a previously rein- In that case, he can use shaping forced response when it is no longer techniques to work the leg into the reinforced. This represents the proper position. surest, most permanent, and most Regardless of the complexity of the Negative reinforcement involves the humane way of eliminating a behavior, analysis of the responses removal of a bad consequence when behavior. For example, a dog kept in and sub-responses allows the trainer the response is performed. Many a yard discovers that by pushing on to plan the training periods and dog trainers are familiar with the ear a certain board near the gatepost, he sessions. Planning of this sort can pinch used in a forced retrieve to can make the gate open. On several give the trainer a huge advantage compel a dog to take the dumbbell occasions the dog gets out this way over a haphazard trainer who may in his mouth; this is the use of and enjoys romping through the approach the problem with no negative reinforcement. When the neighborhood. Finally, the dog analysis and no plan. dog takes the dumbbell, the ear owner fastens the board so that the pinch is stopped. As with positive dog can no longer open the gate C IS FOR CONSEQUENCES reinforcement, negative reinforce- with a push. The dog keeps pushing As we mentioned in the “Science of ment increases the future likelihood unsuccessfully at this board, time Dog Training” article beginning on of the behavior that removes or after time, until gradually the page 1, an operant can have any one avoids the aversive. response of pushing disappears. of three consequences: AVERSIVES (PUNISHMENT) When using extinction to eliminate 1. reinforcement Not all operants are so fortunate as an undesired behavior, it is important to train a substitute 2. aversives (punishment) to be followed by reinforcers that strengthen responses. One type of behavior. Teaching a dog to sit 3. extinction consequence—an aversive—not instead of jump up on visitors is a REINFORCEMENT only weakens the behavior it good example. Reinforcement, which can be either follows, but can actually deal it a Two things may happen in the positive or negative, is the process deathblow to the extent that the course of extinction that sometimes of strengthening the operant that behavior may never reappear. discourage trainers who are trying produces a desired consequence. There are two categories of aversives: to use this method to get rid of a The reinforcer can be the natural primary and secondary. Primary problem behavior. The first of these result of the dog’s behavior in its aversives, like primary reinforcers, is the “extinction burst,” a sudden everyday environment, or perhaps a have their roots in natural substances continued on next page

NOV/DEC 2001 9 The ABCs of Behavior continued from previous page occurrence of several rapid emis- response may appear to be com- trainer who wants to increase the sions of the response in question. pletely gone. A period of time later— energy a dog puts into a response For example, the dog who has a few hours, overnight, perhaps may be able to “catch” some of the almost given up pushing the board longer—the dog again offers the vigorous responses caused by an may suddenly emit several quick response in a training session. extinction burst, reinforce them, pushes. This extinction burst can extinguish the weaker responses, occur any time during the extinction Neither extinction bursts nor and build to a generally more pow- process. In fact, such a burst often spontaneous recovery are incidences erful response that can then be put occurs at the beginning of an of the dog testing the trainer or under stimulus control, (i.e., on cue). extinction period (or training being stubborn. Rather, each is a Extinction may occasionally induce session). Many novice trainers normal phenomenon that occurs in a small amount of aggression, but observe the burst and then con- the process of changing behavior. nothing like that which occurs with clude, erroneously, “See, extinction Extinction often encourages the use of punishment. Extinction does not work. The response has variability of behavior, allowing also does not seem to create the fear become more frequent and faster.” new responses to appear and present in the animal after use of The second thing that may happen is receive reinforcement. Many aversives. Hence, it is preferable as a “spontaneous recovery.” After an experienced trainers take advantage means of weakening or eliminating initial extinction session, the of this phenomenon. For example, a behavior.

OC DEFINITIONS AT A GLANCE

Operant Conditioning: a naturalistic, Behavior: anything an animal does Extinction: the gradual weakening of a scientific description of how animal previously reinforced response when it Consequence: the result of an animal’s behavior changes—in other words, how is no longer reinforced behavior animals learn; an objective science that Aversive: a consequence that weakens emphasizes quantitative data, THE FUNDAMENTAL PRINCIPLES OF OC the behavior it follows; also called replications, verification, and follow-up Trainers who use operant conditioning punishment when applied by a human THE ABCSOFBEHAVIOR apply five fundamental principles. Each (trainer, parent, etc.) or another animal (usually a conspecific, e.g., a mother These are the elements of behavior that of these principles describes a process bear cuffing her cubs) are analyzed to determine how to best that alone, or in combination with change an animal’s behavior. others, provides trainers with essential Generalization: the ability of an animal tools in changing behavior. to learn to respond to a range of stimuli Antecedent: something that happens rather to a single stimulus (“stimulus before the behavior occurs; the Stimulation: a change in physical generalization”) and the ability to learn most common of these are stimuli, energy to which an animal can respond to perform a certain behavior or group establishing operations, and or that an animal can perceive with its of behaviors in response to a specific previous history sensory organs (e.g., vision, touch, hearing) stimulus (“response generalization”) Establishing operation (EO) or setting Editor’s note: Please refer to the factor: an operation or event that Reinforcement: the process of Sep/Oct 2001 issue for definitions of changes the value of a reinforcer and strengthening the behavior that positive reinforcement, negative increases the strength of all responses produces a desired consequence; can reinforcement, positive punishment, pertaining to that reinforcer be either positive or negative and negative punishment.

10 NOV/DEC 2001 Meet the Baileys

arian and Bob Bailey are applied behavior MManalysts. Some people might call them operant trainers. Marian, one of B. F. Skinner’s early undergraduate and then graduate students, holds a doctorate in psychology and was a university professor for almost 20 years. Bob, who holds degrees in chemistry and biology, pioneered operant- based animal training methods in free-environment and production settings. Marian began training animals in 1938, and she trained her first dog in 1941. She was the first person to apply operant conditioning com- mercially. Bob began training in the late 1950s and was a pioneer in the open ocean use of dolphins while (l-r) Bob Bailey, Marian Bailey, and student he was the Director of Training for the U.S. Navy. involved releasing trained animals behavior analysis. They are The Baileys have trained over (both birds and mammals) in currently documenting the history, 15,000 animals representing more environments that are sometimes science, and practices of operant than 140 species in the past 50+ hostile and always distracting. conditioning and behavior analysis. years. Among these are numerous Although the Baileys are retired When teaching, they often use dogs for commercial, research, and from business and from academic chickens as behavioral models. military applications. Large teaching, they consult, lecture, and For more information, go to segments of their work have conduct workshops on applied www.hsnp.com/behavior.

The Science of Animal Training continued from page 7

OC IS NOT A THEORY of how rat behavior changes as gathering and analysis of data. circumstances change. Some of his Many people believe that operant As a science, OC is objective and later books were elaborations or conditioning is just a theory. It is not. open to the world, and it extensions of the same observations. Operant conditioning is a natural- emphasizes quantitative data, Skinner did not postulate or specu- istic, scientific description of how replications, verification, and late on the reasons or mechanisms of animal behavior changes—in other follow-up. As a technology, it is a such changes. As a personal aside, in words, how animals learn. powerful tool for changing our private conversations with behavior. On a personal level, we Skinner’s book, The Behavior of Skinner, he did not hold theorizing earned our livings using it for Organisms, contains his observations in high regard, rather preferring the almost half a century.

NOV/DEC 2001 11