The ABCs of Behavior Analysis An Introduction to Learning and Behavior

A. Charles Catania

2017 Sloan Publishing Cornwall on Hudson, NY 12520 Contents

Preface for Student Readers vii Preface for All Readers ix

Part I. Introduction 1. Learning and Behavior 1 2. A Behavior Taxonomy 8

Part II. Behavior Without Learning 3. Evolution and Development 18 4. Motor and Sensory Systems 27 5. Elicited and Emitted Behavior 38

Part III. Learning Without Words: Consequences 6. Consequences of Responding: Reinforcement 47 7. Reinforcers as Opportunities for Behavior 58 8. Reinforcement, Free Reinforcement, and Extinction 67 9. Consequences of Responding: Punishment 74 10. Consequences of Responding: Escape and Avoidance 83

Part IV. Learning Without Words: Operant Classes 11. Operants: The Selection of Behavior 91 12. The Structure of Operants 101 13. Motivating Variables and Reinforcer Classes 109

Part V. Learning Without Words: Contingencies 14. Parameters of Reinforcement: Schedules and Delays 119 15. Discriminated Operants: Stimulus Control 131 16. Conditional Discrimination and Stimulus Classes 145 17. Sources of Novel Behavior 156 18. Behavior Synthesis 168

v vi • CONTENTS

Part VI. Learning Without Words: Conditioning 19. Respondent Behavior: Conditioning 181 20. Operant-Respondent Interactions: Emotion 191

Part VII. Learning with Words: Verbal Behavior 21. Social Learning 198 22. Words as Stimuli and Responses 205 23. Antecedents and Consequences of Words 215 24. Contact of Verbal Behavior with the Environment 223 25. Verbal Behavior Conditional on Verbal Behavior 235

Part VIII. When Verbal and Nonverbal Behavior Interact 26. Verbal Governance 241 27. Prejudice as Verbally Governed Discrimination 254 28. Verbal Function: Coordinations among Classes 263

Part IX. Remembering and Knowing 29. Remembering 272 30. Knowing 284

Part X. Conclusion 31. Applied Behavior Analysis 295 32. Learning and Behavior Revisited 305

Appendices 317 Glossary 325 References 351 Acknowledgements and Notes 374 Index 375 Preface for All Readers

This book started as a brief outline tentatively entitled A Primer of Behavior Analy- sis. One of its antecedents was George Reynolds’ Primer of Operant Conditioning, published by Scott, Foresman in 1968. That book ran to 130 text pages plus sup- porting material and brought together the basic concepts of the fi eld later to be known as behavior analysis. In those days primer was pronounced to rhyme with trimmer, befi tting a book that’s supposed to be short; nowadays it’s sometimes pronounced like rhymer. My earlier book, Learning, had grown longer over successive editions and had become more a graduate than an undergraduate text. I’d often thought about a brief version that could function as George Reynolds’ book once did. But we know much more now than we did then, so I’m happy to have ended up with a bit more than 300 text pages plus supporting material. Of course it could have been much shorter, but that would have made it impersonal and far more abstract. I hope readers will fi nd I’ve struck a reasonable balance among concise presentations of basic concepts, discussions of relevance, concrete examples and illustrations of applications. I can’t date when I fi rst drafted the outline, but I stopped worrying about pro- nunciation when length made it obvious the Primer title was inappropriate. The main title, The ABCs of Behavior Analysis, captures the basics while also highlight- ing the three-term contingency: Antecedents–Behavior–Consequences. The sub- title, An Introduction to Behavior and Learning, provides a reminder of the linkage between behavior and learning. The essence of learning is creating new behavior, and that encompasses much of what we behavior analysts do. Three features seemed especially important as the manuscript evolved. First, it had to represent behavior analysis as a discipline in its own right rather than as a component of psychology: psychology has become an umbrella designation for a variety of approaches only some of which are compatible with behavior analysis, and behavior analysis encompasses many topics treated by psychology as separate domains. Second, selection by consequences had to be its organizing principle: in both its methods and underlying assumptions, behavior analysis has more in com- mon with biology than with most varieties of psychology and cognitive science. Third, it could not be limited to the basic nonverbal processes: it would assume no one unfamiliar with at least the rudiments of the science of verbal behavior could truly be counted as a behavior analyst. These three prerequisites made the new work inevitably parallel in many ways the organization of my earlier Learning. Anyone comparing the respective tables

ix Chapter 2 A Behavior Taxonomy

Science…is becoming the study of organisms. — Alfred North Whitehead

A behavior taxonomy is a vocabulary for orga- tion corresponds most closely to what happens nizing the various procedures and phenomena in the Discussion section. of behavior. Please don’t expect explanations of Behavior of course occurs in natural settings behavior based on formal laws. Instead, think as well as artifi cial ones arranged by an experi- of a systematic classifi cation of behavior based menter. We startle in response to an unexpected on its origins: given any example of behavior, loud noise whether produced by accident on where did it come from? The taxonomy won’t be the street or deliberately in a lab. For that rea- exhaustive, because we can’t anticipate every- son, these terms are applied both to artifi cial thing we’ll run into. Behavior analysis remains procedures and their outcomes and to natural work in progress, but we can at least aim for a sequences of events. For example, Chesapeake, descriptive system that organizes the phenom- our cat , has learned he often gets fed after going ena we’ve begun to know while not excluding to his feeding dish in the kitchen. It hardly mat- those we’ve yet to study. ters whether I deliberately trained his behavior We study the relation between environmental or it came about as a matter of course during events and the organism’s behavior by chang- routine feedings. Either way, being fed was a ing the environment and observing how this frequent consequence of going there. It’s con- affects what the organism does. In the analysis venient to call such circumstances procedures of behavior, procedures or operations are what or operations whether or not I arranged them the experimenter or the environment does or explicitly. arranges, and outcomes or processes are the result- The simplest procedure is to observe behavior: ing changes in behavior. A convenient analogy just watch. We discover what an organism can comes from medicine, where the surgical opera- do. But we have no control over events when we tion, a procedure, is what the physician does to simply observe, so we may not be able to draw the patient, and the processes that follow, such conclusions about the causes of behavior. To learn as changes in circulation, respiration and so on, more we must intervene, and the simplest inter- are the outcomes. I’ll usually favor procedure over vention is to present stimuli. A more complicated operation and outcome over process, but within intervention is to arrange the environment so the each pair the terms are usually interchangeable. organism’s behavior can change it or, in other If you’re familiar with the standard sections words, so behavior has consequences . Once behav- of experimental papers you’ll recognize this ior has consequences it may occur more or less distinction as paralleling the difference between often, when we may sometimes call the respec- the Method section, which describes the pro- tive outcomes reinforcement and punishment . cedures and other details of an experiment, We haven’t exhausted the possibilities. We and the Results section, which describes the can arrange things so stimuli signal the pre- obtained data. We can also interpret changes in sentation of other stimuli or so stimuli signal behavior once we’ve observed them; interpreta- the opportunity to produce consequences. We

8 Chapter 4 Motor and Sensory Systems

.…he has ascribed the process to internal sounds and assumed that the ear produces a sound within, like a bell…. but how should we hear this internal sound itself? —Theophrastus criticizes explanations of hearing (in Stratton, 1917)

Selection shaped the bodies of organisms based at least safe to assume that response systems on how those bodies behaved. Among verte- evolved before sensory ones. An organism that brates, selection operated on muscular and skel- can’t do anything about what it sees gains no etal and neural structures allowing them to eat advantage by seeing. Some organisms remained and mate and get around via walking, running, immobile but others began to twitch and squirm. swimming, fl ying, climbing, tunneling, slither- Some were passively moved by ocean currents ing, and so on (arthropods and cephalopods but others anchored themselves to a site. Still and organisms in other phyla do these things others developed cilia and fl agella. The organ- too, but in other ways). Sensory systems were isms from which we are descended developed advantageous because without them behavior ways to get from one place to another. would have been inappropriate to the settings The most primitive patterns of movement in which organisms found themselves. As their were probably driven primarily by eliciting touch and taste evolved, they could respond dif- stimuli. Bright light elicits random movement ferentially to things they contacted. With smell in an insect larva. In what’s called a kinesis (plu- and vision and hearing they could respond to ral: kineses), it moves until by chance it reaches more distant stimuli. I could have started here darkness; then it stops (Fraenkel & Gunn, 1961; with sensory systems, because discriminative Loeb, 1918/1973). We fi nd few larvae in the light stimuli come before responses in the three-term but many in the dark, such as under the rotting contingency. But in the evolution of behavior, bark of a fallen tree limb. Their behavior wasn’t responding had to be there fi rst or there’d be no directed toward dark places. They got there by behavior to participate in a three-term contin- chance and congregated there because that’s gency, so that’s where we’ll go next after some where they stopped. Directional control by stim- general points. uli probably came later. A kinesis differs from a taxis (plural taxes), orientation directed toward or away from some stimulus. Examples include THE EVOLUTION OF MOTOR movements toward light (positive phototaxis) AND SENSORY SYSTEMS and upward movement, away from gravity, as in climbing a tree (negative geotaxis). Directed What can we say about the evolution of behav- movements are sometimes also called tropisms , ior and learning? Behavior leaves only indirect but that term is more often applied to plant evidence in the fossil record (nests can hint at behavior, as when fl owering plants turn toward ancient ways of raising young; footprints can light (phototropism). reveal some details of walking or running). It’s

27 28 • Part II. Behavior Without Learning

Each example involves a fi xed pattern of day to day, such as the passage of predators or responding to events. First there was movement ; the places where food or water are found. The then came movement in relation to stimuli. Such capacity to learn and remember likely follows a patterns were especially advantageous in stable developmental course. Those looking for learn- environments. For example, given a sharp stim- ing mechanisms in the nervous system must rec- ulus to its paw, a dog fl exes its leg, drawing its ognize that evolutionary contingencies may have paw up toward its body. For an animal that walks selected different kinds of learning: some pro- on the ground, this response is advantageous: if ducing fairly permanent changes and others eas- the dog steps on a thorn, its fl exion pulls its paw ily reversed; some restricted to relatively narrow away from the thorn. The sloth , however, hangs situations and others broadly general; and some from trees and its comparable refl ex involves an operating at different times in life than others. extension rather than fl exion of its limb. If the sharp stimulus is a thorn, the hanging sloth pull- ing its limb toward its body instead of extending MOTOR SYSTEMS it only drives the thorn deeper (B. L. Hart, 1973). Not all environments are stable. A major In vertebrates, motor systems range from those evolutionary step was when patterns of behav- involved with eating and drinking and mating to ior became modifi able or, in other words, when those of locomotion. Depending on the medium, some organisms became able to learn. Learning organisms get around by moving fi ns or legs or may at fi rst have been selected within restricted wings or other appendages. What we humans domains. For example, upon leaving its nest the do artifi cially with rotary devices such as wheels digger wasp fl ies in ever-widening circles. Its or propellers or the screws that move vessels later return to the nest is based on landmarks through water most organisms accomplish by such as the rocks or plants it fl ew over before back and forth motions, in walking or running its departure (Tinbergen, 1972). Its capacity to or fl ying or swimming (Gray, 1953). This behav- learn landmarks, very specifi c to fi nding its nest, ior is sophisticated: robotics is getting better at is part of its phylogenic heritage. designing machines that walk successfully on The selection of the capacity to learn no doubt irregular surfaces, but novel environments con- occurred in different ways in different species at tinue to present challenges. different times. Things about environments that Walking demands complex muscle coordi- ordinarily remain constant over a lifetime need nation along with maintenance of balance. In to be learned only once. Things about changeable knee fl exions, for example, one muscle relaxes environments need to be learned and then dis- as the opposing muscle contracts. Much of this carded and then learned again. In our own lives, coordination is built in. When hatchling chicks the names of people we know don’t change very are deprived of fl apping and fl ight experience often, whereas the items on our shopping lists by wing restraint or other means and their wing usually change from one shopping trip to the coordination is tested later, fl apping is syn- next. We should expect some kinds of learning chronous from the start (Provine, 1984). Many to be hard to reverse while others are transient aspects of fl ight coordination are “prewired” and easily modifi ed. at the level of the spinal cord. Nevertheless, the The survival of a newborn foal may depend environment still matters. However the bird on how well and quickly it learns about the fea- does it, current environments determine when it tures distinguishing its parent mare from oth- fl ies and where it goes. ers. In a stable environment, such learning may A study of leg movements in infancy be effectively permanent. The time of life dur- recorded the timing and topography with which ing which learning occurs may also be limited. 3-month-old infants kicked at a mobile (Thelen As the foal grows and becomes more indepen- & Fisher, 1983). The visual consequences of kick- dent, it will learn about things that change from ing varied: some infants saw the mobile move Chapter 4. Motor and Sensory Systems • 29 when they kicked and others didn’t. These con- cords, and the shift to bipedal locomotion that sequences affected the rate and vigor of kicking freed the lungs and rib cage from the constraints but not the coordination of its fl exion and exten- on control of breathing produced by walking on sion phases. In other words, some features of the all fours (Provine, 2004) kicks were modifi able but others weren’t. Simi- These examples emphasize muscle move- larly, contingencies affect the direction in which ments, but not all behavior involves muscles. you walk but not the detailed coordination of Even beyond tearing and salivating and other your muscles and joints as you do so. glandular effects, we needn’t move to interact Sometimes we recognize people at a distance with our environments. When viewing art I can simply based on their walks. Many aspects of look from one part of a painting to another, but coordination operate independently of the envi- when listening to music I don’t have to turn ronment (Gallistel, 1980; Gray, 1953), when they my head to shift my attention from one musi- may be called motor programs. A horse trainer cal instrument to another. Eye movements differ doesn’t have to shape the details of stepping or from shifts in listening, but they have in com- the order of leg movements as a horse speeds mon that each is modifi able by its interaction up from walk to trot to canter to gallop. A show with the environment. horse might be taught special steps, like the rack, but even in these cases the new topography mod- ulates existing patterns. SENSORY SYSTEMS Motor systems can differ in other ways. For example, avian fl ight can occur in two modes: A pigeon is more likely than a bat to respond energetic fl apping that gets birds off the ground to visual stimuli, whereas a bat is more likely and up to air speed, and sustained fl ight with than a pigeon to respond to auditory stimuli. wings outstretched as airfoils, with movement If an organism’s sensory capacities aren’t taken mostly restricted to the ends of the wings where into account, experiments can yield misleading the feathers provide forward thrust. The feath- results. For example, the sound frequencies at ers are functionally like back-and-forth propel- which sensitivity is maximal are much higher ler blades and work by fl ipping their orientation for rat ears than for human ears. The experi- as the wings move up and down. The breast menter who uses auditory stimuli easily heard muscles driving the energetic takeoff, low in by rats may be unable to tell whether the stim- myoglobin, fatigue rapidly; the postural muscles uli are on or off, but the one who uses auditory of sustained fl ight are richer in myoglobin and stimuli easily heard by humans may be giving less subject to fatigue. the rat sounds hard for it to hear and may incor- Getting around is only one of many motor rectly conclude the rat learns slowly and with functions. Our human hands are special, and we diffi culty. With squirrel monkeys and colors as share opposable thumbs with just a few other visual stimuli, it helps to know some subspecies species (F. R. Wilson, 1998). Grasping opens are color-blind whereas others aren’t, so an easy vast opportunities for interacting with envi- color discrimination for one may be impossible ronments: working with tools, writing, playing for the other. Early studies of maze learning musical instruments and wielding weapons, must be interpreted with caution because rats to mention just a few. Some are clearly more have keen olfactory sensitivity. If a maze isn’t benign than others. I can’t possibly provide an thoroughly cleaned between subjects, a rat’s exhaustive list of motor systems, but one other performance may depend on odor trails left by that demands mention is our vocal apparatus. other rats instead of what it learns on its own Different anatomical changes came together runs through the maze. Issues like these become to make human talk possible. These included even more crucial when a child’s behavior prob- enhanced articulation following from changes lems interpreted as products of contingencies in the control of lips and tongue and vocal turn out to depend instead on a sensory defi cit 30 • Part II. Behavior Without Learning

such as hearing loss or uncorrected vision. These sented in Figure 4–1, the grid search pattern was are all examples of sensing stimuli at a distance: eventually adopted as the primary recommenda- hearing and vision and smell. But let’s begin tion of the American Cancer Society. It’s unfor- with a contact sense, touch. tunate that it hasn’t remained so, because it has undoubtedly saved lives. Despite the data, all Touch: Breast Self-Examination and three search patterns are equally recommended. Signal Detection But any BSE at all is better than none. It’s also worth noting that men too are at risk of breast With our fi ngers we can sense heat and cold and cancer, though at a lower rate than women; their pressure and pain. Touch is of special interest tumors are typically detected at later stages of because much of what we feel depends not on their cancers, perhaps because they’re rarely passive contact but on moving our fi ngertips over informed about the value of doing their own our touched environment. Our touch receptors examinations. are densely packed there and are exceedingly Effective breast self-examination depends on sensitive to subtle differences. There is passive patterns of search and palpation . It illustrates touch sensitivity, but we need the moving fi nger how behavior analysts can extend research to detect the details of forms and textures. We with nonhuman organisms to areas of signifi - can’t just sit there; we must do something. This cant human concern (Pennypacker began his matters not only in feeling things around us but career with pigeon research); it reminds us that also in feeling parts of our own body. One exam- despite our privileged access to our own bodies ple is breast self-examination (BSE), where the we need to be taught much of what we know response is palpation of different breast areas about ourselves; and it also treats stimulus pro- and possible consequences are fi nding poten- cesses as behavior rather than as passive out- tially malignant nodes or lumps. comes of exposure to stimuli. It’s obvious we Breast self-examination has been studied can feel things only when we touch them, but extensively as a teachable discriminative skill we also hear things when we listen to them and (Pennypacker & Iwata, 1990). Early studies see things when we look at them. In any modal- emphasized the detectability of the lump as a ity, shifting attention from one thing to another function of various properties such as its size is behavior. It’s too easy to focus mainly on the or its depth within simulated tissue (Bloom et stimuli rather than the behavior. As this example al., 1982). As the work progressed, methods of illustrates, sensing is something we do. palpation and search patterns emerged as sig- A stimulus may be below threshold or too nifi cant concerns. For example, the movement weak to be detected. Two stimuli may be so close needed to detect something in the simulated together they can’t be told apart, i.e., their sepa- breast tissue depended on how close to the sur- ration may not exceed their difference threshold . face the target was and the search pattern had to But a threshold isn’t some absolute value. It var- be fi ne enough not to bypass the target, which ies from moment to moment and is best regarded could be anywhere in the tissue. Perhaps not as a distribution: the probability of detection as surprisingly, circular fi nger movements hori- a function of stimulus value. The mathematics zontal to the surface were more effective than of these probability distributions provides the pokes. Methods of palpation and patterns of basis for signal detection theory (D. M. Green & search were refi ned based on how well targets Swets, 1966; Swets, Dawes, & Monahan, 2000). were detected. A breast lump qualifi es as a stimulus, so signal Recommendations from the American Can- detection can be applied to it. cer Society at the time advocated a spiral pat- A signal-detection analysis assumes an tern of search, but analyses of area covered and observer responds or doesn’t respond to a signal. detection success demonstrated the superiority Responding to the signal is a hit and responding of a grid pattern. Based on data like those pre- if it wasn’t presented is a false alarm; not respond- Chapter 4. Motor and Sensory Systems • 31

Figure 4–1 Percent tissue examined in pretest breast self-examination (left) and after training with one of three search patterns illustrated by the sketches. Percentages of areas palpated relative to pretest data show strip or grid search (right) was superior to the concentric or radial methods for either cone or total area searches. (Adapted from Figures 13.10 and 13.11 in Pennypacker and Iwata, 1990). ing to the signal is a miss and not responding if it A hit is obviously better than a miss if there is wasn’t presented is a correct rejection. a lump; if not, a correct rejection is obviously bet- The possible outcomes for breast self-exami- ter than a false alarm. Once we know the propor- nation are summarized in Table 4–1. If a lump is tions of hits and of false alarms we can use them there and I say it is, that’s a hit; if I don’t detect it, to calculate two statistics. One, d΄ (d-prime), is that’s a miss . If it isn’t there and I say it is, that’s a an index of discrimination; it depends on hits false alarm; if it isn’t there and I say it isn’t, that’s and correct rejections relative to total responses. a correct rejection . It grows larger with increases in hits and cor- 32 • Part II. Behavior Without Learning

Table 4–1 Signal-Detection Contingencies in Breast Self-examination A lump is there No lump is there “Yes, a lump” Hit False Alarm (Correct Positive) (False Positive) “No lump” Miss Correct Rejection (False Negative) (Correct Negative) rect rejections and smaller with increases in Behavior analysts work with children, and misses and false alarms. A second statistic, bias, even beyond egregious cases like this example is an index of whether one type of response is they should be able to spot sensory issues. Do a more likely than the other. Both misses and false child’s behavior problems arise not from contin- alarms can be costly, though in different ways, gencies but from hearing loss missed in school and the relative costs will bias judgments in one screenings? A quick informal check is to see direction or the other. Misses may seem inher- whether you get different responses depending ently more important than false alarms, but on whether the child sees you while you’re talk- consider what it might be like if someone free of ing. Problems can also follow from visual defi - cancer was mistakenly told a biopsy had tested cits. The more you know about sensory systems, positive. Signal-detection analyses help clarify the better the job you’re likely to do in dealing such issues and have applications across a broad with such cases. range of sensory systems and diagnostic situa- Our auditory system consists of (i) the outer tions. ear , through which sound reaches the eardrum, (ii) the middle ear, where a series of three small Hearing : A Dual System bones transmits the vibrations of the eardrum to the round window of (iii) the inner ear, or cochlea, At a meeting of the Association for Behav- consisting of a spiral tube divided by the basilar ior Analysis International a few years ago I membrane. The vibrations of the round window dropped in on an evening social. As I stood near become movement along the basilar membrane , the entrance a young woman circled around which is displaced more at its wide end by low the dance fl oor with a young child, perhaps six frequencies and more at its narrow end by high months old, in her arms. The music was very frequencies. These waves on the basilar mem- loud. On top of what I knew of the auditory sys- brane in turn activate hair cells that stimulate tem, I’d also had a teaching assistant with perma- the fi ring of cells in the auditory nerves (von nent hearing loss from attending rock concerts. Békésy, 1960). The outer hair cells, so named As the dancer came around again after passing because they run along the basilar membrane on very close to the speakers I waved to her. She the outside of the cochlear spiral, are responsible stopped by me and I told her the ears of such a for the transduction of low-intensity sounds; the young child are very fragile; at close range the inner hair cells kick in at higher intensities (but sound level produced by the speakers could eas- getting into the details of this complex system ily produce permanent damage. She told me it would take us too far afi eld). was none of my business and went right back to The dual system of outer and inner hair cells the dancing. I didn’t see what else I could do and is responsible for a phenomenon called recruit- just left. Having done nothing more to protect ment . Some hearing issues, especially with aging, the child then, I’ve since occasionally introduced are caused by loss of outer hair cell function. Peo- presentations with this anecdote and here I’m ple with that problem don’t hear quiet sounds, continuing to atone for my lapse. but once the sound level gets high enough it sounds just as loud as it would have before the Chapter 4. Motor and Sensory Systems • 33 hearing loss. I encountered the phenomenon ows. The closer an opacity to the retina, the with my mother-in-law. When she fi nally heard sharper the shadow, which will be visible only a question asked repeated at successively louder in the eye with the opacity. The blood I saw was levels, she complained we were shouting at her. in sharp focus, so it must have been very close The solution was straight-forward: it was best to my retina. Furthermore, images on the retina to be where she could see you when you talked, are inverted: light from above falls on the lower instead of standing behind her or to the side. retina and light from below falls on the upper. Those working with geriatric populations need The blood was running up the wall so it must to understand recruitment. have been moving downward in my eye. Visual phenomena such as fl oaters , the shad- Vision : Blood on the Walls, Dust Storms ows on the retina created by material drifting in and Haloes the optic humors, may be precursors of retinal detachment or other serious problems that must It can be useful to know about the visual system. Some be dealt with promptly. I watched the blood in years ago I heaved a heavy bag with a shoulder my eye break into fi laments and go out of focus strap from my shoulder to set it down and saw a as it dispersed while I was on the phone to my stream of blood running up the far wall. I moved ophthalmologist’s offi ce to set up an appoint- my head and the blood moved with it. I covered ment. On the drive there, against a bright sky my left and then my right eye and saw the blood and snow on the ground, I saw clouds of dark only with the right. I concluded something was particles. I covered one eye, then the other; they happening in that eye. As a graduate student I were visible only to my right eye. Their swirl- had spent some time studying vision, so when ing was agitated when the road was rough and I experienced this strange vision I was well pre- calmed when it was smooth. They were the pared to interpret it (Catania, 2007);. shadows on my retina of the blood cells of my The retina is the photosensitive lining at the intraocular hemorrhage, stirred up by the vibra- back of the eyeball. It includes not only the light- tions created as our tires met the road. If I hadn’t sensitive visual receptors, rods and cones, but known better I’d have thought they were a dust also blood supply and other structures. Light storm. entering the eye passes through some of these I learned the bleed was probably from a torn structures to reach the rods and cones. Like retinal capillary in or near the fovea, the retinal hearing, human vision involves a dual system. area of sharpest acuity. The gelatinous vitre- The cones , concentrated in the macula or central ous capsule that fi lls the inside of the eyeball retina, serve color vision and the sharp acuity of between the lens and the retina usually shrinks the fovea . The rods , distributed in the periphery; with age. Sometimes the capsule and the retina are much more sensitive than the cones in that adhere to each other. As the shrinkage separates they respond to much dimmer levels of light, the capsule from the retina, a sharp movement but they’re not much good for detail and color may cause a small tear at the site of the adhe- vision. The foveal cones aren’t sensitive enough sion. My ophthalmologist couldn’t fi nd it. The to work in dim light but the peripheral rods are, bleeding had loomed large for me, but its capil- which is why colors fade out in dim light and lary source was tiny. why you can sometimes see things at night only Within a few days that visual event was fol- by looking slightly away from them. There’s lowed by another. I began seeing a dark ring good reason to be wary when an eye-witness around my central area of vision, again in my who saw a jacket only at some distance at night right eye. When I looked at someone against a claims that it was red. bright background, the person seemed to wear When light enters the eye, passing through a halo . Over a few weeks the halo shrank and the lens on its way to the retina, opacities along blurred and moved off center. Though now the way block the light, creating retinal shad- quite faint it remains visible as I write this, more 34 • Part II. Behavior Without Learning

than fi fteen years later. It was most likely caused and wavelength (color). Different retinal recep- by another retinal detachment. Near the back of tors are sensitive to different wavelengths, the eyeball, somewhat removed from the fovea corresponding to red, blue, and yellow/green in the direction of the nose, is the optic disk, classes. The spectral range we see arises from where neural processes from the rods and cones different mixes of these three classes. Those come together to leave the eyeball as the begin- with defi ciencies in their receptors may be ning of the optic nerve. The optic disk is a blind color-blind. Most color-blindness is in men, but spot, with neither rods nor cones. You can locate my mother was color-blind. She didn’t see dif- it in your visual fi eld by closing one eye while ferences in colors at the violet end of the spec- looking straight ahead and moving your fi nger- trum, so browns and purples looked alike to tip along the horizontal on the side of your open her. She sometimes wore unusual outfi ts. I’ve eye until you fi nd a place where it disappears. had trouble with colors at that end too, though A small ring at the boundary between the optic I’m not color-blind. I fi nally realized my prob- disk and the rest of the retina sometimes breaks lem: I’d picked up some of her color confusions away. Mine seems to have settled down around because I’d learned my color names from her. my fovea for a while. The point is that some sensory problems arise Symptoms like mine were improbable, espe- not from physiological sources but rather from cially in combination. They’re suffi ciently few behavioral ones. and far between that an ophthalmologist might I encountered different color problems later. encounter them less than once a decade. I’d As we age, our lenses get cataracts, which means seen blood on the walls, dust storms and haloes. they gradually get yellow and cloudy. I needed What if I hadn’t known enough about vision to cataract surgery to correct the loss of focus. After interpret them and had lived when no knowl- my fi rst eye was done, I looked at the sky fi rst edgeable others were around to consult? Simi- with one eye and then the other. I saw a bright lar visions must have appeared to many people blue sky with the new lens, but to the eye with throughout history. What stories they must have the old one it was a smoggy day. The yellowing told! It’s a good bet some have come down to us. lens had been blocking blue light, which I saw But I know enough to be skeptical about claims vividly with the new clear one. The yellowing they should be taken as messages or signs of the had come on gradually over years, so I hadn’t paranormal. noticed my losses in the blue region of the spec- My interpretation of these phenomena trum. Now I can again tell navy blue from black. depended on getting the direction right. We see Not only have eyes evolved many times in things because light enters our eyes, traveling many species. We now know that the same gene either directly from light sources or as refl ected is the source of all eyes across the entire ani- from objects, not the other way around. The mal kingdom, from worms and fl ies and squid early Greek philosophers debated whether sight and crabs and fi sh to mammals like us (Carroll, depended on light from objects reaching the eye 2006); all of these eyes descended from a com- or on emanations from the eye making contact mon ancestor that produced the fi rst photosensi- with objects. The confusion is less likely in hear- tive cells. In vertebrates, eyes typically come in ing, though the echolocation of bats involves pairs, but with different arrangements. They’re both directions: the bat’s generate high-fre- usually back-to-back in likely prey, mainly her- quency sounds and then locate their prey based bivores, facing out to each side as in deer and on the echoes. With either vision or hearing, cows and robins; they face forward in likely those who get the direction wrong won’t be able predators, mainly carnivores and omnivores, to understand how the sensory system works. thereby enabling depth perception via binocu- Sound varies in intensity (loudness) and lar vision as in lions and wolves and eagles and wavelength (pitch). At each location in a visual humans. Figure 4–2 illustrates the back-to-back array, light also varies in intensity (brightness) arrangement as it occurs in pigeons . Chapter 4. Motor and Sensory Systems • 35

Each eye has a central fovea; these foveae The question was whether visual discrimina- receive light coming from either side of the tions learned with one eye transferred over to pigeon’s head. Their eyes also have a sensitive the other. I had trouble teaching discriminations region at the rear of the eyeball, receiving light in which the stimuli appeared on either side of from the front; in some birds this region is a sec- the pigeon’s head as it pecked a key in front of ond fovea. The three areas of acute vision, one in it. It took a while to recognize that the stimuli front and the two on either side, are sometimes were too close. With vision stimuli placed to the called the visual trident (Catania, 1964). Except at pigeon’s sides, things go easier if they are far- the back of the head, their vision extends almost ther away. all the way around, but it isn’t panoramic. Acu- Another complication in studying interocu- ity falls off with distance from the foveae or sen- lar transfer of learning is that the pigeon’s two sitive regions, just as in the human eye. eyes are separated only by a thin septum of bone. Not only is their vision acute in these three If in a dim room you shine a light into one eye, directions. In the front they’re near-sighted but the other will glow red from the light passing to the sides they’re far-sighted: a good arrange- through the septum. We can worry about things ment given they eat food close to them, in front that go in one ear and out the other in humans, of the beak, but must look out for predators but in pigeons visual stimuli can literally go in who can appear in the distance from different one eye and out the other. Interocular transfer directions. I learned about these features of might occur simply because some of the light to pigeon vision the hard way, when I undertook one eye directly stimulates the other one. an experiment on their vision (Catania, 1965). Their optic nerves cross over so each connects Proprioception and Biofeedback to the opposite hemisphere of the pigeon brain. Let’s skip over smell and taste and close with one more system, proprioception . Stimulation at the level of our own muscles and joints is essential to balance and locomotion and other interactions with the world. When I was an undergraduate, I served as a subject for Ralph Hefferline, a for- mer gestalt psychotherapist who had switched to studying small-scale muscle movements as a precursor of what later became known as biofeed- back (Hefferline, 1958; Hefferline & Perera, 1963). Biofeedback provides meter readings or other visible correlates of your own muscle move- ments or other physiological processes, helping you to see what you may otherwise have been unaware you were doing. Equipment in those days was sensitive to ambient currents from nearby wiring, so Heffer- line’s subjects sat inside an electrically shielded cage. I watched a meter that showed the activ- ity of my masseter muscle: the harder I clenched my teeth, the more the needle moved. My task was to relax my jaw to get the needle down as low as I could. I’d had some problem with brux- Figure 4–2 Schematic of the pigeon’s visual trident, its three fi elds of acute vision. Details in text. (Adapted ism (jaw-clenching); after brief sessions of this from Catania, 1964, Figure 1) relaxation training, my bruxism eased up. The 36 • Part II. Behavior Without Learning biofeedback procedure had taught me to dis- inhibitory interactions are common across dif- criminate properties of my own behavior to ferent sensory systems (Ratliff, 1965; von Békésy, which I’d previously been indifferent. Much is 1967). They’re especially relevant to us because shaped by natural contingencies, but here again, similar interactions occur within behavior sys- as with breast self-examination, we see that tems, as when responding is reinforced in the sometimes we need help in learning things even presence of stimuli within only part of a spatial about our own bodies. stimulus array and differences in responding are enhanced at the boundary between the two parts Sensory Gradients and Inhibitory (Catania & Gill, 1964). Interactions Figure 4–3 illustrates how inhibition works in the context of interactions among the rates of Whether in the skin, the ear or the eye, individ- fi ring of receptors in the visual system, shown ual receptors respond to stimulation. As their across a series of graphs. The top section of each activities trigger the fi ring of sensory nerve cells graph depicts an array of ten photoreceptors the input from single activated points is trans- with a light level for each ranging from zero to formed to patterned interactions, so that more nine. In 1a, for example, the third and seventh centrally in the nervous system cells fi re based cells (C and G) are lit at a level of 8 and the rest at on whether the stimulus includes some confi gu- a level of 0. The graph below displays the activ- ration, such as a moving edge (Hubel & Wiesel, ity of each cell as a rate of fi ring on an arbitrary 2005). The beginnings of the patterning occur scale. In 1a, C and G show substantial activity because the activity passed on by each receptor and the other cells remain inactive. In 1b through isn’t just excited by stimulation; it’s also inhib- 1d, the light level is held constant at 8 for cell C ited by the activity of neighboring cells. These but the light is moved closer, to cell F (1b), to

Figure 4–3 Firing rates of a hypothetical array of ten photoreceptors given various light levels, shown by the numbers above each array. Graphs 1a through 1d: reduction of one cell’s activity by activity of a neighboring cell increases the closer together they are (1a through 1d). Graphs 2a through 2c: higher rates at the edges when a cell group is lit, because those in the middle are inhibited by neighbors on both sides. Graph 3: amplifi ed diff erence at a boundary between cells lit at one level and those lit at another. Chapter 4. Motor and Sensory Systems • 37 cell E (1c) and fi nally to the adjacent cell D (1d). cell relative to what it would have done if lit all The closer together the lit cells, the lower their by itself. rates of fi ring; the fi ring of each cell reduces the Another effect, not shown, is that the initial rate of fi ring of the other. This mutual reduction fi ring produced by lighting a cell is typically is called reciprocal inhibition (Hartline & Ratliff, substantially higher than its later fi ring under 1957). continued light (Hartline, Wagner, & Ratliff, Graphs 2a through 2c show effects of light- 1956). This may be a result of inhibitory effects ing groups of two, four or six adjacent cells. With on the cell from its own activity. More impor- two cells, each is equally active. With four or six, tant, these interactions enhance differential the end cells are more active than the ones in the responding to temporal boundaries as well as middle; cell activity on the end is inhibited only spatial ones. Effects like these of are great gen- by neighbors on one side, whereas in the middle erality across a range of species, from the visual it’s inhibited by neighbors on both sides (Ratliff systems of horseshoe crabs, fl ies, and fi sh to & Hartline, 1959). The enhanced activity differ- those of people, and from the visual system to ence between unlit and lit cells at the boundary is hearing and touch and other senses (von Békésy, a contrast effect. All these differences are created 1967). Behavior, too, is a biological system, so we solely by inhibition, which reduces the fi ring of a should expect behavioral units to interact in cor- responding ways, and they do. Chapter 5 Elicited and Emitted Behavior

…virtually all of the organized maternally oriented behavior of most infant animals can be shown to be heavily infl uenced by experienced events —Hall and Oppenheim (1987)

Presenting a stimulus is a basic way to change Descartes’ concept of the refl ex captured behavior. If I’m standing and talking, for exam- the fact that behavior is sometimes caused by ple, a sudden loud noise will probably shut me environmental events, as when I quickly pull up and produce the change in my posture called my hand back upon touching an open fl ame. a startle reaction. This reliable relation between a Eventually physiologists turned their attention stimulus and the behavior it produces has been to the mechanism of such behavioral relations called a refl ex. The vocabulary began with René and explored the components of the refl ex arc, Descartes , a seventeenth-century French philos- the path from the original sensory impact of opher. He was familiar with hydraulic devices the stimulus through the central nervous sys- constructed to amuse visitors in the royal gar- tem and then back to the muscular or glandu- dens of France. Stepping on a concealed trigger lar system within which the response occurred. released a fl ow of water that made statues move. Analyses became more and more sophisticated Descartes saw a similarity between such devices (Sherrington, 1906), and eventually the condi- and behavior. According to Descartes, stimuli tioned-refl ex concepts of Pavlov (1927) and the were comparable to the garden visitors who, related behaviorism of Watson (1919) treated the refl ex as the basic unit of behavior. entering into one of the grottoes containing many fountains, themselves cause, with- out knowing it, the movements which they THE REFLEX: ELICITATION witness. For in entering they necessarily tread on certain tiles or plates, which are The startle reaction and the withdrawal of a so disposed that if they approach a bathing hand from a fl ame are just two examples. We Diana, they cause her to hide in the rose- can add the knee jerk produced by a tap on the bushes, and if they try to follow her, they patellar tendon; salivation caused by food in the cause a Neptune to come forward to meet mouth; the postural adjustments triggered by them threatening them with his trident. an abrupt loss of support. They have the com- (Descartes, translated in Fearing, 1930, pp. mon feature that some stimulus reliably pro- 20–21) duces some response. In such circumstances, we say the stimulus elicits the response, or the Just as a step on a concealed plate triggered response is elicited by the stimulus; the stimu- the movement of a statue, a stimulus triggers a lus is an eliciting stimulus, and the response is response. For Descartes, the pipes and water in an elicited response, But we never use elicited those statuary systems played roles similar to the other way around, to refer to the effect of a nerves and animal spirits in living organisms. response on the occurrence of a stimulus.

38 Chapter 5. Elicited and Emitted Behavior • 43

Once that distinction had been made, qualifi - adjustments of keeping balance (Gallistel, 1980). cations were added. In particular, some argued Human behavior with sometimes ambiguous that instrumental or operant behavior consisted antecents includes laughter, yawning, tickle and of skeletal responses, such as movements of the the shedding of tears (Provine, 2012). limbs, whereas refl ex or respondent behavior consisted of autonomic responses, such as glan- Successive Elicitations: Habituation dular secretions. This was also seen as parallel- ing the traditional distinction between voluntary A complication with stimulus presentations and involuntary action. Such distinctions have is that two different presentations of the same since been seriously questioned. stimulus may have different effects. I may startle For example, the crouching posture elicited much more to the fi rst lightning fl ash in a thun- by a sudden loud noise in the startle refl ex is a derstorm than to later fl ashes. Furthermore, the motor response best described as involuntary. effects of stimuli may depend on how quickly But swallowing seems voluntary and yet also they follow each other. If I’m peeling onions, the involves a refl ex relation. It’s elicited by stimu- tears elicited by the present onion may depend lation of the back of the throat, which is why on whether I began working on it right after you can’t swallow if your mouth is dry and you fi nishing the last one or just after taking a long have nothing in it to pass back to that area to break from peeling. And in another effect, called stimulate it. On the other hand, driving cer- summation, a stimulus below threshold in elicit- tainly seems both voluntary and operant. Yet ing a response if presented once may become an an experienced driver sitting in the passenger effective elicitor if presented repeatedly. A sin- seat of a car may involuntarily press hard on the gle whiff of pollen might not be enough to make fl oor even though there is no brake pedal there me sneeze, but several whiffs might do it, even when something suddenly looms up ahead if the pollen in each whiff is about the same. In on the road. We have here all of the possibili- other words, elicited responding often depends ties: whether behavior is operant or respon- on the number of stimulations and their separa- dent, we can identify examples that seem either tion in time. voluntary or involuntary. In other words, the everyday distinction between voluntary and Habituation. The startle reaction is produced involuntary actions has nothing to do with our by an unexpected event such as a lightning fl ash distinction between operant and respondent or a sudden loud noise. Even without other behavior. Instead, the most important determi- events to signal it, a repeated loud noise usu- nant of whether we call our behavior voluntary ally produces successively smaller startle reac- or involuntary may be whether we can identify tions, until eventually there is no startle at all. the source of our actions. Many stimuli elicit responses called orienting or Skeletal responses can be elicited and auto- observing responses; for example, a dog pricks nomic responses can be emitted. It’s important up its ears in response to a novel sound or begins to maintain the distinction between elicited and sniffi ng in response to an unusual odor. As these emitted responding. But the same response may stimuli recur, the dog’s responding decreases; be sometimes elicited and sometimes emitted; it occurs with smaller magnitude and longer we can’t classify responding effectively into latency, perhaps even vanishing completely. these two categories based on physiological This reduction in responding with repeated criteria such as the difference between skeletal stimuli is called habituation. Habituation occurs and autonomic responses. Some other types of with responses ranging from distress calls of unlearned behavior don’t fall easily into such birds when a predator passes overhead to con- classifi cation, such as those with the character- tractions in earthworms exposed to light. istics of oscillators, as in circadian rhythms, or Habituation is central to studies of whether of servomechanisms, as in the compensating preverbal children can distinguish features of 44 • Part II. Behavior Without Learning

spoken language such as the basic speech sounds successive one felt stronger so it became progres- or phonemes of the languages of their caregiv- sively more diffi cult not to pull my hand away ers (Eimas & Miller, 1992; Ramus, 2002). With even though shock level stayed constant. My nonnutritive sucking as a response, the infant response to the shock had potentiated. is given a pacifi er connected to a device that records the pressure exerted while sucking. If the Time Since the Last Eliciting Stimulus. If the infant hears a voice speaking a consonant, say b, stimulus is absent for a while after habituation the infant will typically stop or slow its sucking or potentiation, response probability returns to briefl y. If b is repeated, it has less and less effect. earlier values. For example, my startle to loud In other words, the response to b habituates. noise may diminish or even disappear after Now the voice says p instead of b. The pho- several noises in succession, but after hours nemes b and p differ in that the former is voiced of silence I’m likely to startle again to the next (the vocal cords vibrate) whereas the latter isn’t. noise. Habituation and potentiation aren’t per- Given an infant several weeks old raised by manent; responding returns to earlier levels as English-speaking caregivers, the fi rst p will ordi- time passes. narily produce a pause in sucking, even though the response to b has habituated (this might be Patterning of Behavior in Time called dishabituation). The different responses to b and p tell us the infant can distinguish between Presenting a stimulus may determine the the two consonants even though not yet able to responding that occurs over an extended time. produce them. For example, if we give a rat food it eats but once Not so, however, if the infant has been raised it fi nishes it typically drinks if water is available. by caregivers who speak a language in which This relation between eating and drinking is the b and p distinction is unimportant. Consider so strong that by repeatedly delivering food in a child raised in an Arabic household, where a small amounts we can make the rat drink sev- single voiced consonant somewhat like the Eng- eral times its ordinary daily water ration. This lish b has no corresponding voiceless p. Once increase is called polydipsia , an example of adjunc- this infant’s pause in sucking has habituated to tive behavior, behavior in which one response the sound b, changing to p makes no difference. reliably accompanies some other response (Falk, There is no evidence the infant distinguishes 1977; Wetherington, 1982). between them. This research uses habituation to Polydipsia follows at least in part from the teach us that infants learn about the sound fea- rat’s normal feeding and drinking pattern. With tures of the languages they hear around them food and water freely available, the rat ordinar- long before they become skilled in producing ily takes a few large meals daily, drinking after those features. each one. If we force it to take many meals by Potentiation. Stimuli sometimes have the delivering food in small portions every few opposite effect. For example, electric shocks elicit minutes, the rat still drinks after each meal but squealing in rats; if several shocks are delivered, doesn’t reduce each drink enough to compen- later ones produce more responding than earlier sate for the more frequent drinking. ones (Badia, Suter, & Lewis, 1966). This is called Respondent procedures called temporal con- potentiation, which is most likely to occur with ditioning can also impose temporal structure on stimuli regarded as aversive. For example, as an behavior. When a stimulus is presented at regu- undergraduate I was a subject in class demon- lar intervals, responding may become patterned strations involving electric shock (they might not within these intervals, with those most closely have passed contemporary institutional human- related to the behavior occasioned by the stimu- subject review boards). One involved holding lus become more likely as time passes within the my hand against an electrode as shocks were interval. For example, when food is presented to delivered. The fi rst shock just tingled, but each a pigeon by operating a feeder at regular inter- Chapter 5. Elicited and Emitted Behavior • 45 vals, responses such as preening and pacing a possible role for exercise or practice became and turning occurred early, but pecks directed neglected. toward the feeder or the fl oor and orientation The Russian physiologist, Ivan Sechenov, toward the feeder became more likely as next was an early advocate of repetition, arguing that feeder time approached (Staddon & Simmelhag, refl ex integrity is “maintained by the frequent 1971). The early responding, interim behavior, repetition of the refl ex” (Sechenov, 1863/1965, varied from one pigeon to another and at dif- pp.28–29) and, further, that the response may ferent times for a given pigeon, but the peck- become independent of the effects of eliciting ing and feeder-related behavior was consistent stimuli. In infants, for example, sucking is elic- across pigeons. The later behavior had some- ited by stimuli; later these stimuli no longer elicit thing in common with the responses produced the response, but through adulthood individu- by the stimulus presentations. Repeated stim- als remain able to produce the response even in uli not only elicit responses but at other times their absence (W. G. Hall & Oppenheim, 1987; may produce behavior related to the elicited Schoenfeld, 1966). responding. Not all responses begin with elicitation. Some of the earliest in an organism’s lifetime occur spontaneously, in the absence of identifi able FROM ELICITED TO EMITTED eliciting stimuli. For example, inside its egg the BEHAVIOR embryo chick makes uncoordinated movements of its limbs. These may prevent the developing One problem in tackling this topic is the very bones from becoming fi xed in their sockets and word response. It implicitly suggests a response may modify the form of growing bones and con- to something. As a derivative of response without nective tissue. Later in development eliciting the implication of responding to something, I’m effects appear, perhaps simply as the embryo’s sorely tempted to introduce the term sponse (cf. sensory apparatus matures. The progression Provine, 1988), but it would be presumptuous from spontaneous to elicited responding may be of me to do so because the term has little prec- summarized by saying of the embryos that “they edent in the literature of behavior and learning. ‘act’ before they ‘react’” (Provine, 1976, p. 210). Nevertheless, if the distinction between elicited Stimuli become more important later, as when and emitted responding sometimes seems too the chick’s rotating movements in pecking its esoteric, it might be helpful to try a translation way out of its shell during hatching are affected in which elicited responding becomes responding by whether the chick continues to encounter while emitted responding becomes sponding. intact portions of the shell.

The Role of Exercise Stimulus Presentations in Imprinting

“Practice makes perfect” is a familiar saying When a duckling hatches, the fi rst moving thing about the role of repetition in behavior. Before it ordinarily sees is its mother. Even on its fi rst consequences were appreciated the mere rep- day outside the egg the duckling will probably etition of responding, without regard to its stay close to her. But if the mother is gone and consequences, was believed suffi cient to main- the duckling fi rst sees something else in motion, tain behavior. Early accounts treated effects of such as a human, the duckling will behave response repetition, described as basic to learn- toward this stimulus as it otherwise would have ing in laws of exercise or practice . The laws were toward its mother. Such stimuli are said to be usually ambiguous on whether it mattered if imprinted or, in a fi gurative sense, stamped into the repeated response were elicited or emit- the duckling (Lorenz, 1937). ted. As the study of learning turned to rein- Imprinting has been demonstrated with forcement and signaling functions of stimuli, stimuli ranging from real and model birds to 46 • Part II. Behavior Without Learning electric trains; some work better than others. that it could elicit following, but rather that it had Imprinting usually develops within a critical become important to the duckling. Depending period of one or a few days, so it may not occur at on the consequences, therefore, it could reinforce all if it doesn’t during this time. A complication varied responses: following, pecking, standing is that fear of novel stimuli develops toward the still. In natural environments, the duckling’s fol- end of the critical period (Hoffman, 1996). As lowing ordinarily keeps it close to the imprinted the birds get older, they move away from novel stimulus (usually its mother), but in a labora- stimuli and make distress calls. Thus, older tory ducklings can learn other responses if they birds don’t ordinarily stay near novel stimuli instead have the important consequence of keep- long enough for imprinting, which would hap- ing the imprinted stimulus close. pen otherwise. In imprinting, the initial presentations of the In any case, the duckling begins behaving in to-be-imprinted stimulus don’t change response signifi cant ways to a stimulus, whether mother probabilities. Rather, the presentations are estab- duck, human or some arbitrary moving object, lishing operations; they change the signifi cance if the stimulus is introduced under appropriate of the stimulus. The imprinted stimulus acquires conditions and at appropriate times. The typical its signifi cance simply by being presented under response is following the imprinted stimulus as appropriate circumstances. Imprinted stimuli it moves, and it’s sometimes said the duckling’s don’t elicit following. They begin as stimuli following is elicited by the imprinted stimulus. toward which the duckling is relatively indiffer- But here the term elicitation is misleading. If the ent and end as stimuli that can serve as reinforc- imprinted stimulus is the mother duck, the duck- ers and shape the duckling’s behavior. The point ling follows her, but how does the imprinted should have been obvious to early researchers stimulus produce following? on imprinting. In natural environments, swim- When the duckling walks toward the mother, ming replaces walking when the duckling fol- it fi nds itself closer; when it walks from her, it lows its mother into a body of water. If walking fi nds itself farther away. In other words, chang- had been mere elicited behavior, it shouldn’t ing the distance from its mother is a natural con- have done so. sequence of the duckling’s walking. If closeness Mixing up eliciting and establishing effects to the mother is important to it, it’s no surprise it can have profound clinical implications. For walks toward her rather than away from her. It example, interpreting a hospitalized child’s follows that if we change the duckling’s world so problem behavior as elicited behavior when the mother’s closeness requires a response other instead it arises from reinforcement contin- than walking, the walking should be replaced by gencies might prevent appropriate treatment that response, as in an experiment by Peterson options from being considered. Misdiagnosis can (1960). go either way. For example, if problem behav- A dark compartment on one side of a window ior is produced by eliciting stimuli, perhaps for contained a moving imprinted stimulus. On the neurological reasons, interpreting it as shaped other side the duckling could make a response by reinforcement contingencies might similarly that lit up the dark side. The duckling did so lead to ineffective treatment. It can get even more even when the response was incompatible with tricky. In multiple causation, eliciting stimuli following: pecking at a disk on the wall or stand- and reinforcement contingencies may operate ing still on a platform near the window. The criti- at the same time, so identifying the role of one cal property of the imprinted stimulus wasn’t shouldn’t rule out assessments of the other. Chapter 7 Reinforcers as Opportunities for Behavior

The reward of a thing well done is to have done it. — Ralph Waldo Emerson

We identify reinforcers by their effects. Without Many events regarded as unconditioned making a stimulus a consequence of respond- reinforcers, such as food, water and sexual con- ing we can’t say if it will serve as a reinforcer. tact, have obvious biological signifi cance. But But even familiar reinforcers like food vary in reinforcers aren’t limited to such events. For effect depending on deprivation. Food as a con- example, sensory stimuli such as fl ashing lights sequence of lever presses might not do much can powerfully reinforce the behavior of some to the rat’s pressing if it’s already had plenty to children on the autism spectrum (Ferrari & Har- eat. Reinforcers are inevitably oversimplifi ed if ris, 1981), and when intellectually challenged we treat them merely as stimuli. A fundamental children are briefl y deprived of music or social property of a reinforcing situation is the respond- praise, these events may become more effective ing for which it sets an occasion. When we study reinforcers of other behavior, such as operat- motivation or drive we’re concerned with what ing a switch; conversely, when they are satiated makes consequences more or less effective as with music or social praise, the effectiveness of reinforcers or as punishers. In the taxonomy of these events as reinforcers decreases (Vollmer & establishing operations, deprivation and satia- Iwata, 1991). tion are important ways to change the effective- ness of stimuli as reinforcers or punishers, but not the only ones. THE RELATIVITY OF Reinforcers are sometimes distinguished REINFORCEMENT based on the types of operations that established them. For example, a conditional reinforcer is one Reinforcers exist in great variety. Some are con- that has become effective by virtue of its relation sumed. Others aren’t. Some appear effective on to some other reinforcer; the light that comes on the organism’s fi rst experience with them. Others when a pigeon feeder is operated will eventu- acquire their reinforcing properties during the ally become a conditioned reinforcer because of organism’s lifetime. No common physical prop- its relation to food. The establishing operation erties allow us to identify reinforcers indepen- here is arranging the relation between the stim- dently of their effects on behavior. For example, uli: setting up the feeder so food deliveries are it is diffi cult for us to say what it is about teacher accompanied by the light. In human behavior, attention that reinforces student behavior, but money often works as a conditional reinforcer, we do know that when a teacher moves to a fi rst- and because it’s related to so many other possi- grader’s desk with perhaps a pat on the shoulder ble reinforcers (all the things we’ve bought with or an encouraging comment contingent on the it), it’s sometimes called a generalized conditional student’s reading, the student’s study behavior reinforcer. A reinforcer that doesn’t depend on a often increases and behavior such as dawdling relation to other reinforcers is called an uncondi- decreases (R. V. Hall, Lund, & Jackson, 1968). We tional reinforcer. also know that changes in student behavior can 58 66 • Part III. Learning Without Words: Consequences duration or intensity of stimulation. These light couldn’t easily deal with effects of schedules of and dark adaptations depend on the depletion reinforcement, in which not every response is and recovery of photosensitive pigments dur- reinforced. It may be, however, that he dropped ing periods of light and dark and can be studied it prematurely (Catania, 2005d; Killeen, 1988). by examining single cells. But the fi ring of each photoreceptor will also inhibit and be inhibited Activation and Coupling by the fi ring of neighboring cells. These interac- tions will determine patterns of fi rings that can Are there other ways to analyze what happens be studied only by examining the activity of cells in reinforcement? A survey of the many quanti- in the context of the activity of the cells that sur- tative treatments of reinforcement and behavior round them. is beyond the scope of this chapter, but some If the effects of reinforcers on responses are provide useful hints about how to proceed (e.g., a variety of stimulation, then that stimulation Killeen, 1992; Killeen & Bizo, 1998; Nevin, 1992, might produce more or less resistance to change, 2015; Nevin & Grace, 2000). Reinforcers produce but the specifi c rate of responding produced by multiple effects. Converging evidence suggests the contingencies will depend not only on the that reinforcement outcomes are the product of reinforcers delivered but also by the interactions two different effects: general activation or arousal, of this response with other responses for which as for example when a rat that had not been eat- reinforcers have been arranged, or, in other ing is now able to eat again; and a more specifi c words, by the context in which the responding effect of the response-reinforcer contingency, a occurs. Thus, just as visual adaptations depend coupling between response and reinforcer (e.g., on the direct stimulation of receptors whereas Killeen, 1994; Killeen & Sitomer, 2003). As a rough patterns of fi ring of visual receptor cells depend approximation, we can say that the total activa- on interactions among them, so also resistance tion of responding over its reinforcement history to change depends on direct effects of rein- determines its resistance to change, whereas its forcers whereas response rates and probabili- rate is determined by its context, including such ties depend on interactions among reinforced current variables as the reinforcement of other response classes. response classes (cf. Nevin, McLean, & Grace, Another metaphor for the effects of rein- 2001). To this point our analyses have mainly forcers on behavior is that reinforcers build up shown how basic processes can enter into com- a reserve of potential responding, with subse- plex behavior, as in attributing imprinting to quent response rate proportional to the size of acquired reinforcers rather than to elicited behav- the reserve and with the emission of responses ior. But decomposing reinforcement into acti- depleting it. Skinner (1938) called his the refl ex vation and coupling shows that even the basic reserve but later dropped the idea because it processes themselves can be subject to analysis. Chapter 8 Reinforcement, Free Reinforcement, and Extinction

When reinforcement was again introduced... the animal did not have to relearn the whole sequence, because the whole sequence had not been extinguished — Murray Sidman (1960)

The consequences of many responses remain rea- ing a bare wrist isn’t an effective reinforcer. The sonably constant throughout life. For example, decrease in responding during extinction isn’t a we usually touch the objects we reach for and we special process requiring a separate treatment; usually get from one fl oor to another when we it’s one of the properties of reinforcement. climb a fl ight of stairs. But for other responses, consequences change. Responses reinforced during childhood may no longer be reinforced EXTINCTION VERSUS in adulthood. Educational systems may arrange INHIBITION consequences such as praise or grades for solv- ing arithmetic problems or answering factual Extinction was long assumed to actively sup- questions, but sooner or later these artifi cial con- press responding. It was said to have inhibitory sequences are discontinued, with the hope that effects, in contrast to assumed excitatory effects more natural consequences will maintain the of reinforcement. This treatment went back to responses when the student moves on to other a language applied to data from Pavlov’s con- settings. When a response is reinforced, its prob- ditioning experiments. Skinner’s operant lan- ability increases. But the increase isn’t perma- guage depended in part on escaping from the nent. Responding decreases to its earlier levels implications of those early usages (Skinner, when reinforcement is discontinued. 1938, pp. 96–102). Once those usages were car- The discontinuation of reinforcement is ried over to the language of consequences, they called extinction ; when this reduces responding were kept because they seemed consistent with to its earlier level the responding is said to be other effects often accompanying extinction. extinguished. This mainly demonstrates that the Thus, textbooks tended to devote separate chap- effects of reinforcement are temporary. Respond- ters to reinforcement and extinction rather than ing is maintained while reinforcement continues treating them as two sides of a single phenom- but not after it stops. If extinction didn’t occur, enon. the effects of reinforcement would be perma- One such effect was spontaneous recovery. In nent. Any responding engendered by reinforce- extinction, responding decreases over time. But ment would last through a lifetime. Clearly that response rate at the start of one extinction ses- can’t be generally so. For example, if you wear sion is usually higher than it was at the end of a watch you probably often turn your wrist to the last one. Responding was said to have recov- look at it; the consequence of looking is seeing ered spontaneously from inhibition built up by the time. But if you stop wearing the watch for the end of the last session; this inhibition was some reason, you’ll eventually stop looking; see- assumed to increase within sessions, actively

67 68 • Part III. Learning Without Words: Consequences suppressing responding, and to dissipate SIDE EFFECTS OF EXTINCTION between sessions. Such phenomena were taken to mean that responding reduced by extinction Discontinuing reinforcement has not one but two was somehow “there all the time but inhib- effects: (i) the contingency between responses ited” (Reid, 1958). The trouble was that when a and reinforcers ends, so (ii) reinforcers are no response was said to be inhibited in extinction, longer delivered. In this context, the term con- there was no way to measure what was doing tingency simply describes the consequences of the inhibiting. responding; here it is the effect of a response on It isn’t necessary to assume that extinction stimulus probability . For example, in a contin- requires active suppression. For example, the gency in which a rat receives food if and only if it effects of handling may make the start of a ses- presses a lever, a lever press raises the probabil- sion different from later times. If so, effects of ity of food from zero to 1.0, but in a contingency extinction late in one session might not transfer in which lever presses do nothing, the prob- to the start of the next session. On these grounds, ability of food is independent of lever presses. Kendall (1965) reasoned that the usual pattern (Strictly, a response-stimulus contingency is vir- of response rates in extinction sessions could tually always part of a three-term contingency, be reversed under the right conditions. The key but we needn’t address that issue here.) pecks of three pigeons were fi rst reinforced in 1-hr If an extinction procedure terminates both sessions. Repeated 1-min sessions of extinction a contingency and reinforcer deliveries, how followed. The fi rst long extinction session came much of what happens depends on the end of the only after responding had reliably decreased to contingency and how much on the end of food zero in the brief sessions. Within a few minutes, deliveries? In extinction, the rat’s responses no each pigeon began to respond. Until this session, longer do anything, but the rat is also no longer responding had never extinguished at times later eating. Taking away reinforcers will affect more than 1 min into a session; responding occurred than just the reinforced response. If food is sud- at these later times when the opportunity was denly taken away from a food-deprived rat that fi nally available. In a sense, Kendall had dem- has been eating, for example, the rat becomes onstrated spontaneous recovery within a session more active and perhaps urinates or defecates. rather than at its start. If you looked at it, you’d say this was an angry Another example of recovery of extinguished rat. If food was produced by lever presses, the responding is called regression or resurgence rat might bite the lever. If other organisms are in (Epstein & Skinner, 1980; F. S. Keller & Schoen- the chamber, the rat might attack them (Azrin, feld, 1950, pp. 81–82). Suppose a rat’s chain pulls Hutchinson, & Hake, 1966). And the oppor- are extinguished and then lever presses are rein- tunity to engage in such aggressive responses forced. If the lever presses are later extinguished could even reinforce other responses (Azrin, the previously extinguished chain pulls are likely Hutchinson, & McLaughlin, 1965). to reappear. By analogy to clinical terminology, These effects, though observed in extinc- the phenomenon suggests regression from cur- tion, aren’t results of terminating the reinforce- rent behavior (lever presses) to older behavior ment contingency. They occur not only during that was once effective (chain pulls). It is of con- extinction but also when response-independent siderable practical signifi cance. For example, the or free food deliveries stop. We call them side training of show dogs sometimes requires dur- effects because they are indirect products of the ing a show the withholding of reinforcers that change in contingencies. Whether after free food had been available during training. This must be or food produced by responses, a rat that has done carefully to prevent regression to behavior been eating can no longer do so. In extinction, that had been common before but would now these side effects get superimposed on decreases spoil the dog’s performance. in the previously reinforced responding because Chapter 8. Reinforcement, Free Reinforcement, and Extinction • 69 the termination of reinforcers is necessarily part effects, in favor of alternatives that didn’t pro- of extinction. duce such effects. These effects, such as the aggressive respond- Dealing with problem behavior presented ing generated by terminating reinforcer deliver- by children with severe developmental issues ies, could have been observed in situations that is hard enough. Why deal also with aggressive removed reinforcers but didn’t involve response behavior or other emotional side effects of extinc- consequences. The implications for applications tion, especially when other techniques are avail- of reinforcement and extinction procedures are able (Lalli, Casey, & Kates, 1997; Lerman, Iwata, signifi cant. For example, those who work with & Wallace, 1999; R. H. Thompson et al., 2003)? children sometimes use free or response-inde- The reason I hadn’t been able to fi nd appropriate pendent reinforcers rather than extinction to extinction data in the applied literature was that avoid the side effects of terminating reinforcer extinction was not a preferred procedure, so few deliveries (e.g., Hart, B. M. et al., 1968). data were collected and presented. What, then, Extinction was long regarded as the most are the alternatives to extinction? appropriate way to get rid of problem behavior, but that view has fallen from favor as the side effects of extinction were increasingly recog- EXTINCTION VERSUS FREE nized. You can see the difference even across my REINFORCEMENT own writings on learning. While I was writing an earlier book (Catania, 1992), my colleague Eliot Of the two essential components of extinction, Shimoff urged me to use more human exam- terminating a contingency and terminating ples, arguing that it would be helpful to include the delivery of reinforcers, the latter is primar- not just anecdotes and hypothetical cases but ily responsible for most of the unwelcome side also actual human data presented in fi gures. effects of extinction. A contingency can be ter- I agreed it was a good idea and began search- minated without terminating the delivery of ing out candidate data sets, fi rst in the Journal reinforcers by arranging the free or noncontingent of the Experimental Analysis of Behavior and then delivery of reinforcers (noncontingent reinforce- in the journal where it might be assumed they’d ment is sometimes abbreviated NCR). Figure 8–1 be more likely to appear, the Journal of Applied illustrates differences between contingent rein- Behavior Analysis. Here and there I found cases forcement, extinction, and free reinforcement of the extinction of human responding, but typi- or NCR . Baseline conditions, left of the dashed cally embedded in complex data presentations. vertical line, show contingent reinforcement; for A clear description of the procedure and out- illustrative purposes, the three left examples are come seemed to require a distracting detailed shown as identical. elaboration and even those examples were rare, The three frames show contingent reinforce- which is why you won’t fi nd any human extinc- ment (reinforcers produced by responses) up to tion data in this book either. the vertical dashed line, followed on the right At the time I was heavily involved in a new by continuation of contingent reinforcement Masters track in Applied Behavior Analysis in (top), extinction (middle) and free or noncon- my department, a joint effort with the behav- tingent reinforcers (bottom). For convenience, ioral programs at the Kennedy-Krieger Institute the baseline reinforcement conditions on the in Baltimore (Catania, DeLeon, & Cataldo, 2001). left are shown as identical, and this hypotheti- My colleagues there were much involved in cal case assumes a rich intermittent schedule in work with self-injurious and other serious prob- which reinforcers are delivered after a small but lem behavior in children, so I consulted them variable number of responses; on average every about their use of extinction. What I learned was third or fourth response produces a reinforcer. that their procedures had evolved to minimize When contingent reinforcement contin- the use of extinction, with its attendant side ues (top), reinforcers follow closely after the 70 • Part III. Learning Without Words: Consequences

Figure 8–1 Hypothetical responding maintained by intermittent reinforcement (left) followed by one of three conditions (right): continued reinforcement (top), extinction (middle), or free reinforcement (bottom). With the response-reinforcer contingency discontinued (middle and bottom), responding decreases. The emoticons represent diff erent side eff ects: the organism is happy as long as it continues to eat whether food is response- contingent or not, but it is unhappy if the food stops. Chapter 8. Reinforcement, Free Reinforcement, and Extinction • 71 responses that produce them. The timing of the ior. As lever pressing continues, some are free reinforcers at bottom are matched or yoked still followed closely by food. Lever pressing to those in the top frame, which means the rats declines slowly because these frequent acci- in the top and bottom procedures receive the dental successions of responses and reinforcers same amounts of food distributed identically counteract the effects of terminating the rein- over time. The difference is that when free rein- forcement contingency. Eventually lever press- forcers are delivered (bottom), they can occur at ing drops out, but extinction would probably varying times after responses. On the far right have been quicker. For this reason, arranging the bottom frame shows some substantial delays a transition from a reinforcement contingency between a response and a subsequent reinforcer. to response-independent reinforcer deliveries With extinction (middle), of course, no reinforc- has sometimes been regarded as a poor way ers are delivered. to examine the effects of terminating reinforce- When contingent reinforcement continues ment contingencies (Boakes, 1973). (top), response rate remains roughly constant. Again, there are practical implications. Free When the response-reinforcer contingency is reinforcers aren’t the only option. Suppose an discontinued, during either extinction (middle) institutionalized boy with developmental prob- or free reinforcement (bottom), response rate lems often engages in self-injurious behavior decreases. Now consider the emoticons. In all such as head-banging or eye-poking. Suppose the conditions save one, food is delivered, either further we discover his self-injurious behavior contingent on responding or freely, so the food- is in large part maintained by staff attention deprived rats who had been eating continue to as a reinforcer. Extinction is not well-advised eat; food is absent only in extinction, and only because of the harm he might do to himself if we in that case do the rats who had been eating no ignore the behavior. Giving him attention inde- longer do so. Discontinuing food has the behav- pendently of the self-injurious behavior is one ioral effects described earlier and often referred possibility, but that might reduce the behavior to as emotional. only slowly. An alternative procedure is to use The contingency between responses and attention to reinforce an alternative response, reinforcers is terminated in both extinction and and especially one incompatible with the self- free reinforcement in the fi gure, and in both injurious behavior (Repp & Deitz, 1974). The cases responding decreases. In other words, self-injurious behavior will inevitably decrease responding decreases once the response-rein- as the alternative response increases. Such pro- forcer contingency is broken, whether or not cedures, sometimes called differential reinforce- reinforcers are delivered. But only extinction ment of other behavior, have been widely applied produces emotional behavior. In other words, to problem behavior (e.g., Skiba, Pettigrew, & the discontinuation of reinforcers is what pro- Alden, 1971, on thumbsucking). duces this side effect; the response-reinforcer Reinforcers are important, after all. If atten- contingency has nothing to do with it. tion has contributed to increases in the self-inju- For convenience, the response decrements rious behavior of a child with developmental in the middle and bottom right frames were issues, that tells us how important attention is to shown as equal, but in practice responding usu- the child, who will be better served if we shape ally declines somewhat more slowly with free alternative and more functional ways of com- reinforcers than with extinction, especially on manding attention than if we simply remove the fi rst as opposed to later exposures to these attention (Carr, Severtson, & Lepper, 2009; Goh, procedures. Why the slower decline with free Iwata, & DeLeon, 2000; Hagopian et al., 2000). reinforcer s than with extinction? First a rat’s Unfortunately, generations of students lever presses are reinforced with food; then, given cursory accounts of behavioral methods when presses no longer produce food, food in introductory textbooks seem to have taken deliveries continue independently of behav- away as the primary message that you should 72 • Part III. Learning Without Words: Consequences never reinforce behavior you want to get rid of; The Vocabulary of Free Reinforcement instead, you should just ignore it. But if attention as a reinforcer is implicated in the development Though both terms are common in the literature, of self-injurious behavior, then it must be very I’ll usually refer to the delivery of reinforcers important and taking attention away may not be independently of behavior as free reinforcement the solution. Reinforcement isn’t everything, rather than noncontingent reinforcement. To but extinction isn’t anything. If the reinforcers speak of noncontingent reinforcers may seem are already there, they shouldn’t be wasted; they a non sequitur or perhaps even an oxymoron, should instead be used constructively. because reinforcers are defi ned by the effects on We all shape each other’s behavior. The more responding of contingencies between responses we know about how reinforcement works the and their consequences and no such contingen- more likely it is we’ll use it productively and its cies operate when reinforcers are delivered inde- avoid pitfalls, such as coercive practices that can pendently. The terminology isn’t ambiguous if follow if control over reinforcers remains one- used to refer only to the response-independent sided. Free reinforcers coupled with the shaping delivery of stimuli known to be effective as of other behavior are a better alternative than reinforcers and not to the effects of that proce- extinction. Though it might seem more compli- dure, but in that case it wouldn’t parallel other cated to arrange these contingencies, the absence common usages (e.g., extinction as terminat- of the emotional side effects typically more than ing reinforcers or as the resulting decrease in makes up for the difference. responding). This issue is fi nessed to some Even better, of course, is not getting to the extent by free reinforcement, though free reinforc- point of being tempted to impose extinction. ers might be preferable. One way to reduce a child’s bad behavior is to reinforce good behavior, and that’s one reason Extinction and Superstition why “Catch them being good” is such an effec- tive slogan for both parents and teachers. Yet Given the advantages of free reinforcement, how many still take away from an introductory why has extinction remained the primary way psychology course the message that what par- to study the effects of terminating contingencies ents should do to get rid of a child’s problem for so long? One factor is that disconnecting the behavior is to ignore it? Left unresolved are the lever from the feeder is easier than disconnect- inevitable subsequent issues, such as what par- ing the lever and at the same time substituting a ents should do when other problem behavior device that operates the feeder periodically. But emerges that is maintained by the same rein- the answer may more likely lie in properties of forcer. We shouldn’t teach parents to ignore the behavior than in a mere change in apparatus. behavior of their children; we should teach them In the phenomenon called superstition (Skin- how to use reinforcers productively. You need ner, 1948), food is repeatedly presented to a more than a superfi cial understanding of rein- food-deprived pigeon at short intervals (e.g., forcement to do it right. every 10 or 15 seconds); this is, again, free rein- We’ve made the case that free reinforcement forcement. Responses that occur just before food is typically preferable to extinction in applied deliveries are likely to be repeated and therefore settings. Given it involves terminating only the to be followed closely by still more food deliv- response-reinforcer contingency and doesn’t eries. The effect of the accidental succession of produce some of the side effects of extinction, responses and reinforcers perhaps the time has come to argue it’s not merely another way to study the effects of ter- is usually obvious. The bird happens to minating contingencies but is the preferable way be executing some response as the hop- to do so. per appears; as a result it tends to repeat Chapter 8. Reinforcement, Free Reinforcement, and Extinction • 73

this response. If the interval before the next Drift itself demonstrates that the effects of presentation is not so great that extinc- accidental correlations between responses and tion takes place… [this] strengthens the reinforcers are transient. The evidence that free response still further…. The bird behaves reinforcers don’t ordinarily maintain behavior is as if there were a causal relation between substantial (Catania, 2005c). Nevertheless, wor- its behavior and the presentation of food, ries about creating and maintaining superstitious although such a relation is lacking. (Skin- patterns have persisted. Superstitious respond- ner, 1948, pp. 168–171) ing was seen as a ubiquitous problem because such accidental sequences can occur whether The topography or form of each pigeon’s reinforcers are independent of responses or are responding changed gradually as the procedure their consequences. But superstition, in its behav- continued, as accidental relations developed ioral sense, is no longer regarded as the enduring between responding and food deliveries. Skinner phenomenon it was once thought to be. Organ- called such changes topographical drift. A pigeon isms contact only the temporal relations between responded temporarily as if its responses were responses and reinforcers and not the contingen- producing food as a consequence, but no specifi c cies that generated them. Nevertheless, ordinar- one stayed consistently in place as superstitious ily they are exquisitely sensitive to the differences behavior. Over long periods of time, variations between events produced by their behavior and on pecking often dominate as the behavior just those that by coincidence follow the behavior. We preceding food deliveries (Staddon & Simmel- humans are sometimes an exception, but that’s hag, 1971). If behavior persists for long periods because human superstitions involve much more in the absence of a reinforcement contingency than coincidence (cf. Skinner, 1987). They’re typi- it is often more appropriate to look for other cally intricately interwoven with verbal behavior sources of the behavior than to attribute it to as well as natural contingencies. For that reason superstition arising from adventitious correla- we won’t here consider these verbal varieties of tions of responses and reinforcers. human superstition. Chapter 9 Consequences of Responding: Punishment

In nature there are neither rewards nor punishments; there are consequences. — Robert Green Ingersoll (1881)

In reinforcement, the consequences of responses just taken from a hot oven, but its removal or pre- make responding more likely. In punishment, the vention may reinforce, as when I learn to treat a consequences of responses make responding burn with medication or put on kitchen gloves less likely. Paralleling the vocabulary of rein- while handling things around a stove. Except forcement, a punisher is a type of stimulus, but for positive reinforcement, reinforcement by punishment is neither stimulus nor response. presenting a stimulus, these relations are often The term punishment names a relation between grouped together as instances of aversive control, behavior and environment. The relation which includes both punishment and negative includes at least three components: (i) responses reinforcement, i.e., reinforcement by removal or must have consequences; (ii) their probability prevention of aversive stimuli. must decrease: they must become less probable Biological parallels are relevant. The gradual than when not having those consequences; (iii) disappearance of legs in the ancestors of whales the decrease must occur because they have those was a phylogenic variety of extinction; that dis- consequences and not for some other reason. appearance is analogous to the disappearance For example, if we knew only that responding of unreinforced responding in the extinction decreased, we couldn’t say that it must have of operant behavior. Whether in phylogeny or been punished; maybe it was previously rein- ontogeny, use it or lose it applies (Carroll, 2006). forced responding that has been extinguished. But in either realm, extinction is not the only pos- It wouldn’t even be enough to know that the sible cause of a population decrease. Decreases response was now producing some stimulus it can be the result of environments that actively hadn’t produced before. We’d still have to know reduce a population, as when in phylogeny a whether responding decreased because that mutation causing a distinctive marking makes stimulus was its consequence. an organism more likely to be spotted by preda- A stimulus that reinforces responding when tors, or as when in ontogeny some responses but responses produce it may serve the opposite not others cause painful burns or other tissue function when those responses remove it: its damage. Punishment in ontogenic selection is removal may punish the responding. Inversely, an analog of the effects of harmful variations in a stimulus that punishes responding when phylogenic selection. responses produce it may reinforce responding when those responses remove it. For example, money may reinforce, as when a child is paid for THE VOCABULARY OF completing a chore, but its removal may pun- PUNISHMENT ish, as when the child’s allowance is canceled because of a misdeed. Similarly, a painful burn As a procedure, punishment is arranging a may punish, as when I learn not to touch a pan response consequence that makes the respond-

74 Chapter 9. Consequences of Responding: Punishment • 75 ing less likely. The stimulus arranged as a con- that a parent intent on spanking should go to the sequence is called a punisher . For example, if a child instead; reinforcing responses incompatible rat’s lever presses produce electric shock and with the misbehavior and other better alterna- this procedure reduces lever pressing, the lever tives don’t require spanking at all. press is said to be punished and the shock is The point isn’t just grammatical. We’re more said to be a punisher. The response that changes likely to see what’s happening if we state the must be the one that produces the punishing punished response explicitly (spanking pun- consequence. It doesn’t qualify as punishment ished the child’s approach) than if we settle for a if a rat’s jumping produces shock and only its less precise description (spanking punished the lever pressing decreases. child). In endorsing the grammar of reinforcing Like reinforcement, the term punishment responses and punishing responses, we needn’t applies to both procedures and outcomes. Thus, prejudge how these operations affect behavior. saying a response was punished may mean They’ll often affect other responses besides those either that the response produced a punisher or for which they’re arranged (e.g., a spanking may that responding decreased because it produced also elicit crying). A vocabulary that states the a punisher. As with reinforcement, our preferred consequences of behavior helps us to describe usage will be to restrict the term punishment to such effects without ambiguity. the vocabulary of procedures or operations and to describe the outcome directly as a change in responding. As with reinforcement, however, COMPARING REINFORCEMENT the outcome usage has so much precedent that AND PUNISHMENT we can’t avoid it. The vocabulary of punishment also paral- The effect of punishment is essentially the lels that of reinforcement in its object: we say opposite of that of reinforcement. The relation that responses are punished, not organisms. If between the two is illustrated in Figure 9–1, a rat’s lever pressing produces shock and lever which shows hypothetical reinforcement and pressing decreases, it’s appropriate to say the rat punishment data. The top graph shows a rat’s was shocked and the lever press was punished; lever pressing during food reinforcement and it goes against colloquial usage, but it is not then during extinction. We could as easily sub- appropriate to say that the rat was punished. As stitute scenarios involving the effects of reinforc- with reinforcement, this grammatical distinction ing help with household chores or of punishing encourages us to be precise when we describe errors in school work. Coming up with plausible behavior. One reason it differs so dramatically everyday human alternatives is a useful exercise. from everyday usage is that the everyday con- In any case, during baseline , when lever pressing cern is too often with retribution rather than has no consequences, responding occurs infre- with changing behavior. quently. When reinforcement begins, responding A child misbehaves. A parent calls the child increases over the fi rst few sessions, after which and administers a spanking when the child it remains at a fairly stable level. In extinction, comes. It may be convenient to say that the par- responding gradually returns to its former low ent punished the child, but this usage makes it baseline level. far too easy to omit the responses that might The bottom graph shows the rat’s lever be affected. The immediate consequence of the pressing during and after punishment of lever child’s misbehaving was that the parent called; presses with electric shock. Because punish- the spanking occurred after the child obeyed the ment reduces responding, some responding call. Although the child might misbehave less in must exist to begin with or we couldn’t observe the future because of the spanking, the spanking a decrease. In this example, responding is will probably also make the child less likely to already maintained by food reinforcement, come when called. By the way, this doesn’t mean which continues throughout all sessions; the 76 • Part III. Learning Without Words: Consequences

Figure 9–1 Eff ects of reinforcement and punishment on the hypothetical lever pressing of a food-deprived rat. Top: unreinforced pressing (baseline), the increase when it produces food (reinforcement), and a return to earlier levels when reinforcement ends (extinction). Bottom: pressing maintained by reinforcement (baseline), the de- crease with response-produced shock added (punishment), and a return to earlier levels when punishment ends (recovery).

effects of punishment are assessed by super- stamped out by annoyers as well as stamped in imposing it on this baseline. The baseline shows by satisfi ers. Statements of Thorndike’s law that the responding maintained before lever press- included the punishment component were called ing is punished. When punishment begins, lever the strong Law of Effect. Later, Thorndike with- pressing decreases to a maintained low level. drew the punishment component. The version In recovery , punishment is discontinued and that remained, including only the stamping in responding gradually returns to its higher of behavior, was called the weak Law of Effect. baseline level. Thorndike based his conclusion on experi- Reinforcement and punishment are sym- ments on human verbal learning in which say- metrical: the former increases responding ing “right” to the learner enhanced responding whereas the latter decreases it; in both cases whereas saying “wrong” had less effect than the effects continue as long as the procedures saying nothing. Thorndike accepted the fi nding are maintained and disappear after they end. as evidence against the effectiveness of punish- Nevertheless, the effectiveness of punishment ment. has long been controversial. Punishment was Thorndike’s conclusion had so much impact part of Thorndike’s early versions of his Law of that even data showing punished responding Effect, when he argued that behavior could be recovered after it was discontinued were inter- Chapter 9. Consequences of Responding: Punishment • 77 preted to mean punishment was ineffective. It Research on punishment not only changed wasn’t to be taken seriously as a way to change the criteria for its effectiveness; it raised ques- behavior because it suppressed responding only tions about whether punishment had been temporarily. Yet by this criterion reinforcement judged adequately according to the old criteria. should be judged ineffective too. Perhaps people A single punisher or a few punishers, if intense were looking for a procedure that would get rid enough and the consequence of a weakly main- of behavior permanently even if applied only tained response, might make a response disap- once or only briefl y. Yes, an extreme punisher pear for most or all of a lifetime. Such exceptional can suppress behavior for a very long time, but as effects might as well be regarded as permanent, a rule punishment keeps working only while the but why should we be surprised that recovery punishment contingency continues. Once you’ve can sometimes be so slow that it exceeds a life started punishment, you must be ever alert for a span? On the reinforcement side, would we reappearance of the punished behavior. We have be surprised if winners of million-dollar lot- reasons to believe techniques other than punish- tery prizes continued to buy occasional lottery ment should be used whenever possible, but if tickets throughout their lives even if buying a this is correct it’s because Thorndike and his suc- ticket was never again reinforced by a win? We cessors were right for the wrong reasons. wouldn’t reject the existence of extinction on Punishment depends on what maintains such grounds, so we shouldn’t reject recovery responding; for example, punishment by shock after punishment either. probably will reduce food-reinforced lever Prejudices against recognizing punishment pressing less if a rat is severely rather than mildly were at times so strong that effective proce- food-deprived. A variety of parameters can dures were even given a different name, passive modify its effectiveness (Azrin & Holz, 1966). avoidance. For example, consider a rat on a plat- With electric shock as the punisher of a pigeon’s form above an electrifi ed grid set up so the rat food-reinforced key pecking, the more intense is shocked when it steps down to the grid; once and immediate the punisher the more effective it shocked, the rat is less likely to step down again is. A punisher introduced at maximum intensity later. Stepping down is here punished by shock, suppresses responding more effectively than one but some would say instead that by holding back introduced at low intensity and then gradually from stepping down the rat passively avoids the raised to maximum intensity. The effectiveness punisher. This usage allowed effective punish- of punishers may change over time, as when a ment to be discussed as passive avoidance while low-intensity punisher gradually becomes inef- other procedures that didn’t reduce responding fective after many presentations (Azrin & Holz, were used to defend the claim that punishment 1966). As with extinction, it’s easier to punish was ineffective. one response when some other response pro- The issue remains. A study of the behavior duces the same reinforcer than when no other of teen drivers after car collisions showed that alternatives are available. And, in a parametric g-force events, a measure correlated with risky relation especially relevant to human applica- driving, decreased during the fi rst month or so tions, punishers delivered after short delays are after a collision but recovered to earlier levels more effective than those delivered after long after about three months (O’Brien et al., 2017). ones; with either pets or children, aversive con- In other words, the contingency between risky sequences delivered long after unwanted behav- driving and collisions produced temporary pun- ior are unlikely to be very effective, even when ishment effects, just as we might have predicted. accompanied by a verbal specifi cation of the But the report of the study made no reference to behavior for which the punisher was intended. what we know about punishment and instead If such delayed punishers work at all, they prob- mainly attributed the temporary effects to per- ably depend on an extensive verbal history on sonality variables such as maturity. My guess, the part of the one at the receiving end. 78 • Part III. Learning Without Words: Consequences however, is that similar temporary effects would THE RELATIVITY OF have occurred with adult drivers. PUNISHMENT The direct effects of punishers were exam- ined in a historical context in which punishment In experiments on punishment, punishers are was typically regarded either as ineffective or, usually chosen for their reliable effect on a vari- when effective, as the indirect product of other ety of responses, because such stimuli reveal processes (Azrin, 1956; Azrin & Holz, 1966). For the effects of punishment most clearly. One example, Skinner (1953) argued that behavior such punisher is electric shock, which can be leading up to a punished response generated measured accurately and can be presented at aversive proprioceptive stimuli that ceased effective levels that don’t damage tissue. Such when the individual turned to doing some- stimuli, however, are only extreme instances thing else. According to this interpretation, the of punishers. For example, some children with punished response only indirectly became less developmental problems chronically engage in likely, because switching to something else head banging, hand biting and other self-inju- avoided those aversive cues. Responding was rious behavior. A brief squirt in the face with said to decrease not because it was suppressed water mist from the sort of spray bottle used by the punisher but because everything else was to moisten indoor plants is at worst a minor reinforced by avoidance of the aversive stimu- annoyance. When applied contingent on such lus. behavior, however, it is an effective punisher The argument that punishment is reducible (Dorsey et al., 1980). As such it is relatively to avoidance is curious, because punishment is innocuous, especially when compared with the a robust phenomenon in which the abrupt intro- serious damage these children can do to them- duction of a strong punisher typically reduces selves. Unfortunately, some who oppose any responding very quickly, whereas it is usually use of punishment fi nd even this application diffi cult and time-consuming to establish avoid- unacceptable. ance, though it’s robust once in place. Why Punishment is inevitable because it is invoke hypothetical behavior presumed to arise arranged by many natural contingencies. A child from relations that are diffi cult to establish to who teases a barking dog may get bitten and explain behavior that is easy to establish? one who plays with fi re may get burned. Fur- Evolutionary considerations add to the thermore, even stimuli that ordinarily serve as argument for the direct effects of punishers. reinforcers can become punishers. For example, If the phylogenic origins of reinforcement are food that is reinforcing at the beginning of a holi- rooted in the effects of consequences it would day feast may become aversive by the time the be strange indeed if operant behavior evolved meal has ended. On the other hand, events that in only one direction, so consequences could superfi cially seem aversive, such as falling from increase behavior but couldn’t reduce it. Even a height, may be reinforcing under certain cir- if punishment worked that way, however, cumstances (consider sky diving and ski jump- so we learned not to stick our hands into fi res ing). Like reinforcers, punishers can’t be defi ned only because we avoided the aversive stimuli in absolute terms or based on common physi- occasioned by approaching one, it would make cal properties. Rather, they must be assessed little practical difference. For those who have to by examining the relation between punished make decisions about whether or when to use responses and the responses occasioned by the punishers, punishment contingencies will work punishers. pretty much the same way whichever position The Premack principle of reinforcement one assumes, Nevertheless, we’re treating pun- states that opportunities to engage in more prob- ishment here as a behavioral process in its own able responses will reinforce less probable ones. right, not as a derivative of something else. That analysis can be extended to punishment Chapter 9. Consequences of Responding: Punishment • 79

(Premack, 1971). In an apparatus that controls a lever while lying on their furry backs (Azrin & rat’s opportunities to run in a running wheel or Holz, 1966). In either case, such side effects must drink from a drinking tube, depriving the rat of be taken into account. opportunities to run while giving it free access to water makes running more probable than Eliciting Effects of Punishers drinking, and depriving it of water while giv- ing it opportunities to run makes drinking more Another diffi culty is that punishers are likely probable than running. to have some effects that occur whether or not Rotation of the wheel can be made a con- they’re produced by responses. Whether or not sequence of drinking: whenever the rat drinks it was response-produced, shock is likely to the wheel begins to turn and the rat is forced to elicit a rat’s jumping. As with reinforcement, the run. When running is more probable than drink- punishment effect must depend on the relation ing, this procedure increases drinking and it is between responses and punishers (contingency), appropriate to say drinking is reinforced by run- not simply on the delivery of punishers. ning. But when running is less probable than Response-produced or response-indepen- drinking, this procedure has an opposite effect: dent shocks were superimposed on rats’ lever now drinking decreases when running is its con- pressing maintained by food reinforcement sequence and it is appropriate to say drinking (Camp, Raymond, & Church, 1967). Measured is punished by running. If their relative prob- against a no-shock control group as a baseline, abilities can be reversed, as in this example, any shock reduced responding in both groups, but response can be either reinforced or punished by response-produced shock suppressed respond- any other response. ing much more than response-independent shock. This difference makes it appropriate to call response-produced shock a punisher. Just as SIDE EFFECTS OF PUNISHMENT we must distinguish between effects of reinforcer deliveries and effects of the response-reinforcer As with reinforcers, punishers can have effects contingency, we must also distinguish between independent of their contingent relation to effects of punisher deliveries and effects of the responses. If an organism is shocked or burned response-punisher contingency. or pinched, some of its responses may have little If at fi ve-minute intervals a monkey’s lever to do with whether these events were brought presses deliver shock to its own tail, the mon- on by the organism’s own behavior. Diffi cul- key increases its rate of pressing within each ties arise in analyzing punishment because such interval, slowing down only after the shock is effects must be distinguished from those that delivered; if the shocks at fi ve minutes are dis- depend on the relation between responses and continued, the monkey stops pressing (Morse their consequences. As with reinforcement and & Kelleher, 1977). Shocks depend completely extinction, once again we must deal with side on the monkey’s behavior. It would receive no effects. shocks if it didn’t press the lever. Why doesn’t Some effects of shock may be primarily it just stop? The paradox is that the same shock physiological, as when successive shocks sys- that maintains lever pressing when produced tematically reduce a rat’s skin resistance. The by pressing according to this fi ve-minute con- effectiveness of later shocks may then vary with tingency suppresses lever pressing when pro- the rat’s resistance. Other effects are primarily duced instead by every press; the same shock behavioral, as when the apparatus fails to pre- can also be used to initiate and maintain escape vent responses by which the rat can reduce its or avoidance behavior (J. E. Barrett & Stanley, contact with the shock source. For example, fur 1980). So how do we decide whether the lan- is an insulator and rats have sometimes mini- guage of punishment is appropriate? If every mized punishing effects of shock by pressing the lever press produces shock, we call the shock a 80 • Part III. Learning Without Words: Consequences punisher because it reduces responding. Should that here the shock acquired its power to rein- we call the shock a reinforcer because it gener- force through its relation to food; it loses its ates responding? power if that relation ends. Perhaps these pro- It wouldn’t help to call the monkey a mas- cedures are relevant to human behavior. For ochist. Masochism is just a name we apply when example, a battered child might provoke a par- a stimulus we think should be a punisher serves ent to the point of a beating because the beat- as a reinforcer; it doesn’t explain anything. Other ings are often followed by more attention from research suggests the eliciting effects of shock the then-remorseful parent than ever follows were strong enough to override the punishing less traumatic parent-child interactions. In this effects, so that the monkey’s pressing occurred example, the beating is analogous to the shock in spite of and not because of the punishment in the pigeon example and the parent’s attention contingency (Pitts & Malagodi, 1991). If so, the is analogous to the food. A parent’s attention case is analogous to that of the parent who tries can sometimes be so potent a reinforcer that it to stop a child from crying by punishing the cry- overrides the effects of consequences that would ing and has trouble because the punisher elicits otherwise serve as punishers. Thus, a behavioral the very response that the parent is trying to analysis may be relevant to human problems suppress. We may not like these contingencies, such as child abuse. but if we ignore them that doesn’t mean they’ll go away. As long as human behavior includes such problems as self-injury, our concern with NEGATIVE PUNISHMENT: such phenomena is justifi ed. TIMEOUT

Discriminative Effects of Punishers Responses can be punished by the termination of events. For example, removing food contin- Another side effect of punishment comes about gent on a food-deprived rat’s lever presses is because punishers can acquire discriminative likely to reduce the rate of pressing. The prob- properties, as when a response is reinforced lem is that it might be hard to demonstrate nega- only when it is also punished. An experiment tive punishment. If the rat is food-deprived and designed to make response-produced shock sig- food is available, it will probably eat rather than nal the availability of food arranged two alter- press, so we’ll have few opportunities to pun- nating conditions (Holz & Azrin, 1961). In one, ish lever pressing by removing food. For this a pigeon’s key pecks had no consequences; in reason, studies of negative punishment usually the other, every peck produced shock and some opportunities to work for reinforcers rather than pecks also produced food reinforcers. A low rate the reinforcers themselves. of pecking was maintained when pecks pro- Suppose a monkey has two levers available duced no shock, because then they never pro- and presses on one of them produce food when- duced food either; but pecking increased once ever a light is on. We can expect some presses pecks began to produce shock, because only on the other lever, but we can punish them by then did they sometimes also produce food. making each one initiate a time during which With no shocks, the rate of pecking was low. the light turns off and presses on the fi rst lever When responses began to produce shock, the won’t produce food. Such periods are called rate increased. timeout or, more fully, punishment by timeout Again we can ask whether the shocks should from positive reinforcement (Ferster, 1958). Time- be called punishers. In fact, responding that pro- out originated in experiments with pigeons and duced shock increased relative to responding rats and monkeys and was common in the labo- that did not, so we should conclude that shock ratories in which I worked as a graduate stu- was a reinforcer. The main difference between dent and early in my career, but those origins the shock and other more familiar reinforcers is are unfamiliar so no doubt it’s now best known Chapter 9. Consequences of Responding: Punishment • 81 in its human applications (e.g., Wolf, Risley, & likely to be placed with a new owner than one Mees, 1964). that doesn’t present these problems (e.g., Wil- As with any form of punishment, the main kes, 1994). Is it unethical to use punishment if it function of timeout is to reduce behavior, but might eliminate such behavior so that instead of it’s too often applied without attention to alter- being put down the dog can successfully live in native behavior that might be reinforced (Winett a human home? & Winkler, 1972). Some who use timeout with- Punishment can modify human behavior, so out understanding its behavioral basis apply it questions inevitably arise about the ethics of its counterproductively. For example, some years application. But punishment can’t be eliminated ago I conducted research with preschoolers in a from natural environments (Perone, 2003). With- local daycare center and occasionally had oppor- out punishment, a child who has been cut while tunities to observe how the staff managed the playing with sharp objects or scratched when behavior of the children in their care. Timeout teasing the cat would remain undeterred from was one of their procedures. By then, isolated doing so again later on. Artifi cial punishment timeout rooms were already a thing of the past; contingencies, however, are constrained by prac- instead the staff used designated timeout areas. tical considerations. Because responding usually A child who disturbed other children in a play- recovers after punishment is discontinued, pun- group might be sent to sit for fi ve minutes on ishment must include plans for environments in a timeout carpet near the play area, in view of which the relevant contingencies may be absent, the other children. I watched as one boy in time- just as reinforcement must include plans for out sat quietly for about four minutes and then what will maintain the behavior when reinforce- began to cry and act up (four minutes of sitting ment ends. It may do little long-term good to quietly is a very long time for a preschooler). The eliminate a child’s self-injurious behavior with staff person nevertheless rigidly adhered to the punishment in a hospital setting if the punish- fi ve minutes of timeout and let him return to the ment contingency does not exist when the child playgroup at the very time when his disruptive returns home. was most dramatically increasing. It would have The behavioral properties of punishment been better to “catch the child being good” and have implications consistent with ethical argu- to have allowed his return at the end of three ments against aversive control. For example, a or four minutes. Unfortunately I couldn’t inter- parent who arranges frequent aversive conse- vene: I had academic credentials but no applied quences for a child’s behavior may acquire aver- behavior analysis credentials, and in any case a sive properties. To the extent the child learns condition of my research arrangements was that to escape from or avoid the parent’s company, I not intrude on staff practices. But it is good to contingencies other than those available to the know that timeout procedures have been falling parent will increasingly infl uence the child’s out of favor and are gradually being replaced by behavior. But if punishment seems the only other practices based on reinforcement rather effective technique for reducing the self-mutilat- than punishment. ing behavior of a brain-damaged child, punish- ment might be a lesser evil than the permanent damage the child might self-infl ict. Self-injuri- THE ETHICS OF PUNISHMENT ous children have been known to disfi gure and even blind themselves. Every day animal shelters put down unadopted One of the ethical standards for interventions dogs and cats and other pets that exceed their in applied behavior analysis is to impose the least capacities. Sometimes behavior makes the dif- restrictive treatment. This means a treatment that ference between life and death. If an owner empowers a child by shaping the child’s commu- delivers to a shelter a dog that barks exces- nication skills will always be preferable to one sively or damages furniture, the dog is far less that calls for restraints or other environmental 82 • Part III. Learning Without Words: Consequences restrictions (Bailey & Burch, 2005; Goldiamond, delivering a reinforcer. That means people usu- 1974). Assessments and applications of reinforce- ally more readily learn techniques of aversive ment and extinction and punishment contingen- control than techniques of reinforcement, but it cies have become increasingly sophisticated (e.g., doesn’t mean the aversive techniques are better. Fisher & Bouxsein, 2011; Lerman & Toole, 2011; Some argue against any kind of modifi cation Vollmer & Athens, 2011). We may look forward of behavior, whether involving aversive stimuli to the day when parents and other caregivers or positive reinforcers. But those who make such as well as behavior analysts share this level of arguments should recognize that our behavior understanding of basic contingencies. is being modifi ed all the time, both by natural Unfortunately, if we look at the behavior of contingencies and by the artifi cial ones created those who deliver reinforcers or punishers, we by those around us. Politicians and businesses see that effects of reinforcement often show up and educators and religious groups and govern- only long after the reinforcer is delivered (the ments arrange all sorts of contingencies affect- effect of reinforcing a child’s diligent school- ing our behavior in many ways. Denying that work each day might not be obvious until sev- the contingencies exist doesn’t make them go eral days have gone by), whereas the effects away. Ignorance rarely provides good grounds of punishment often show up right away (one for ethical judgments, so the counterargument effect of slapping or scolding a child who is is that our best defense against the misuse of teasing a sibling stops the teasing stops immedi- behavioral techniques is to learn as much as we ately). Thus, delivering a punisher is much more can about how they work. likely to produce immediate consequences than Chapter 13 Motivating Variables and Reinforcer Classes

We say that an animal is hungry if, when we give it food, it eats. — B. F. Skinner

Tasting unsalted soup doesn’t make it more Food is part of our environment. To deal with likely salt will be passed when you ask for it; how we behave with respect to it we should look arriving in front of a locked door doesn’t make there instead of searching inside for drives, feel- it more likely you’ll fi nd the key in your pocket. ings, hunger pangs or other inner causes. No one But you’ll be more likely to ask for the salt or looks inside us to decide whether we’re hungry. to reach into your pocket. In each case some- We think we know better than anyone else what’s thing neutral (the salt or the key) has become inside, but we often judge our own hunger by reinforcing. our behavior: I didn’t realize I was so hungry, we The semester after my class on discrimina- say, while scarfi ng down a meal. Certainly we’re tion with Schoenfeld , I took his next course in the different when hungry than when well-fed, and Experimental Psychology sequence. It was called it’s worthwhile to fi nd out how we’re different. Motivation. This too was a laboratory course, but But, like the probability of eating, hunger can’t this time the experiments were with rats rather be pulled out of an organism and stuck under a than humans. One involved arranging schedules microscope. Sure, physiological systems and pro- of what were then called conditioned reinforc- cesses affect the probability of eating, but they’re ers. Courses with this title now would prob- not what hunger is. ably cover the topic by providing a systematic Here’s a Skinner quotation: “In our everyday survey of motivational systems—hunger, thirst, use of the word…we ordinarily attribute hun- sex, social drives, and so on—concentrating on ger to an animal only because it eats or because physiological mechanisms and perhaps specu- it exhibits behavior that we have frequently lating on the evolutionary or social precursors observed to be followed by eating” (Skinner, that might have created them. The fi elds are so 1932, p. 22). He goes on to say, “the use of the broad the coverage could hardly be exhaustive. word depends on the conditioning of the experi- We considered the meaning of hunger in the menter.” Skinner identifi es the problem as one context of discussions of a hypothetical caveman of verbal behavior, anticipating critical argu- who caught and caged a small animal and then ments about the origins of our language of pri- began to observe its eating habits. The word hun- vate events. ger seemed so much a term for a private felt state that buying the argument for how the term must have evolved from observations of behavior ASSESSING REINFORCERS came slowly (Catania, 1997b). Counterintuitive as it may seem, the point was the term couldn’t The topic of motivational or establishing opera- have acquired any function in everyday lan- tions is about those events that make stimuli guage except to the extent it was based on obser- more or less effective as reinforcers or punishers. vations of the likelihood of eating as a function We’ve devoted lots of attention to reinforcers so of time since the last meal and other publicly far, and that’s as it should be. To get to know observable variables. other people, one of the most important things 109 110 • Part IV. Learning Without Words: Operant Classes you can look for is the reinforcers maintaining man & Piazza, 2011). Sometimes such problems their behavior. Some are obvious, as when we require interventions and sometimes they take notice what people spend lots of time doing: care of themselves. For example, a parent might are they mostly involved in reading or music or be seriously concerned about a child’s persis- athletics or social events or politics? The ques- tent thumb-sucking when suddenly it drops tionnaires on values sometimes used to advise out. This might happen because new social rein- people about the careers for which they may be forcers emerge as the child gets older: at some best suited often ask directly or indirectly how point the social consequences of thumb-sucking they like to spend their time. become more important than the oral sensory When I was close to offi cially becoming a ones. major in psychology I discussed my options Identifying reinforcers is crucial to the appli- with Schoenfeld. One question he asked was cations of behavior analysis. Descriptive analy- whether I wanted to go into the fi eld because sis is a method for assessing and comparing I thought I could use it to solve my own prob- reinforcers (Vollmer, Borrero, & Wright, 2001). lems. I was puzzled by the question, though For example, one reinforcer may be masked by I knew that then, as now, clinical psychology another that is more potent, so its effectiveness was attractive to many students. I probably said must be assessed not only when the more potent something about being fascinated by the quan- reinforcer is available but also when it isn’t titative data I’d seen in his experimental classes (Francisco, Borrero, & Sy, 2008). A major com- and in Keller’s introductory course. I think he ponent of behavioral economics is assessing the was probing whether the reinforcers maintain- relative potency of different reinforcers based ing my academic behavior were fundamentally on the behavior they maintain; a measure called about fi nding out how behavior works. Given value summarizes that potency. where things went from there he must have Negative reinforcement mustn’t be over- approved of my answer. looked (Flanagan, Goldiamond, & Azrin, 1958). In my own interactions with students after For example, aggressive or fearful behavior in I began teaching, I often took Schoenfeld’s lead dogs and other pets is often interpreted as elic- and probed potential majors about why they ited or emotional behavior, but it may be better wanted to enter the fi eld. In giving students viewed as operant behavior reinforced by keep- advice it made a difference to learn whether ing aversive events at a distance. A growling dog they mostly wanted to help others or learn is less likely to be approached than a placid one. about behavior or make money or have a chance A fearful cat is more likely to run and hide when to meet well-known scholars such as Skinner. strangers visit than one that’s more relaxed. Some were single-minded about one or another Contingencies like these have been exploited but more often it was a mix. An inventory of to shape calmer behavior in pets whose behav- the reinforcers maintaining someone’s behav- ior includes aggression and other topographies ior provides a personality test of sorts. Your usually interpreted as emotional (Rentfro, 2010, judgments of other people can change dramati- Nov/Dec; Rosales-Ruiz, 2011). With an aggres- cally if you believe they’re doing things based sive dog, for example, another dog is gradually on friendship or love and then discover they’re brought closer until aggressive behavior begins. instead doing it for other reasons: e.g., to make The intruder dog is withdrawn only when the money, advance professionally, get even with target dog reduces its aggressive behavior. Soon someone, achieve a position of power, or escape the intruder dog is again brought closer and the from aversive responsibilities. Reinforcers can procedure is repeated. Gradually the intruder be diagnostic. is brought closer still as the criterion reduction Reinforcers are also changeable. Behavioral in aggression becomes stricter. The aggressive pediatrics deals with both the common and the dog becomes a friendly one. These dramatic unusual behavior problems of children (Fri- changes in behavior can be shaped in a few days Chapter 13. Motivating Variables and Reinforcer Classes • 111 or even hours and can be maintained by similar REINFORCER CLASSES AND contingencies. The distancing that shapes these REINFORCER-SPECIFIC EFFECTS changes in behavior involves negative reinforce- ment: the intruder’s approach is aversive, and In three-term contingencies, the fi rst term, ante- using the intruder’s retreat to reinforce calm cedents, includes both motivating or establish- behavior creates behavior more acceptable to all ing operations and the stimulus classes defi ning concerned, not just the pet owners. discriminated operants. The second term, behav- Evolutionary contingencies made some ior, corresponds to the response classes called reinforcers signifi cant: e.g., food, water, sexual operants. The events included in the third term, contact, air. Others probably emerged as by- consequences, are also classes of events (Cuvo, products of those motivational systems. These 2000). Successive reinforcers arranged in experi- are sometimes called addictive. Some criteria mental settings may be similar but they’re not exist for classifying some reinforcer dependen- identical. Pellets produced by a rat’s lever presses cies as addictions, but the term is often used more will never be totally uniform. Pieces of grain in broadly. As a long-distance runner, can you get a pigeon’s feeder will always differ somewhat in addicted to endorphin highs? Can you become shape and color. A parent’s reinforcing hugs or addicted to a person? To a political stance or a smiles or positive comments will inevitably vary religion? To dependence or co-dependence? from one instance to the next. Aren’t we already addicted to food and drink? The self-injurious behavior (SIB) of chil- What then do we say if food is ingested but then dren with severe developmental issues may be rejected, as in bulimia, or is no longer reinforc- maintained as a higher-order class because it ing and perhaps even becomes aversive, as in produces attention. By sharing its consequences anorexia ? with other responses like shouting obscenities These are all questions about reinforcers. It’s or throwing things, it may become part of the useful to know how they become established, larger class, attention-getting behavior. Within as in the transition from casual to habitual drug this class some responses may be more probable use. It’s also useful to identify the components than others or may be differently available in dif- of such reinforcers. For example, cigarettes ferent settings (Lalli et al., 1995). For example, a are a vehicle for nicotine , but nicotine by itself child may be more likely to engage in self-injury isn’t addictive (Killeen, 2011); people don’t get if nothing to throw is close at hand, or a child hooked on nicotine patches. Cigarette manufac- might be more likely to shout obscenities given turers may have exploited the interaction of nic- one audience than given another. Nevertheless, otine with other products of smoking as a route their membership in a common class makes it to addiction. Cigarette smoking is especially likely they’ll vary together. relevant to self-control, which involves imme- But what if attention from staff members on diate consequences pitted against opposed this child’s hospital unit doesn’t function like later ones, as when the immediate effects of the mother’s attention when she visits? If we inhaling cigarette smoke are pitted against fi nd one kind of attention can’t substitute for remote effects such as lung disease or cancer. the other, we might best treat attention from In abstinence programs, smokers must commit these two sources as two separate reinforcer to nonsmoking, and what we know about ver- classes. This is important to know, because bal behavior suggests they’re more likely to be the behavior shaped by attention from the successful if their verbal commitment has been staff during therapeutic interventions may be shaped during their own talk about smoking incompatible with the behavior shaped by the than if it’s grown mainly out of instructions mother’s attention at home. An effective treat- given by others. Where verbal behavior comes ment program must shape the mother’s behav- from sometimes may be as important as what ior as well as the child’s or the treatment gains it says. 112 • Part IV. Learning Without Words: Operant Classes realized on the unit will be lost soon after the for them to be discriminated. For example, if a child’s discharge. pigeon’s pecks on one key produce a green light Operant classes are created by common inside its feeder and those on a second key pro- consequences. These consequences are the glue duce a red light, with the lifting of the feeder holding the classes together. But what does tray requiring a peck on a left switch inside the this imply about using a common reinforcer feeder given green and on a right switch given across components of a complex task. In a typi- red, the two routes of access to the food are likely cal matching study, for example, all correct to function as two different reinforcer classes responses, whether to one comparison stimu- (Catania, 1969). It’s also important to distinguish lus or the other, produce the same reinforcer. between common consequences and common If the reinforcers enter into a single functional contingencies. Different reinforcement sched- class along with the stimuli and responses of an ules arranged for two different response classes arbitrary-matching task, using this procedure will typically maintain different performances may be a mistake. While the contingencies work even if both produce the same reinforcer. Here to separate the different matching classes, such the different contingencies arranged for the two as green pecks given a square sample and red response classes matter most. pecks given a circle sample, the common rein- forcer class may work instead to keep them together, just as a common reinforcer keeps all CONDITIONAL OR forms of lever pressing together in a single oper- CONDITIONED REINFORCERS ant class. The problem can be addressed by arranging Once reinforcers are available they can be used different outcomes each correlated with a differ- to create new ones. Some reinforcers, called con- ent class of discriminative stimuli and responses. ditional or conditioned, become effective through When this is done, acquisition of stimulus-con- their relation to other reinforcers. Some of those, trol tasks such as matching-to-sample is typically called unconditional or unconditioned, aren’t enhanced (Urcuioli, 1985, 1991, 2005; Urcuioli really endpoints but are instead convenient & DeMarse, 1994). This works with both non- places to switch from talking about reinforcers humans and humans (Dube et al., 1993; Fedor- to talking about physiology. For example, foods chak & Bolles, 1986). In arbitrary matching by may seem fundamental but many are acquired children, they may learn slowly even with cor- tastes and can be traced to more basic sources. rection procedures. If the tasks are modifi ed so Chewing and swallowing are followed by diges- responses from the different problem classes tion and eventually the circulation of nutrients in each produce a different visual reinforcer, such the bloodstream, but for our purposes it’s usu- as different cartoon pictures displayed on a com- ally suffi cient to talk about the reinforcing effects puter screen, the acquisition of accurate arbitrary of an opportunity to eat. matching usually proceeds more rapidly than Conditional reinforcers related to many dif- when all responses produce the same reinforcer ferent source reinforcers, of which money is the (e.g., Dube & McIlvane, 1995; Dube et al., 1989; prominent example, are generalized reinforc- Kastak, Schusterman, & Kastak, 2001; Pilgrim, ers . But verbal behavior may be a more critical Jackson, & Galizio, 2000; Urcuioli, 2005). The determinant of the functions of money than how moral is whenever possible we should arrange the money is related to specifi c other reinforc- different reinforcers rather than a single rein- ers; for example, the response most likely to be forcer for the shaping and maintenance of dif- reinforced by a direct deposit into your bank ferent response classes. account is checking your account balance. Just as This needn’t mean two reinforcer classes variable reinforcers may maintain more behav- must differ qualitatively; it’s probably suffi cient ior than fi xed ones (Steinman, 1968), conditional Chapter 13. Motivating Variables and Reinforcer Classes • 113 reinforcers based on a broader range of rein- Figure 13–1 illustrates how this procedure forcers are likely to be more potent than those can be arranged on two pigeon keys. The initial- based on a narrower range. In the clicker train- link schedules are typically equal and relatively ing of pets (Pryor, 1999, 2009), it’s easy to control short variable-interval schedules operating con- the relationship between the clicker and edible currently (e.g., VI 30-s VI 30-s: each schedule treats, but the clicker becomes far more effective arranges a reinforcer for the next response after if it’s broadly generalized to a variety of other every thirty seconds on the average). Each ter- reinforcers, such as playful interactions or access minal link operates separately and the pigeon to new environments. returns to the initial links after the reinforcer for that terminal link is delivered. In this example, the conditional reinforcers are the green and red CONCURRENT-CHAIN terminal link stimuli. In their presence responses SCHEDULES: PREFERENCE produce food, so we can ask how potent each one is in maintaining initial-link responding. Widely used procedures for assessing the This procedure separates the reinforc- potency of conditional reinforcers are concur- ing effectiveness of the terminal link from the rent-chain schedules (Herrnstein, 1964b). Two contingencies maintaining responding in that schedules operate concurrently, but their respec- link. If we looked at concurrent responding tive reinforcers are entries into other schedules. maintained by a variable-ratio schedule on the More technically: these schedules include con- left key and by differential reinforcement of current initial links, and a consequence of initial- low rate on the right key, we’d see a lot more link responding is entry into a terminal link, in responding on the left than on the right. But which another schedule operates. For example, that wouldn’t tell us whether a pigeon prefers one of two equal initial links might give access the left-key schedule over the right-key one. The to a fi xed-ratio (FR) schedule, in which the last of same point holds for responding in the terminal a fi xed number of responses is reinforced, while links: schedule A might produce a different rate the other might give access to a fi xed-interval of pecking than schedule B, but this rate differ- (FI) schedule, in which the fi rst response after a ence couldn’t tell us how hard the pigeon might fi xed time since some event is reinforced. More work on the white initial-link keys to get to one responding in the fi rst initial link than the sec- or the other terminal link. We judge preferences ond would tell us the FR is preferred to the FI; among situations not by how much behavior the opposite outcome would tell us the FI is pre- they produce but by the relative likelihoods ferred to the FR. with which an organism enters them. Suppose I’m standing in front of two doors Concurrent chains have shown that rein- behind each of which is an area where some forcement rate is a more important determinant activity is available and I must knock on one of of preference than the number of responses per the doors to gain access to the area behind it. reinforcer (Neuringer & Schneider, 1968) and Once I’ve fi nished the activity in that area I can that variable schedules are preferred to fi xed come around and knock on the doors again. After schedules (Herrnstein, 1964a). Studies of prefer- a while I might fi nd the activities behind the left ences among various parameters of reinforce- door are more interesting than those behind the ment schedules can be technically complex, right door, so I end up knocking more often on because they must control for differences in time that door than the other. Knocking on the doors or responses per reinforcer in terminal links and are my initial links and what I do after entering for occasional biases toward specifi c colors or an area is my terminal link. If I knock most often sides. on the left door you’ll know its activities are the When two or more responses are available at ones I prefer. the same time they’re sometimes called choices 114 • Part IV. Learning Without Words: Operant Classes

Figure 13–1 Diagram of a concurrent-chain procedure as arranged for a pigeon’s key pecks. In initial links, equal but independent VI schedules operate on each white key (W). According to its VI schedule, left-key pecks produce terminal link A, in which the left key is green (G), the right is dark, and pecks on green produce reinforcers accord- ing to schedule A. Similarly, according to its VI schedule, right-key pecks produce terminal link B, in which the right key is red (R), the left is dark, and pecks on red produce reinforcers according to schedule B. and engaging in one or the other is sometimes chain procedures are especially well-suited for called choosing. But is choice something I do the analysis of preferences. just before responding or is it my responding itself? If it’s something I do just before respond- Self-Control ing, then it too is behavior and I should be able to analyze it. For example, I may look over We usually speak of self-control when we forgo a menu before placing an order; I may read a relatively immediate consequence in favor of reviews before going to a movie; I may list alter- a later larger one. Both the immediate and the natives and their pros and cons before making deferred consequences may be reinforcing or an important decision. But none of these is a aversive (Rachlin, 1974; Skinner, 1953). A small choice; they are all preliminaries to a choice, reinforcer now may be declined in favor of a which is simply what I end up doing (cf. Skin- large one later, as in skipping a small purchase ner, 1950). In any case, a dominance of one now and saving up for a larger one later. We alternate over others in a sequence of choices might call this self-control or self-restraint. If this is typically called a preference, and concurrent- kind of behavior under these kinds of contingen- Chapter 13. Motivating Variables and Reinforcer Classes • 115 cies characterizes self-control, we should be able of food; a peck on green produced four seconds to synthesize it in the laboratory. of food after a four-second delay. In terminal- Concurrent-chain schedules using the proce- link B, the keys also were dark for T seconds; dure in Figure 13–2 provide an example of how then only the green key lit. As in the other ter- to do it (Rachlin & Green, 1972). Initial links minal link, a peck on green produced the large consisted of concurrent FR 25 FR 25 schedules: reinforcer after a four-second delay. every 25th peck on a key produced its terminal Confronted with both red and green in A, link. With this arrangement, the pigeon could the pigeon almost invariably pecks red, produc- enter one terminal link more often than the ing the small immediate reinforcer and not the other. In terminal-link A, the keys were dark for large delayed one; this has been called impul- T s and then lit red and green respectively. A siveness. Confronted with only green in B, the peck on red immediately produced two seconds pigeon necessarily produces the large delayed reinforcer. But what about the pigeon’s prefer- ence for A versus B, in its initial-link respond- ing? The answer depends on T, the time until the terminal-link keys are lit. When it’s short, say one second, the pigeon usually produces A and then pecks red. When it’s longer, the pigeon is more likely to produce B, in which only green is available. Figure 13–3 shows the relative rate of pecking the left initial-link key (left initial-link pecks divided by total initial-link pecks) as

Figure 13–2 A concurrent-chain procedure for syn- thesizing some properties of impulsiveness, commit- ment and self-control. According to FR 25 schedules, pecks on white (W) initial-link keys are followed after T s by terminal links. In terminal-link A, red (R) and green Figure 13–3 Relative initial-link pecks producing (G) keys respectively make an immediate small rein- terminal-link A (left initial-link pecks divided by total forcer or a delayed large one available. In terminal-link initial-link pecks) as a function of T, the time to the B, the green key alone makes only the delayed large onset of the terminal-link keys. (Adapted from Rachlin one available. (Adapted from Rachlin & Green, 1972) & Green, 1972, Table 1) 116 • Part IV. Learning Without Words: Operant Classes a function of T. As T increased from 0.5 to 16 seconds, the proportion of pecks producing A decreased; the pigeon became more and more likely to enter B and produce the larger rein- forcer. Translating to more colloquial terms, when the opportunity to choose was coming up very soon, the pigeon was likely to be impul- sive; when it came up later, the pigeon was likely to show self-control. During initial links, the time to food equals T for the small reinforcer but T plus the delay for the large one. When T is short, this difference is relatively large (e.g., with a T of one second, the respective delays are one and fi ve seconds) and the short delay outweighs the difference in rein- forcer magnitudes. When T is long, however, the Figure 13–4 Delay-of-reinforcement gradients for difference becomes relatively small (e.g., with a an early small reinforcer at A and a later large one T of ten seconds, the respective delays are ten at B. Before time C the B gradient is higher than the and fourteen seconds) and the difference in rein- A gradient, so the B reinforcer should be preferred; forcer magnitudes becomes effective. By produc- after time C the A gradient is higher, so the A reinforcer ing B when T is long, the pigeon commits itself should be preferred. to the large reinforcer even though it wouldn’t do so at the onset of green if red were also pres- reinforcers in favor of large delayed ones, per- ent. For this reason, pecks that produce B have haps because human self-control involves verbal been called commitment responses: they guarantee behavior. But if more complex processes operate the large delayed reinforcer by making the small in our self-control, we can only identify them by immediate one unavailable . seeing whether human cases are consistent with Figure 13–4 illustrates two hypothetical analyses based on reinforcer delays. For this rea- delay-of-reinforcement gradients, one produced son, these behavior syntheses provide an essen- by a relative small reinforcer at A and the other tial reference for analyzing self-control. by a later large reinforcer at B. If at any time the pigeon’s peck is determined by which gradient Procrastination has the higher value, then where the A gradient is higher as the pigeon approaches time C, it will Self-control may involve deferring a small imme- commit to reinforcer B, showing self-control. At diate reinforcer for a larger later one, but it may C the gradients cross over so the A gradient is also involve immediate and deferred aversive higher; between times C and A it will choose events. For example, a small reinforcer now may reinforcer A, showing impulsiveness. be declined in favor of avoiding a large aversive With concurrent-chain procedures we can event later, as when an alcoholic refuses a drink study impulsiveness and commitment with and avoids a later hangover or when a smoker immediate and delayed reinforcers or immedi- declines a cigarette and avoids later lung disease; ate and delayed aversive stimuli (Deluty, 1978). a small aversive event now may be accepted in We can also see whether commitment is modi- favor of a large reinforcer later, as when strenu- fi able and whether this synthesis of self-control ous exercise prepares an athlete for a major event involves variables comparable to those operat- later; and a small aversive event now may be ing in human behavior (Grosch & Neuringer, accepted in favor of avoiding a large one later, 1981). We humans are far more likely than as when a maintenance dental cleaning avoids pigeons to forego small and currently available gum disease or toothache later. Chapter 13. Motivating Variables and Reinforcer Classes • 117

Most situations we’d discuss as examples of one terminal link and only a single key in the procrastination involve deferring a task from other, we can ask whether pigeons prefer free now until later, as in playing an online game choice to forced choice (Catania & Sagvolden, now rather than preparing for an exam coming 1980; Sran & Borrero, 2010). up later in the week. Because the task is deferred, This procedure uses equal VI 30-s schedules one of two assumptions seems reasonable: (i) the operating in initial links; on either initial-link key, task has some aversive elements, or (ii) the task pecks produce their terminal links once every is weakly reinforcing relative to reinforcers avail- thirty seconds, on average. Four keys are lit red able for what can be done right now. While writ- or green in both terminal links. In free choice, ing this passage, I glanced over at a paperweight pecks on any of three green keys are reinforced on my desk given to me as a gift. Inscribed on it is at the end of a fi xed interval (FI 30-s); pecks on the saying “The ultimate inspiration is the dead- red are ineffective. In forced choice, the same line.” A deadline may reduce procrastination, FI 30-s schedule operates for pecks on green; but it works best as the deadline approaches; pecks on any of the three red keys are ineffec- completing a major task under time pressure is tive. The FI schedule delivers a reinforcer for the often more aversive than getting it done earlier fi rst peck after thirty seconds in either terminal at a more leisurely pace. On the other hand, an link, so the pigeon earns reinforcers after about increase in competing immediate reinforcers will the same time and with about the same respond- probably enhance procrastination. ing whether doing so on one of three keys in free What might we do to synthesize procrastina- choice or just one key in forced choice. tion in the laboratory? Suppose we start a pigeon In these schedules. pigeons prefer free choice out with forty-second trials in which free food to forced choice. Furthermore, control proce- is delivered on the average once every twenty dures show the preferences don’t depend on seconds in the presence of either green or red terminal-link differences such as responses per houselights; the pigeon can initiate a green or a reinforcer or how responses are distributed red trial by pecking either a green or a red key to keys or colors or locations (Catania, 1975a). (Mazur, 1996, 1998). Six seconds into a green Could it be the pigeon has learned that differ- trial, the green keylight turns on and the pigeon ent contingencies exist in free and forced choice? can get back to the free food condition by peck- If one key fails during free choice, the other is ing the green key fi ve times. Twenty seconds available as an alternative; if the single key fails into a red trial, the red keylight turns on and the during forced choice, no other is available to fall pigeon can get back to the free food condition by back on. If free-choice preference is learned this pecking thirty times. We now look at whether the way, we should be able to reverse it by making pigeon prefers to initiate green trials or red ones more reinforcers available during forced choice and discover it prefers red, containing the larger than during free choice. But if we do so, the later task, over green, containing the smaller ear- effects are only temporary; the free-choice pref- lier task. Pigeons, like people, procrastinate. erence returns when the reinforcers in the termi- nal links become equal again. We can’t produce Free-Choice Preference a durable forced-choice preference (Catania, 1980b). Concurrent-chain schedules can be used to deter- Perhaps the free-choice preference has a phy- mine whether opportunities for free choice can logenic basis. For example, food supplies are function as reinforcers. Questions about freedom sometimes lost to competitors or disappear in are questions about whether organisms prefer other ways, so choosing environments with two alternatives. We may believe it’s good to have or more food sources will probably have advan- options, but can we demonstrate such prefer- tages over choosing environments with only one ences and are they important to pigeons as well food source. If such preferences exist even in the as to people? By making two keys available in behavior of pigeons, they can’t be mere products 118 • Part IV. Learning Without Words: Operant Classes of human cultures (cf. Page & Neuringer, 1985, p. signaled whether or not the rat has been food- 451). They may occur because evolutionary con- deprived. tingencies have selected organisms who prefer Here’s an example. Shock is delivered to a free choice to forced choice. If so, we may be able rat when a light is either on or off; a lever press to mask it temporarily, as by punishing respond- removes the shock for a while when the light’s ing during free choice but not forced choice, but on but not when it’s off (Bersh & Lambert, 1975). we won’t be able to eliminate it. This conclusion Under these circumstances the rat comes to press is based on data from pigeons, but maybe this the lever when the light’s on but not when it’s makes free-choice preference even more funda- off. The discriminative stimulus here is the light, mental, because it seems to imply that this aspect because the contingency between lever presses of our human concept of freedom has biological and shock removal is signaled by whether the roots. If so, no political entity can take it away. light is on or off. The shock doesn’t function as a discriminative stimulus because it doesn’t sig- nal the operation of the contingency. It makes MOTIVATING EVENTS IN shock-free periods reinforcing and its presenta- AVERSIVE CONTROL tion is therefore a motivating operation. The contingencies operating when the light The presentation of aversive stimuli makes their is off in this example are properly called extinc- removal reinforcing. Even more so than with tion contingencies. Lever presses remove shock positive reinforcement, these establishing effects during light but not during dark, but shock must be distinguished from discriminative, elic- absence would be an effective reinforcer dur- iting and other effects of stimuli. When we deal ing either one given appropriate contingencies. with positive reinforcement, we usually speak This wouldn’t be so if shock were never pres- of motivating operations such as satiation and ent when the light’s off. And even without cor- deprivation. Food is less likely to reinforce the related lights a rat might come to discriminate behavior of a rat that very recently ate than of periods during which its presses turn off shock one that hasn’t eaten for some time. But we must from those in which they don’t, but that would present negative reinforcers to get behavior depend on its sensitivity to relations between its going. The motivating operation is presentation; behavior and consequences rather than to the we’d call that satiation were the stimulus food presence of shock per se. instead of shock. Why the motivating event in The aversive stimulus is the motivating or the case of negative reinforcement is the aver- establishing event because there’s no reason to sive stimulus itself may not be obvious. escape or avoid an aversive stimulus unless it’s The aversive stimulus is the motivating event either actually or potentially present. A similar because there’s no reason to escape or avoid an case for shock presentations as motivational aversive stimulus unless it’s either actually or can be made for avoidance. It should then be potentially present. It’s tempting to think of the a straightforward exercise to extend the argu- aversive stimulus as signaling a contingency, ment to cold or bee stings or loud noise or other but contingencies in which responses turn off aversive stimuli. In each case contingencies are a shock can’t exist in the absence of shock. This about the consequences of responding whereas is different from positive reinforcement; when motivating events are about whether those responses produce food, the contingency can be consequences are important enough to serve as reinforcers. Part V: LEARNING WITHOUT WORDS: CONTINGENCIES Chapter 14 Parameters of Reinforcement: Schedules and Delays

It is a capital mistake to theorize before you have all the evidence. It biases the judgment. —Sir Arthur Conan Doyle, 1888

Not all classes of responses have consistent con- time or rate requirements can be combined to sequences. The reinforcement of some responses produce more complex schedules. We’ll look at but not others, intermittent or partial reinforce- effects of reinforcement schedules as valuable ment, is a general feature of behavior. I don’t experimental tools and as ubiquitous properties always fi nd what I’m looking for when I shop of behavior in their own right. and I don’t always get a satisfactory reply when Let’s return to the telephone. Suppose I call I ask a question. Suppose I phone a friend. a cousin whose voicemail is inactive and get no Sometimes I’ll get to talk to my friend, but other answer. My chances of getting an answer later times I’ll get no answer or a voicemail greeting will depend on when and not on how many or someone else will answer. Games of chance times I call. My cousin will answer only if I call routinely provide intermittent consequences. at a time when my cousin is available; if not it Continuous or regular reinforcement, the rein- won’t make any difference how many times I try forcement of every response within an operant calling. Some varying time will pass depending class, is the exception rather than the rule. For only my cousin becoming available. I must call this reason, we must examine the effects of sched- at a good time to get an answer. This is an every- ules of reinforcement , arrangements specifying day approximation to a variable-interval or VI which responses within an operant class will be reinforcement schedule. It schedules a reinforcer reinforced (Ferster & Skinner, 1957). for a single response occurring after a specifi ed Three basic schedules are (i) those allowing time has elapsed, and that time varies from one a response to be reinforced after some number instance to the next; earlier responses do noth- of responses (ratio schedules), (ii) those allowing ing. A VI schedule is designated by the average a response to be reinforced after some elapsed time to the availability of a reinforcer. time since some event (interval schedules), Now I’d like to get something from a vend- and (iii) those allowing a response to be rein- ing machine, but the machine won’t accept the forced depending on the rate or timing of prior change I have. I can start asking those who pass responses (schedules that differentially reinforce by for change and maybe I’ll get what I need. rates or interresponse times). These number, In this case, whether I get the change doesn’t

119 Chapter 14. Parameters of Reinforcement: Schedules and Delays • 127 with increased reinforcement. The performance (i), no other responses intervene; in (ii), responses is highly resistant to change though its rate c through h occur between them. One way to remains relatively low, and again illustrates the interpret these cases is to say half the responses independence of response strength and response are reinforced in the fi rst sample whereas only rate (Nevin & Grace, 2000). In DRL and related one-eighth are reinforced in the second. But that procedures involving latencies or temporal assumes the effect of the reinforcer is limited just judgments, with both humans and nonhumans, to a, even though it follows the other responses responding tends to overestimate short time cri- too. The alternative is to assume the reinforcer’s teria and to underestimate long ones: the longer effect depends on its relation to all the preceding the time, the more likely it is that responses will responses, some after longer delays than others. be emitted too soon (Catania, 1970). From that perspective, only two responses were Other classes of differential-reinforcement reinforced in the fi rst sample whereas eight were schedules set other temporal criteria. For reinforced in second. The partial reinforcement example, the fl ip side of DRL is the differen- effect isn’t paradoxical, because schedules allow tial reinforcement of high rate or DRH schedule, each reinforcer to act on several responses at a usually specifi ed by the number of responses time instead of just the most recent one. required within a time window. For example, an Early responses in sequences ending with a 8-peck/2-second DRH delivers a reinforcer any reinforcer contribute less to future responding time the pigeon pecks eight times in less than than later ones, because they’re separated from two seconds. If a pigeon slows down for any the reinforcer by longer delays (Dews, 1962). This reason, pecking fails to meet the rate criterion as means we need to know the form of the delay often and thus produces fewer reinforcers. This gradient to interpret schedule effects. The delay reduces the rate even further, so this vicious gradient has entered into mathematical models circle will probably end in extinction. To restore of operant behavior (e.g., Catania, 2005d; Cata- high rates, the experimenter may have to start nia, 2011a; Killeen, 1994), but technical problems over from scratch. complicate its experimental determination. For example, if we arrange delayed reinforcers some fi xed time after a response, either we must allow DELAY OF REINFORCEMENT additional responses to occur in the interim, in

The suggestion that the FI scallop involves a gra- dient of delayed reinforcement carries the impli- cation that the reinforcer produced by the last of a sequence of responses has effects depend- ing on its relation to all preceding responses and not just the one that produced it. In the early days of schedules, so-called partial reinforcement effects seemed paradoxical: more behavior was generated by reinforcing a fraction of the total responses than by reinforcing all of them. But looking at schedules as the delayed reinforce- ment of all responses preceding the reinforced response suggests that partial reinforcement works as it does because it allows each reinforcer Figure 14–6 Schematic of reinforced response to reinforce many responses instead of just one. sequences: (i) the last of two responses is reinforced; (ii) the last of eight is reinforced. Reinforcement Figure 14–6 makes the point schemati- probabilities are respectively 1/2 and 1/8, but only cally with two time samples including an early two responses are followed by the reinforcer in i as response, b, and a later reinforced response, a. In opposed to eight responses in ii. Details in text. 128 • Part V. Learning Without Words: Contingencies which case the time from the last response to reinforce precede behavior we’ve targeted for the reinforcer will be shorter than the one we reinforcement, we may mistakenly conclude scheduled, or we can reset the delay with each the reinforcer isn’t doing its job very well. But response, in which case the timer reset will dif- if that other behavior is maintained by delayed ferentially reinforce pauses at least as long as the effects of our reinforcer, it may simply be doing delay and that differential reinforcement will be very well a job other than the one we wanted it confounded with the delayed effects of the rein- to do. We should keep behavior we don’t want forcer (Catania & Keller, 1981). We can’t avoid to reinforce from getting consistently close to the problem by presenting a stimulus during reinforcers produced by other responses, and the delay, because that substitutes an immedi- we should again be reminded that shaping is ate conditional reinforcer for the delayed one, often more art than science. but we can address it by arranging reinforcers for one response class and studying its delayed Delays and the Ratio-Interval effects on others (Bruner et al., 1998; Catania, Difference 1971; Escobar & Bruner, 2007). Some practical implications may be more What makes ratio response rates higher than important than the technical problems. The interval rates? Procedures like yoked schedul- effects of delayed reinforcement aren’t restricted ing rule out reinforcement rate and other input- to responses in a single operant class. For output alternatives as candidates. The question’s example, a pigeon’s pecks on one key may be been around a long time (Skinner, 1938). One maintained because they’re followed later by suggestion was the differential reinforcement reinforced pecks on another key (Catania, 1971). of interresponse times or IRTs (Anger, 1956). In Now consider a task involving correct responses VI schedules, the more time that passes with- and errors over successive trials (e.g., matching- out a response, the more likely the interval has to-sample). If we reinforce all corrects and repeat ended and the next response will be reinforced; any trial with an error until the pigeon gets it reinforcement probability grows with IRT. In right, we guarantee any string of errors will end VR sequences, reinforcement probability is after some delay with a reinforced correct. Cor- independent of IRT; reinforcement probability rects will probably dominate eventually because is constant. The higher reinforcement probabil- they’re the ones most closely followed by the ity for long than for short IRTs in VI is in effect reinforcer, but errors may diminish only slowly differential reinforcement of long IRTs, which and even continue at a modest level because corresponds to a decrease in response rate. This they too are reliably followed, after a delay, by a happens only in VI schedules, so VI rates should reinforced correct. generally be lower than VR rates. The IRT argu- The moral of this story is that teachers ment seems reasonable, but the rapidity with and clinicians must be alert for sequences in which the VI-VR rate difference emerges works which errors are followed by reinforced cor- against it. Acquisition of DRL performance usu- rects, so errors aren’t strengthened along with ally takes a while whereas the VI-VR difference the reinforced corrects. A reinforcer following emerges quickly. Attributing an outcome that a sequence of corrects will do a lot more good develops quickly to a process that occurs more than one following a single correct after sev- slowly isn’t persuasive. eral errors. Teachers must then judge whether Delays can provide the answer. The delayed corrects are so infrequent they should be rein- effects of each reinforcer work only back to the forced even though preceded by errors or fre- preceding reinforcer (Catania, Sagvolden, & quent enough that the reinforcer can wait until Keller, 1988); each reinforcer only affects its own the student has made several corrects in a row. responses. Ratios allow responses to bunch up When response classes we didn’t intend to closer and closer to reinforcers and as they do Chapter 14. Parameters of Reinforcement: Schedules and Delays • 129 so the time between reinforcers shortens, which has no fl oor indicator, the only consequence means delays get shorter still, so responding of my pressing the elevator button will prob- again increases until it gets to the point where ably be the arrival of the elevator after a vari- rates can’t get much higher. In intervals, how- able delay. But the elevator might also arrive ever, getting closer adds more responses but because someone pressed the button earlier and doesn’t shorten time between reinforcers. If then took the stairs instead, or because someone responding increases, more responses will occur already in the elevator pressed the button for during that interval, but many will be followed the lobby fl oor, or because this elevator always by the reinforcer by long delays and there- returns to the lobby after calls to other fl oors. fore contribute weakly to the maintenance of One day the elevator door might open just as responding. I reached out to press the button; on another it Think of each reinforcer as generating some might arrive and leave and arrive again, even potential for responding that’s greater the shorter though I didn’t press the button in the mean- the delay. If the potentials for each response time because I was standing there talking with leading up to the reinforcer are summed and if someone. When the elevator arrives, I can never each emitted response reduces that potential, be sure my button press brought it to my fl oor. differences between ratio and interval respond- Nevertheless, I’ll probably continue to press ing necessarily follow. These relations have been elevator buttons. demonstrated in computer simulations (Berg There are several elevators in the buildings & McDowell, 2011; Catania, 2005d, 2011a) and I typically visit when I’m on campus. The pro- are consistent with models based on ontogenic grams for when the door closes and the elevator selection (McDowell, 2004, 2013). starts after someone has pressed a fl oor button vary from one elevator to another. In one the door closes automatically a few seconds after the REINFORCEMENT SCHEDULES press of a fl oor button and subsequent presses AND CAUSATION do nothing. The door in another elevator also closes automatically but after a couple of seconds The effects of reinforcers depend on the a second press on any fl oor button closes the responses they follow, but reinforcers can follow door. Still another imposes no delay between a responses when either produced by responses press and door closure. With so many variations or delivered independently of responses. Does I’ve occasionally suggested that students work responding change the same way when it pro- out the arrangements for one or more elevators duces a reinforcer as when it happens by acci- as an exercise in analyzing contingencies. Their dent to be followed by a reinforcer? Causal judgments about causation in these cases, like relations between responses and reinforcers mine, will depend not on constant conjunctions affect behavior differently than coincidental of events but rather on a sampling of outcomes temporal contiguities. Organisms are sensitive produced by complex contingencies. to the consequences of their own behavior, and our concept of causation arises at least in part from the distinction between events caused by THE TAXONOMY OF our behavior and those accidentally correlated REINFORCEMENT SCHEDULES with our behavior. The different effects of response-dependent Some parts of the schedule vocabulary are and free reinforcers depend in a complex way logical but others are admittedly idiosyncratic. on how correlations among events are inte- For example, the names for FI and VI versus grated over time. We can illustrate the complex- FT and VT schedules are arbitrary; either pair ity with a human example. If a lobby elevator could as easily have been fi xed-duration and 130 • Part V. Learning Without Words: Contingencies variable-duration, presumably abbreviated FD summarizes some major schedules by provid- and VD. Though FI, DRL and FT schedules all ing schedule names and standard abbreviations require passage of time, what responding does along with brief descriptions of contingencies is different in each. An FI schedule imposes no and notation. conditions on responding during the interval Schedules of reinforcement are power- but one response must occur after it ends; a ful determinants of behavior. It’s one thing to DRL schedule requires time without respond- describe them and another to watch their effects. ing and then the next response is reinforced, It means going “from something read about while responses emitted too soon start the time passively to a practical and powerful activity over; an FT schedule delivers a reinforcer with- that infl uenced large magnitudes of behav- out regard to responding, so responses don’t ior instantly and in orderly ways…. For some, anything. These names emerged incidentally there is a magical sense of playful achieve- as research evolved and are so well established ment when a procedure is converted to orderly they’d be hard to change. behavioral control” (Ferster, 1978, p. 348). In an Most schedule names apply whether sched- article called “Are Theories of Learning Nec- ules are arranged successively without interrup- essary?” Skinner (1950) contrasted behavioral tion or occur within separate trials. For example, accounts of learning with theories appealing to an FI is usually timed from the last reinforcer, dimensions other than behavior. His conclusion but if other events are arranged between succes- wasn’t to reject theories in general but rather sive intervals, timing can begin with the onset only certain types: of some stimulus, such as a color presented on a pigeon key. Also, the vocabulary of reinforce- Perhaps to do without theories altogether is ment schedules can be extended to punishment a tour de force that is too much to expect as (Azrin, 1956). The symmetry extends to sched- a general practice. Theories are fun. But it is uling effects. For example, superimposing an FI possible that the most rapid progress toward schedule of punishment on maintained respond- an understanding of learning may be made ing produces an inverted scallop, a gradually by research that is not designed to test theo- decreasing response rate as the end of the inter- ries.... This does not exclude the possibil- val approaches, instead of the increasing rate an ity of theory in another sense. Beyond the FI schedule of reinforcement usually produces collection of uniform relationships lies the (Silverman, 1971). need for a formal representation of the data Of several noteworthy attempts to classify reduced to a minimal number of terms. A reinforcement schedules more systematically theoretical construction may yield greater (e.g., Schoenfeld, 1970; Schoenfeld & Cole, 1972), generality than any assemblage of facts. none has attained general usage. Appendix III (Skinner, 1950, pp. 215–216) Chapter 15 Discriminated Operants: Stimulus Control

What we see depends mainly on what we look for. — John Lubbock (Lord Avebury)

When you’re a Jet and you’re walking toward others. Sometimes the term has more favorable some gang members down the street, it really implications, as when we say someone has dis- matters whether you can discriminate between criminating tastes. All this may seem worlds your own gang and your rivals, the Sharks . The apart from what makes a pigeon peck a green two gangs dress differently and talk differently key but not a red one. Here, however, I’m argu- and act differently, so it’s easy to tell them apart. ing for a common thread tying all these issues But discriminating isn’t just easy; it’s inevitable. together. You couldn’t keep from discriminating if you What are my credentials? Well, fi rst, I’m an tried. Continuing down the street is likely to expert in discrimination. By that I mean I got a be reinforced if you fi nd yourself among other grade of A in an undergraduate course, Psychol- Jets, but it’ll probably be punished if you end ogy 3, called Discrimination, and I’ve continued up in the middle of a group of Sharks. This is studying and later teaching on that topic ever a straightforward three-term contingency: given since. My course, part of an Experimental Psy- the discriminative stimulus Jets, walking down chology sequence, concentrated mainly on vision the street is reinforced, but given Sharks it’s and hearing and other sensory systems, but its punished. So too for Sharks who see either other content led directly to the topic of discrimination Sharks or only rival Jets down the street. Under learning in the fi eld eventually called behavior such circumstances, would any member of analysis. either gang fail to discriminate based on stimuli Probably more important, I was fortunate in correlated with gang membership? the diversity represented in the environments in When we discuss discrimination in the tech- which I’ve lived and worked. I grew up in the nical vocabulary of behavior analysis, we do so Washington Heights area of Manhattan in New in the context of specifying conditions under York City in an apartment building at the inter- which organisms come to behave one way in section of several ethnic neighborhoods. My the presence of some stimuli and another in the classmates in the local elementary school, P.S. presence of others. When we discuss it in every- 132, though racially homogeneous, included day talk, however, it’s usually in the context of Hispanics, Irish, Scandinavians, Jews, Greeks concerns about how people deal with and treat and Italians, no doubt among a few others. Later, each other. We ask whether some individuals during high school and through my sophomore might be discriminated against based on race year in college, I took summer jobs at Harlem or ethnicity or gender or sexual orientation, or Hospital. I was the only white summer employee whether discriminatory practices within cer- there, where I worked in the dietary department, tain educational or professional or social or the outpatient department, and then as a ward religious institutions might confer advantages clerk substituting for staff members on vacation. or disadvantages on some individuals but not In those days, most doctors and senior staff were

131 138 • Part V. Learning Without Words: Contingencies

Figure 15–2 Generalization gradients as a function of tone frequency after reinforcement of key pecks during a 1000-cycle/s tone (top: 3 pigeons) or after reinforcement during tone and extinction during its absence (bottom: 5 pigeons). Without discrimination training, gradients were relatively fl at; after presence versus absence training, they peaked at the reinforcement stimulus (SD). (From Jenkins & Harrison, 1960, Figures 1 and 2) tions from vertical. In this group, changing the array of squares with an asterisk substituted line orientation affected its distance from the for one of the squares or the same array with vertical extinction stimulus but not its distance no asterisk, as in Figure 15–4. The asterisk is the from the reinforcement stimulus, the absence of distinctive feature of these stimuli, and a pigeon a line. This is an inhibitory gradient. This gradi- will acquire this discrimination far more quickly ent was shallower than the excitatory one, prob- if the asterisk is correlated with reinforcement ably because organisms are more likely to attend ((feature positive) than if it’s correlated with to stimulus properties correlated with reinforce- extinction (feature negative). The pigeon is ment than to those correlated with extinction much more likely to look at the asterisk in the (Dinsmoor, 1995). former case than in the latter (Jenkins & Sains- bury, 1970); human behavior can be similarly Feature-Positive Discriminations captured by such contingencies. The distinctive stimulus appears in different Suppose the alternating stimuli of a reinforce- places in the array over successive trials. As the ment-extinction discrimination consist of an pigeon masters a feature-positive discrimination, Chapter 15. Discriminated Operants: Stimulus Control • 139

Figure 15–3 Excitatory and inhibitory stimulus-control gradients after VI reinforcement of pigeons’ key pecks during one stimulus (SD) and extinction during another (S). For one group, a vertical line was correlated with re- inforcement and its absence with extinction (fi lled circles); for another, the stimuli were reversed (unfi lled circles). Data obtained during nonreinforcement show responding during various line orientations and during absence of the line. (From Honig et al., 1963, Figure 1) it begins to look at and peck the distinctive stim- nate between its presence and its absence and so ulus wherever in the array it’s located. Because learns this discrimination more slowly. it’s correlated with reinforcement, the distinc- Whatever the stimulus, we can always ask tive stimulus becomes a conditional reinforcer, about the features to which organisms attend. so looking at it is reinforced. With the feature- For example, suppose a pigeon looks only at the negative discrimination, however, the distinc- upper edge of a key with a vertical line projected tive stimulus is correlated only with extinction, on it. If we rotate the line from vertical, its top so looking at it isn’t reinforced. If anything, the is gone from the upper edge of the key, which pigeon looks at the squares without an asterisk, now looks more like it did when no line at all which are correlated with reinforcement during was there. In this case, the line is the stimulus in reinforcement trials. A pigeon that fails to look name only: it’s a nominal stimulus. The functional at the asterisk is less likely to learn to discrimi- stimulus is what appears at the upper edge of the 140 • Part V. Learning Without Words: Contingencies

Figure 15–4 In a feature-positive discrimination, responding is reinforced only if a distinctive stimulus is present (top). In a feature-negative one, responding is reinforced only if it’s absent (bottom). The asterisk is the distinctive stimulus and appears in diff erent places in the block of squares over trials. Feature-positive discrimina- tions are easily learned whereas feature-negative ones are learned with diffi culty. key. We can evaluate such possibilities experi- FADING: STIMULUS mentally, for example by removing parts of the CONTROL BY SUCCESSIVE line to see whether its removal produces the same changes in the pigeon’s behavior as rotat- APPROXIMATIONS ing it (Touchette, 1969). For a pigeon’s pecks the point may seem trivial, but it can be exceedingly As usual, our behavior analytic categories have important in application, as when a teacher tries biological parallels. Just as a response may be to fi nd out whether a child is attending to the differentially reinforced during one stimulus words in a storybook or is faking reading by but not another, organisms may be selected attending mainly to the pictures. by evolutionary contingencies in one environ- Chapter 15. Discriminated Operants: Stimulus Control • 141 ment but not in others. Fine discriminations can nated responding available to be shifted to a develop with the co-evolution of sensory and new stimulus dimension. In an experiment on motor capacities, as when the sensitivity of a errorless discrimination, for example, a pigeon’s primate mother’s hearing is selected based on pecks on a red key were shaped with food rein- her responsiveness to the calls of her offspring forcers and then pecking during three-minute while the calls are in turn selected by their dis- periods of red was reinforced according to a VI tinctiveness from calls by the offspring of other schedule (Terrace, 1963a). Periods of red alter- mothers (Rendall, Rodman, & Emond, 1996). nated with another stimulus during which pecks Here both phylogenic and ontogenic selection weren’t reinforced. At fi rst, this other stimulus for discriminative capacities may be intimately was a dark key lasting fi ve seconds. Its duration intertwined. was gradually lengthened and it was changed Just as the response properties defi ning an from dark to dim and then to brighter and operant can be changed gradually by shaping, brighter green, until to a human eye its bright- the stimulus properties defi ning discriminated ness matched that of the red key. By the end operants can be changed by analogous proce- of these conditions, three minutes of red and dures called fading . Training doesn’t have to start reinforcement alternated with three minutes with stimuli that are diffi cult to discriminate. of green and extinction. Each pigeon treated Instead, it can start with stimuli that are easy to this way pecked the extinction key fewer than discriminate and then move gradually to more ten times in the entire course of training; peck- diffi cult ones. For example, it’s usually harder ing occurred almost without exception on red to teach a pigeon to discriminate between ver- rather than green. On the other hand, pigeons tical and horizontal lines than between red and introduced to this procedure later after initial green. Once a discrimination between red and training or introduced abruptly to green at full green exists, however, one between vertical and duration and intensity pecked the green extinc- horizontal can be successively approximated by tion key hundreds and even thousands of times superimposing vertical on red and horizontal on during equivalent training periods. green and then gradually fading out the colors The gradual fading in of green was effective (Terrace, 1963b). In this section I arranged some in part because turning the red key dark early crude fading by dropping out variable-interval in training stopped the pigeon’s pecking for a in favor of its abbreviation, VI, after both had few seconds (any abrupt stimulus change might sometimes appeared together. have had this effect). Pecking wasn’t likely to Creating stimulus control through fading is start again before the brief fi ve-second extinction often effective (Sidman & Stoddard, 1967), but stimulus ended. Thus, a difference in respond- as with shaping no rules exist for how rapidly ing to the two stimuli occurred at the outset. stimuli should be faded in or out in different The gradual changes in the duration and inten- situations. For example, if we superimpose ver- sity of the extinction stimulus built on that dif- tical on red and horizontal on green and then ference. After such a history, we might convert partially fade out the colors, we might discover the extinction stimulus to reinforcement and after removing the colors completely that the the pigeon, never pecking that stimulus, might pigeon has learned to discriminate between ver- never discover we had done so. The sense in tical and horizontal. On the other hand, fading which such a performance is errorless isn’t obvi- might be unsuccessful; if a pigeon attends only ous. We must be cautious about the language of to colors even when they became very faint, dis- errors, which implies judgments about the value criminated responding might disappear every of responding and may be inappropriate to a time we dim them below some minimum level. behavioral account. But fading isn’t solely of the- Shaping requires some behavior available to oretical interest; it has potential applications to be shaped, and fading requires some discrimi- education that may give it practical signifi cance. 142 • Part V. Learning Without Words: Contingencies THE VOCABULARY OF As illustrated by these applications, differen- DIFFERENTIAL tial reinforcement can be arranged for complex properties of stimuli that aren’t easily quantifi ed. REINFORCEMENT For example, children learning to read must be able to name the letters of the alphabet. But the Both differentiation and discrimination involve properties important for distinguishing among differential reinforcement. The difference is some letters are different from those important whether differential reinforcement is imposed in distinguishing among others (e.g., straight on properties of responding or on properties line versus curve is important in distinguish- of the stimuli during which responding occurs. ing U from V but not V from N), and different One implication of the difference is procedural: distinctions are important for lowercase than for in studies of differentiation the experimenter uppercase letters (e.g., no pair of uppercase let- must wait for the organism’s responses, whereas ters has the up-down or left-right reversals that in studies of discrimination the experimenter must be mastered to read b, p, d and q). The way controls the order and duration of stimulus pre- in which a child learns to distinguish among let- sentations. Table 15–1 summarizes the vocabu- ters of the alphabet depends on properties such lary of differentiation and discrimination. as curvature (e.g., D versus O) and closure (e.g., Differential reinforcement can be based on F versus P). Properties essential to discriminat- simple dimensions of stimuli such as intensity or ing among letters are sometimes called critical location. The experimental question is whether features (Gibson, 1965). responding conforms to the differential conse- It isn’t enough, however, to enumerate criti- quences, in that more responding occurs during cal features. For some letters, the uppercase and the stimuli correlated with reinforcement than lowercase forms differ more from each other during those correlated with nonreinforcement. than from other letters (e.g., e, E and F, or h, n and The implications are profound. What we’ve N). Given the multitude of forms, what defi nes learned from training simple discriminations in the stimulus class that occasions our saying A or pigeons or rats has been used to develop meth- B or C? The problem becomes even more com- ods for training primates to discriminate among plicated when different contexts are considered. different drugs that have been administered to For example, O could be either a letter or zero, them (Schuster & Balster, 1977), for studying and I could be either a letter or a roman numeral. the sensory capacities of nonverbal organisms The concepts of X and Y and Z are defi ned by the (Blough, 1958a), and for teaching young parents stimulus classes to which we respond with corre- to identify their children’s illnesses (Delgado & sponding letter names, but such classes are based Lutzger, 1988) to mention just a few of a vast on behavior, not on common physical properties. number of successful applications.

Table 15–1 The Vocabulary of Diff erential Reinforcement Concentration of Diff erential Eff ects of Spread of Eff ects Diff erential Reinforcement Reinforcement of Reinforcement Reinforcement by (Procedure) (Outcome) (Outcome) Approximations Class With Respect to Diff erentiation Induction Shaping Operant Response Properties With Respect to Discrimination Generalization Fading Discriminated Stimulus Properties Operant Chapter 15. Discriminated Operants: Stimulus Control • 143

Discriminated operants are classes defi ned by those classes they’d never seen before; some- the stimuli that occasion responding. We often times the new pictures were more accurately identify such classes in our everyday vocabu- discriminated than those used in training. lary, as when we speak of stopping at a red traf- Natural concepts are examples of probabilistic fi c light or answering a telephone. The red light stimulus classes, classes in which each member can vary in brightness and size and the ring can contains some subset of features but no feature vary in loudness and timbre, but our behavior is is common to all and the number of features in reasonably independent of variations along such the subset may vary from one class member to dimensions, so we speak of these classes rather another. Such classes, sometimes called fuzzy than specifi c instances. sets, don’t have well-defi ned boundaries, though class members may have family resemblances (Rosch, 1973). Defi ning stimulus classes is a gen- NATURAL CONCEPTS eral problem. Appeals to physical measurement don’t resolve it, because the reading of meters or It’s often diffi cult to defi ne discriminative stim- other instruments is also discriminated behavior. uli by physical dimensions. For example, the Behavior classes depend on the common con- properties defi ning the letter A vary accord- tingencies that created them and not on their ing to whether it’s uppercase or lowercase and physical properties. Even the behavior of the sci- whether it appears as type or script. The capacity entist depends on discriminations learned in the to discriminate among such stimuli exists in non- laboratory. humans as well as humans (Morgan et al., 1976). The behavior of moving from one place to But the diffi culty of defi ning stimuli by measur- another has a privileged place in the study of able physical properties isn’t limited to arbitrary learning (Olton, 1979). Going to food isn’t the classes created by humans, such as letters and same as producing food while remaining in numbers. It’s shared with natural objects and place. One important difference is that environ- events. On what basis do we generalize between ments change more drastically when we move huskies and beagles by calling both dogs while to a new location than when we introduce a we discriminate between huskies and wolves new stimulus into our present one. We humans even though they look more alike than huskies have substituted other responses for ordinary and beagles? locomotion: to go places we step on gas ped- With pictures presented on a screen next als, turn steering wheels, press elevator buttons to a pigeon’s key and pecks reinforced during and stand on escalators or moving walkways. some but not others, pigeons have been taught Furthermore, movement produces continuous discriminations involving pictures containing changes in the environment, whereas sequences natural forms, such as presence versus absence of other responses may produce no change until of leaves, or maple-tree leaves versus leaves the sequence is done. Beyond these differences, from other trees (Herrnstein, Loveland, & Cable, does it matter whether an organism moves to a 1976). Such discriminations have been called new place or produces new stimuli where it is? natural concepts, where concepts can be loosely Such questions were implicit in a controversy defi ned as discrimination between classes and over place learning versus response learning generalization within classes (F. S. Keller & (Restle, 1957). Suppose the right arm of a T-maze Schoenfeld, 1950). Pigeons learned discrimina- points toward the brighter windowed east wall tions between pictures with and without trees, of a laboratory. The rat might learn right turns or with and without water, or with and with- or it might learn to run toward the window. out a person. After training with one set of pic- We could test these alternatives by turning the tures, such as those with and without people, T-maze around so the right arm points west and the pigeons discriminated among pictures from the rat now approaches the choice-point from 144 • Part V. Learning Without Words: Contingencies the north. If the rat still turns right but now away their environments. It’s therefore often framed from the window, it shows response learning. If it in terms of stimulus structure. For example, if turns left but still toward the window, it shows an organism discriminates among stimuli based place learning: it moves toward the same place, on some critical feature, an animal cognitivist even though turning in a different direction. might say the organism represents the stimuli Does the rat learn right versus left turns or east to itself in terms of that feature. For example, in versus west turns? search for prey, a predator may be said to com- The rat’s performance depends on the stimuli pare potential targets with a search image. This available both inside and outside the maze. The is a metaphor. Search here involves differential typical rat can discriminate the general direction attention, and has much in common with what of lights and other gross features of a room. But we do when we search for a jigsaw puzzle piece if those stimuli are eliminated by placing a roof based on image rather than shape. over the maze, the direction the maze faces in the Examples of animal cognition include judg- room becomes irrelevant and the rat can’t show ments of visual symmetry (Delius & Nowak, anything but response learning. Whether the rat 1982), discriminations of numerousness (Davis learns place or response, therefore, depends on & Pérusse, 1988), and visual search (Blough, how the experimenter sets up the maze. By judi- 1989), to mention just a few. As an example, con- ciously choosing conditions, an experimenter sider a pigeon watching a clock hand projected can make things turn out either way. on the center key of three pigeon keys (Neiworth In natural environments, food at a given & Rilling, 1987). The hand starts at vertical and location isn’t necessarily replenished as in the rotates at a constant rate from vertical through goal box of a laboratory maze. In foraging, an 90o; then it disappears. A little later it reappears animal is more likely to move to a new location farther along, at 135o or 180o. The timing of its than to return to one where it has already con- reappearance is either consistent or inconsistent sumed the available food. Again, the features to with it having moved at a constant rate of rota- which the organism attends can vary with con- tion while invisible. After presentations consis- straints imposed by the experimenter (Collier tent with a constant rotation rate, left-key pecks & Rovee-Collier, 1981; Lea, 1979). For example, are reinforced; after inconsistent ones, right-key when a rat is given daily sessions with food at pecks are reinforced. Pigeons learned to dis- the ends of each of several alleys and the food criminate trials consistent with a constant rota- isn’t replenished during the session, the rat tion rate from those in which the constant rate learns not to repeat visits to alleys where it has was violated even though the rotating stimulus already eaten (Olton & Samuelson, 1976). The was absent some of the time; this discrimina- lessons from these once-controversial issues tion also transferred to new locations of reap- in spatial learning are: (i) they’ve usually been pearance of the clock hand, so it couldn’t have resolved by looking at procedural details instead been based on trial durations or specifi c clock- of theoretical assumptions; and (ii) incompat- hand locations. It therefore demonstrates visual ible alternatives from which we seem forced to tracking in the absence of the visual stimulus; choose usually turn out to be different catego- such tracking is sometimes called imagery. Col- ries of behavior each of which in its own right loquially, we might say the pigeon knew where belongs in our taxonomy of behavior. the stimulus was even while it was invisible. The fi eld of animal cognition is concerned This is a useful skill, as when an edible insect with what organisms know as they interact with passes behind an obstruction and the bird waits for it to emerge from the other side. Chapter 16 Conditional Discrimination and Stimulus Classes

…because species often occupy multiple habitats, a mutation may be favored in some locales and selected against or ignored in others. S. B. Carroll, 2006

During many years of team-teaching the learn- reverse itself and start pecking green instead of ing course with my colleague Eliot Shimoff we red. When Shimoff suggested doing the reversal used pigeon and rat demonstrations in the class- in the available time would be impossible and room . One started out with a pigeon pecking the class had been ruined, I said I could com- red but not green. I arranged the contingencies plete the reversal in less than ten minutes. He and the key colors with hand switches, alternat- took me up on the offer and started a clock to see ing between red and green every ten or fi fteen how long I’d take. seconds or so. During red I reinforced about one It was a setup, of course, though most stu- in every four or fi ve pecks on average and dur- dents claimed after class they hadn’t been fooled ing green I didn’t reinforce any. The deal was by our phony fi ght. The fi rst thing I did was to that Shimoff talked while I concentrated on the stop reinforcing pecks on red, but the bird didn’t apparatus. slow down much at all while the key remained But we had a problem. As I worked with the that color. Next I switched to green, and the pigeon Shimoff talked about stimulus control. pigeon, whose performance had been very well His example was a pigeon conforming to traffi c prepared by the teaching assistant, became prac- signals, going on red and stopping on green. At tically immobile, facing the key but at some one point he said he could name the key color distance from it. I operated the feeder and the without even looking at the demonstration pigeon quickly came over to eat, so I immedi- chamber, which was located somewhat behind ately began shaping key pecks during green. It him and to the side on the stage area of our took just two or three reinforcers to get the bird classroom. He said all he had to do was listen, to move its beak toward the key, and with only so while I had the pigeon pecking on a red key three or four more I had it pecking on green. But he said the key must be green and when I had it when I switched the key back to red the bird still stopping during green he said the key must be pecked. It was now pecking both green and red, red. The class seemed amused. so only half the job was done. The bird now went We soon got into an argument about what on green, but I had yet to get it to stop on red. we had agreed on for this demonstration and I began reinforcing one of every three or four what instructions the teaching assistant had pecks on green, and after delivering a few rein- been given. The teaching assistant stood nearby forcers I switched back over to red, during which but wouldn’t take sides. The only thing I could I reinforced no further pecks. That meant green do was offer to fi x things by teaching the bird to was itself becoming more potent as a reinforcer

145 146 • Part V. Learning Without Words: Contingencies while red was becoming less so. In other words, Though we then demonstrated a reversal, it only pecks on green now produced reinforc- lacked the classroom drama we had planned for. ers, so by virtue of that contingency green was What had happened? We fi nally realized our becoming a conditional reinforcer . And if it was a pigeon had been through these procedures so reinforcer, I should be able to use it. The pigeon often it had learned a conditional discrimination, was still pecking red a lot, but the next step was where the conditional stimulus was whether it to change the key from red to green during any was in the lab or the classroom. In the lab we slight pause in pecking. I’d use the onset of green always reinforced pecks on red but not green; in to reinforce pauses during red. the classroom, after the fi rst couple of minutes Within the fi rst minute or so I caught some we always reinforced pecks on green but not red. good pauses and let the bird produce two or The bird had fi nally reached the point where it three reinforcers during green before going back didn’t wait for us before starting its reversal. We to red. As the pauses during red became longer I solved the problem by doing reversals in the lab could now pick up head movements away from as well as in the classroom . the red key and make the onset of green contin- Like contingencies, discriminations may be gent on them. Soon the pigeon was backing away effective under some conditions but not others. from the key as soon as it turned red and moving For example, your response to a green traffi c toward it and pecking as soon as it turned green. light will depend on whether you’re facing it By alternating quickly between green and red I or looking at it from the cross traffi c. Such dis- could even get the pigeon to bob its head in front criminations, in which the role of one stimulus of the key for a few seconds: it moved toward depends on others that provide its context, are pecking the green key, but I turned the key red called conditional discriminations. We examined before it got there; it backed off and I could turn attention in an experiment using triangles or the key back to green again. Less than ten min- circles on red or green as the stimuli. We could utes had elapsed on Shimoff’s clock and he now have added a lamp above the key and reinforced had a pigeon that went on green and stopped on pecks in the presence of triangles when it was lit red. We debriefed the class and were especially and pecks in the presence of red when it wasn’t. pleased if our contrived argument had been con- Under these circumstances, the pigeon learns to vincing enough that some students had really peck triangles but not circles without regard to been worrying about us. color when the lamp is on, and to peck red but But one semester things went strangely. The not green without regard to form when it’s off. preparation for our demonstrations began in our In other words, whether the pigeon attends to lab. Most of our research involved steady-state form or color is conditional on whether the lamp procedures, so our pigeons were around for a is lit. long time (the age of our oldest was twenty when We’ve already considered more socially it died of natural causes). One bird had been signifi cant conditional discriminations, such as devoted to the reversal demonstration for sev- those worth learning on city streets or in aca- eral semesters. Ahead of time each semester one demic institutions. Here we’ll examine other of us or a teaching assistant shaped up the per- ways in which they can function. We’ll consider formance of pecking red but not green. Then we procedures in which stimulus-control contin- did the reversal in the classroom. After return- gencies depend on the contexts within which ing to the lab, we again shaped up pecking red they’re arranged. These procedures sometimes but not green so the bird would be ready the next generate higher-order classes of behavior, in time around. This time the bird seemed just fi ne the sense that the classes are defi ned not by spe- when we checked it out a day or so before our cifi c stimuli or responses but rather by relations demonstration, but when we got it to the class- among them. Matching-to-sample, oddity and room it began to pause on red and peck on green arbitrary matching provide a context in which well before Shimoff and I got to our argument. to explore conditions under which one stimulus Chapter 16. Conditional Discrimination and Stimulus Classes • 147 can become the functional equivalent of another, side stimuli produced by the center-key peck as when, in varied settings, uppercase A and are called comparisons ; one matches the sample lowercase a function as the same letter even and the other doesn’t. A peck on the matching though they look very different. key then produces a reinforcer followed by a Matching and oddity are relational proper- new intertrial interval, whereas one on the non- ties of stimuli. We often treat stimuli as if they’re matching key is followed directly by the inter- limited to concrete objects or environmental trial interval without a reinforcer. events. But as we learn to respond consistently Both the sample stimulus and the left-right to objects or events, we also learn to discrimi- position of the matching comparison change nate among abstract or relational features, that over trials. A common matching-to-sample fea- are independent of specifi c objects or events. ture is a correction procedure, which repeats the When it identifi es abstract or relational features, same samples and comparisons on the next trial the term stimulus can function in this more gen- if a trial ends with a peck on a nonmatching key. eral way. For example, we might say a chair was This procedure keeps responding restricted to placed to the right of a table. The chair and table only one key or color from developing, but it also are concrete objects but being-to-the-right-of guarantees errors are sooner or later followed by isn’t, and yet we discriminate between this rela- reinforcers produced by correct responses. tion and being-to-the-left-of. Suppose a pigeon pecks only the comparison on the left. If the matching comparison alter- nates irregularly between left and right, these RELATIONS AS STIMULUS pecks will be reinforced on half the trials, and DIMENSIONS reinforcement on half the trials can maintain left-key pecking indefi nitely. With a correction Many conditional discriminations involve arbi- procedure, however, the pigeon must switch trary relations between a conditional discrimina- to the right key sooner or later, because a trial tive stimulus and the discriminations for which with a match on the right will repeat itself until it sets the occasion. Some cases in which those a right-key peck occurs and is reinforced. Simi- relations aren’t arbitrary are of special interest. larly, if the pigeon always pecks red compari- For example, whether one stimulus in a stimu- sons, these pecks will be reinforced on half the lus set matches one or more of the others or is trials unless a correction procedure forces the an odd one depends on, or is conditional on, the pigeon to switch occasionally to green. context of other stimuli within which it’s pre- Another refi nement is to enhance the sented. For example, if stimuli A and B are blue pigeon’s attention to the sample by modify- and C is yellow, then B is a matching stimulus ing the contingencies arranged for sample-key in relation to A whereas C is an odd stimulus in pecks. For example, if to produce the compari- relation to A plus B. sons one sample requires a high rate of pecking and the other requires a low rate, the pigeon’s Matching-to-Sample and Oddity rates of pecking the two samples can tell us how well it’s discriminating between them. Such pro- Figure 16–1 illustrates matching-to-sample as cedures sometimes produce faster acquisition of it might be arranged for green (G) and red (R) matching but sometimes also reduce attention to stimuli in a three-key pigeon chamber (Ferster, the comparisons (Urcuioli, 1985). 1960; Skinner, 1950). During an intertrial inter- In any case, suppose now a pigeon is match- val, all keys are dark. A trial begins when the ing accurately given red (R) and green (G). center key is lit, presenting the sample. A peck How should we describe its performance? Has on the center key, sometimes called an observing it learned to peck left given the confi gurations response, turns on new stimuli and also makes it RRG and GGR and right given GRR and RGG? likely the pigeon has looked at the sample. The Or has it learned matching in general, the iden- 148 • Part V. Learning Without Words: Contingencies tity relation? If we added blue or yellow and alized matching or generalized oddity; in others the pigeon matched with the new sample col- they do well (Katz & Wright, 2006). One condition ors we’d feel better about speaking of general- making a big difference is the number of stimuli ized matching. But in fact, matching in pigeons used in the procedure. The matching-to-sample doesn’t usually transfer easily to new colors, procedure of Figure 16–1 uses just two stimuli, though the likelihood of transfer depends on red and green. Other procedures use large pop- the details of training. Even if we saw matching ulations of slides, such as travel scenes, so the with new colors, what if we failed to get match- stimuli used in one match are never repeated in ing with geometric fi gures? We might just say others. A strong case can be made that the two- the pigeon has learned color but not form match- stimulus procedure is far more diffi cult than the ing, noting that the human concept of matching many-stimulus procedure, in that the contingen- seems not so limited by specifi c stimulus dimen- cies arranged for common stimuli across differ- sions. Matching-to-sample has sometimes been ent trials in the two-stimulus procedure interfere called identity matching, but we reserve that term with each other whereas the large populations for cases in which matching generalizes to novel in procedures with many stimuli minimize such samples and comparisons, such as matching of forms after training with colors. With a limited stimulus set, what looks like matching based on identity too often turns out to be matching based on specifi c stimulus confi gurations. Matching-to-sample has many variations. If reinforcement is arranged for pecks on the non- matching rather than the matching comparison, the task becomes an instance of oddity respond- ing, because this key is necessarily the odd one of the three keys. This version of oddity requires a minimum of two pecks per trial, one on the sample and the other on the nonmatching com- parison. Another version of oddity uses no sam- ple; if all three keys are lit at the same time, the odd key can be any one of them and each trial will require just one peck. The relations among stimuli can also involve arbitrary matching. For example, we could train the pigeon to peck green given square as a sam- ple and red given circle as a sample. Would the pigeon then peck square given a green sample and circle given a red sample? Probably not. We expect such reversibility when we deal with words and objects, as when a child who’s Figure 16–1 Diagram of a matching-to-sample trial learned to point to a picture of a car on seeing in a three-key pigeon chamber. An intertrial interval of the word car can also point to the word on see- t s is followed by a sample (green or red) on the cen- ing the picture. This reversibility, a property of ter key. A center-key peck turns on the side-key com- symbolic behavior, isn’t to be taken for granted. parisons. One matches the sample; the other doesn’t . A peck on the matching comparison produces food, How organisms cope with matching or odd- after which the next intertrial interval starts; a peck on ity or related conditional discriminations varies the nonmatching one starts the next intertrial interval considerably depending on procedural specifi cs. without food. The sample and the left-right locations In some experiments pigeons rarely show gener- of the comparisons vary from trial to trial. Chapter 16. Conditional Discrimination and Stimulus Classes • 149 interference (Sidman, 1987; Wright, 2012; Wright Figure 16–2 shows how standard and arbi- & Roediger, 2003). trary matching can be used to demonstrate refl exivity, symmetry and transitivity, where the Symbolic Behavior: Equivalence Classes fi rst equivalence class consists of red, Δ and ◍ (light), and the second consists of green, ○ and If a pigeon’s pecks are reinforced in the presence ● (dark). Each procedure includes two sam- of green but not red, we could say the pigeons ples, shown as Sample A in the middle of the pecks in the presence of green, but we’d hardly three-key array on the left and as Sample B in entertain saying the pigeon greens in the pres- that on the right. For convenience, the matching ence of pecks. Such a reversal makes no behav- comparison is always shown in the left position ioral sense. But this isn’t so in matching. Both the of each three-key array, but in practice the side sample stimulus and the comparison response positions of the comparisons vary from trial to are defi ned by stimuli presented on the keys. trial. As in Figure 16–1, a sample peck produces We could therefore ask about a red response comparisons, and a peck on the matching com- to a red stimulus, or the reversibility of a verti- parison produces food whereas one on the non- cal response to a diagonal stimulus, or whether matching comparison doesn’t. round responses to dim stimuli can be created The top two rows (refl exivity) illustrate stan- by training round responses to large stimuli and dard matching procedures with color and form. then large responses to dim stimuli. The next two (symmetry) illustrate symbolic These cases illustrate relations called re fl exiv- matching with color samples and form com- ity, symmetry and transitivity. Refl exive proper- parisons and then a reversal with form samples ties are those holding between a term and itself and color comparisons. The bottom rows (tran- (as in A = A); symmetrical properties are those sitivity) show how the common stimuli in two in which the order of terms is reversible (as in symbolic matching procedures (color to form if A = B then B = A); and transitive properties and form to intensity) can be combined in a test are those in which the common terms in two of transitivity (color to intensity). An alterna- ordered pairs determine a third ordered pair (as tive test, illustrated in the last row, combines in if A = B and B = C then A = C). Equivalence reversal and transitivity (intensity to color); relations are those having all three properties. it’s called an equivalence test, and the relations The terms entering into them are said to be among the stimuli of the equivalence test are members of an equivalence class (Sidman, 1994). sometimes called equivalence relations. The rela- An equivalence class is rarely considered in iso- tions of the reversal and equivalence tests were lation. For example, with three-member classes never explicitly taught. If appropriate matching we might identifi ed one as A1, B1 and C1 and occurs in these tests, the new relations are called another as A2, B2 and C2, or one as , β and γ and emergent, in the sense that they’ve emerged another as X, Y and Z. without explicit training; they’re examples of For other relations, only some of the three novel behavior produced by the matching con- properties hold. For example, a magnitude rela- tingencies. tion such as greater than is neither refl exive nor When we say a pigeon that pecks in the symmetrical, but it is transitive, because if G is presence of green can’t green in the presence of greater than H and H is greater than I, then G is pecks, we’re saying the relations between ante- greater than I. As another example, the relation cedent stimuli and the responses they occasion opposite is symmetrical: if D is the opposite of E, in a three-term contingency aren’t symmetrical. then E is the opposite of D. But it’s not refl exive: This means we can’t reduce equivalence classes D isn’t the opposite of itself. Nor is it transitive: to classes of discriminative stimuli. Equivalence if D is the opposite of E and E is the opposite of relations are symmetrical, but the relations F, then D isn’t the opposite of F; instead, D is the among the terms of a three-term contingency same as F. aren’t. Chapter 17 Sources of Novel Behavior

We turn now to a different type of multiple control, in which functional relations, established separately, combine possibly for the fi rst time upon a given occasion — B. F. Skinner

Despite its initial support, Darwin’s account of matical sentences is astronomical. This feature evolution as a product of natural selection soon of language is called productivity, but language met diffi culties because of what was known shares this property with nonverbal behavior. about genetics in the nineteenth and early We can deal with novelty by identifying fea- twentieth centuries. It couldn’t work without tures the new sentence shares with earlier ones; variations from which to select and it wasn’t novel productions involve new combinations clear where those variations could come from. of established syntactic and semantic classes What was the source of the novel variations on (Esper, 1973). New combinations have also been which the changes in species over evolutionary important in evolution through natural selection time depended? The answer came when data (West-Eberhart, 2003). on mutations were added to what was known Because children couldn’t possibly make con- about genetics. Genetics provided the transmis- tact with all possible sentences in the language sion from one generation to the next and muta- they were mastering, Chomsky appealed to the tions provided the variations. Even now the poverty of the stimulus (Chomsky, 1959; Crain, origins of novelty rather than natural selection 1991). Based their limited samples, how could itself have been favorite targets of Darwin’s crit- children learn the difference between grammati- ics (Kirschner & Gerhart, 2005, p. 265). cal and ungrammatical sentences? Given some The ontogenic selection of behavior by its ungrammatical forms were so unusual chil- consequences faced similar challenges, espe- dren were hardly ever likely to encounter them, cially with regard to the role of reinforcement how could they know they shouldn’t use them? in the acquisition of language by children. After Chomsky concluded that children had to start out the publication of Verbal Behavior by Skinner with some built-in language acquisition device or (1957), the linguist Noam Chomsky (1959) wrote they’d be unable to learn their native languages. a critique of the book in which he argued that But a pigeon doesn’t need to see all other Skinner couldn’t account for the child’s capac- wavelengths of a visual stimulus before learn- ity to produce grammatical sentences the child ing to peck green but not red (Catania, 2003b). could never have encountered before. Neither does a child need to hear ungrammati- When we write a sentence, it’s likely to differ cal sentences before learning past and present from any others we’ve written. We couldn’t have tenses or active and passive voices. The selec- learned them individually because the number tion of operant classes by their consequences of ways in which words can combine into gram- doesn’t depend on extensive sampling of nega-

156 162 • Part V. Learning Without Words: Contingencies

Figure 17–1 Diagram of a serial-coordination procedure. In a three-key pigeon chamber, one or two circles or triangles were presented on lit keys (unfi lled circles). Training began with the bottom keys in B, in which left pecks were reinforced during singular on each bottom key and right pecks were reinforced during plural. Then, in top-key trials, as in A, slow pecking was reinforced during circles and fast pecking during triangles. Finally A and B were put together: when top-key rates met their high-rate or low-rate criteria during A, they produced B. Did top-key rate diff erences emerge on the bottom keys, where they’d never been diff erentially reinforced. (Adapted from Catania & Cerutti, 1986, Figure 8) and fast responding given triangles. During this differences developed gradually and occurred training, appropriate rates produced food rein- despite continued reinforcement of roughly forcers. Throughout the training conditions, no equal rates on the bottom keys across all stimuli rate contingencies were ever arranged for bot- during the sessions immediately after A and B tom-key pecking. Then the two parts, A and B, were combined. Just as voicing persists in the were arranged in series, so that successful com- plural –s in human verbal behavior, differen- pletion of A produced B, as diagrammed in the tial rates persist in the plural responding in the fi gure. pigeon analog. These are of course small coordi- Within about two weeks after A and B were nations, but they show how response properties combined, rate differences corresponding to can constrain sequential behavior when differ- those on the top key emerged on the bottom ent classes are combined. keys. The differences were smaller than those on the top key but occurred even though differ- Coordinations in Parallel ential reinforcement of bottom-key responding depended only on response location and not on Never before having met a green dog, we’d response rate. Reinforcement was maintained probably still say something like Look at that during all sessions in which the A and B com- green dog if we met one today. This example of ponents were combined, but the bottom-key rate adduction comes about through a new combi- Chapter 17. Sources of Novel Behavior • 163 nation of the well-learned words, green and dog, in response to a novel combination of color and species. Such novel combinations are common- place in human verbal behavior. One account of productivity is that novel behavior emerges from the novel intersection of two or more properties of behavior, each deter- mined by a different stimulus property (cf. Esper, 1973). This is an instance of multiple causation. We often speak of stimuli setting the occasion for responses. Strictly, however, discrimina- tive stimuli don’t occasion responses; instead, stimulus properties occasion response properties. For example, the introductory example, green dog, involved a new combination of color word and species word, with each part of the verbal response depending on a different property of the nonverbal stimulus (cf. Skinner, 1957). Fig- ure 17–2 illustrates a pigeon panel from experi- ments designed to provide a nonhuman analog of such relations (Catania & Cerutti, 1986; Cata- Figure 17–2 Four-key pigeon panel for demon- nia, Ono, & de Souza, 2000). strating control of multiple response dimensions The objective was to separately establish con- by multiple stimulus dimensions. Identical stimuli trol of response location by form (left-right) and including number, form and color appeared on all by number (top-bottom) and of response rate by four keys within 3-s trials. Top-row pecking was rein- color (green-red), and then to combine the sepa- forced given singular and bottom-row pecking given rate discriminations. Stimuli were presented on plural (number); left-column pecking given circle and all four keys in 3-s trials, after which a peck on right-column pecking given plus (form); fast pecking any key ended the trial with food if contingen- given green and slow pecking given red (color). For cies had been satisfi ed or without if not. Stimu- example, with plural-plus-green on all four keys, the fi rst bottom right peck was reinforced at the end of lus number, singular versus plural, determined the trial if at least eight responses occurred there dur- whether a peck on the top or bottom row was ing the trial; with singular-circle-red, the fi rst top left reinforced. Stimulus form, circle versus plus, peck was reinforced if three or fewer pecks occurred determined whether a peck on the left or right there (Adapted from Catania et al., 2000, Figure 1). column was reinforced. Stimulus color, green versus red, determined whether the peck was upper left key during singular circles then peck reinforced only after a high rate of pecking dur- rapidly on the upper left key given singular ing the trial or only after a low rate. Differential circles on green, a stimulus combination it had reinforcement of rates of pecking during green never seen before? In fact that didn’t happen. and red gradually established two nonoverlap- Novel stimuli often disrupt behavior and did ping classes of responding, high rates during so in this case. The newly combined dimensions green and low rates during red. By the end of of the stimulus combinations rapidly acquired shaping, a peck at the end of a trial including control once relevant contingencies were added green was reinforced only if at least eight pecks for them, but that didn’t qualify as adduction during the trial, and at the end of one including because it might not have happened without the red only if fewer than three pecks were emitted. added contingencies. Would a pigeon trained to peck rapidly dur- In the next experiment, the adduction test ing green and separately trained to peck the didn’t include extinction. Instead, for pigeons 164 • Part V. Learning Without Words: Contingencies who were experimentally naive prior to train- ior during its fi rst few encounters with a novel ing, incomplete contingencies were arranged stimulus is unreinforced one organism stops for some stimulus combinations, so differ- engaging with the stimulus, but another persists ential reinforcement operated only for some in that behavior. The latter may be at a disadvan- response dimensions: all the former contingen- tage relative to the former, because novel stimu- cies were in place, except no rate contingencies lus combinations in natural environments aren’t were arranged during trials with singular-cir- necessarily correlated with corresponding com- cle-green or plural-plus-red. I’ll call these the binations of contingencies. A food with some adduction stimuli. The question was whether combination of properties such as color, size and an appropriate novel combination of response shape may have become well-established as a properties would emerge for these stimuli in reinforcer based on its nutritive properties, low the absence of differential reinforcement. toxicity and so on. A new food may share some From the start, location contingencies worked of these properties and is likely to be sampled for all keys and peck locations quickly came to when encountered, but if it’s not very nutritive conform to them. Rate differentiation proceeded and has delayed toxic effects besides, an organ- more slowly, with response rate changes for the ism that persists in consuming it even though its two adduction stimuli lagging behind those for consummatory behavior is no longer effectively which rate contingencies operated. Over about a reinforced may soon face both ontogenic and month, however, rates for the adduction stimuli phylogenic extinction. caught up with the rest, though no differential It’s all too easy to make up evolutionary reinforcement operated for those two stimuli just-so stories, but perhaps phylogenic contin- and though intermediate rates of pecking had gencies like these have selected organisms to be often been reinforced during the adduction more like the former fi nicky sort than the latter stimuli in the early weeks of training (Catania et obstinate one. If so, many procedures for testing al., 2000). In other words, rates of pecking dur- novel stimulus arrangements might be funda- ing the two stimuli not participating in the rate mentally fl awed even if they include a history contingency gradually came to conform to the of intermittent reinforcement prior to extinction. differential rates maintained during the other If novel stimulus combinations lead organisms stimuli. These patterns of responding were to behave this way given extinction contingen- maintained for more than three months of sub- cies during tests of transfer or emergence, then sequent sessions, though these rates were never novel stimulus combinations will quickly lose differentially reinforced given these stimuli. control and we may often incorrectly conclude If even a pigeon can do this, should we be that adduction doesn’t occur. Perhaps we some- surprised if a child who’s learned to say one deer times fail to observe emergence simply because running, two sheep walking, one sheep running, two we arranged testing procedures operating deer walking, two deer running and one sheep walk- against it. The emergence of novel combinations ing will say one deer walking or two sheep running of response rates and locations in this procedure when appropriate circumstances arise? Here too but not in an extinction test is consistent with we have multiple causation: singular or plural these speculations. determines the number, the creature determines These experiments don’t explain behavior; the noun, and the fast or slow pace determines they’re demonstrations. To some extent they’re the gerund verb. nothing more than thought experiments realized But why didn’t the novel combination work in practice (Catania, 1980a). In fact, when I fi rst during extinction? Testing for stimulus control became involved in discussing them they hardly in extinction, as in generalization gradients, has seemed worth doing because it seemed obvious an extensive history in behavior analysis, but they could be made to work. I needed to do them that kind of test may have problems. Consider only to show those who found the analogies the phylogenic contingencies. After its behav- unconvincing that nonverbal organisms could Chapter 17. Sources of Novel Behavior • 165 indeed produce such behavior. Nevertheless, I been said to exhibit parity, a match between learned some things along the way, such as the two sources of behavior arising from differ- different effects of extinction versus relaxed con- ent sources (Palmer, 1996, 1998), as when bab- tingencies during adduction tests. bling children react to having produced sounds resembling those made by their caregivers. But Joint Control what sort of history could create the discrimina- tion between jointly controlled responding and Now it’s time to ask how adduction works in responding not involving joint control? our own behavior. Joint control occurs when Instances in which some response is occa- common responses to different stimuli mediate sioned by two different events at about the judgments of equivalence or other relations, as same time are sometimes notable. Skinner (1977) when a child matches one arbitrary stimulus remarked on his reaction to hearing his teacher to another after having been taught to give the say a word at the same time he was reading same name to each (Lowenkron, 1997). “Joint it. Our attention is often captured by unusual control is a discrete event, a change in stimulus events, including coincidences: we notice if control that occurs when a response topography someone says something the same time we say evoked by one stimulus (e.g., the sample) and or think it; in the cocktail-party phenomenon , preserved by rehearsal, is emitted under the we start attending to someone else’s conversa- additional (and thus joint) control of a second tion on overhearing our own name in it; in our stimulus (e.g., the comparison)” (Lowenkron, lottery-ticket example of joint control, we react 1998, p. 332). Joint control, by the way, is about to the coincidence between the number on our two stimuli jointly determining the behavior of ticket and the winning number on the list. single individual; it shouldn’t be confused with We’ve learned to look at parents and teach- joint attention, which involves one individual ers and others when they’ve called us by name, attending to a stimulus someone else is pointing so attending on hearing our name has had con- to or otherwise indicating. sequences. It’s reasonable to think of attending As an instance of joint control, Lowenkron to our name when we hear it in the context of (2004) offers the example of someone trying to a new source, as in the cocktail party phenom- fi nd a number in a list, as in checking a set of enon, as a product of that history. This implies winning lottery numbers. Say the number is that pre-attentive behavior can be reinforced 135476, as read from the lottery ticket. If I were (some might prefer to invoke search images that doing this, I’d probably repeat the number to govern our attention; this is no problem if they myself as I ran my fi nger down the list. What work like rehearsing a number to oneself). Aside else must happen before I can shout I won! if I from the source, how different is this from lis- fi nd 135476 in the list? I’m already saying the tening for your name as attendance is taken or number to myself but now I’m also saying it as I searching for it on a list? And if this works for see it. Together, these two sources of saying the hearing your name, how about hearing a word number have come together so they jointly con- at the same time you’re reading it? trol my saying it. But here’s the critical part of That too has a history. If a child is shown a the argument: before I can say I won! I must be letter in class and is asked by the teacher to say able to distinguish this instance, in which there it, the child must attend to both the seen and is a match, from all the other cases in which the the spoken letter together. An enormous num- number I’m saying and the number I’m reading ber of occasions for these sorts of combinations don’t match. of events will arise as the child matures. Maybe In other words, for joint control to work we the more appropriate question is not why joint must discriminate between instances in which control becomes important but instead what we fi nd correspondences and those in which we happens if a child fails to become responsive to don’t. These correspondences have sometimes 166 • Part V. Learning Without Words: Contingencies it? Could this be a component of some develop- child must answer both accurately and rapidly, mental defi cits ? If it’s absent, can it be taught? and this means continuing to practice beyond Joint control involves discriminative respond- the level of high accuracy alone. After all, this ing to a property of our own behavior. We can is what we expect of fl uent language. It matters no more pin down the stimulus dimensions on in other skills too (Polson, Grabavac, & Parsons, which this discrimination is based than we can 1997). Once the child becomes fl uent in arithme- say what defi nes redness or time or beauty. We tic and translating word problems, adduction don’t appeal to physics before deciding whether typically follows (Binder, 1996; K. R. Johnson & some stimulus class enters into a discriminated Layng, 1992). Given the question above, the child operant. Once we’ve seen a few instances, we will quickly reply Four dollars. Fluency training is can usually identify joint control when we see it. a powerful educational tool, and students taught this way come to excel in reasoning and putting Fluency and Teaching facts together and the various other skills mak- ing up critical thinking; when confronted with To some, teaching a child number facts until the pieces of larger problems, adduction comes the child reaches a level of profi ciency of 75% easily (Greer & Ross, 2004; Twyman, 1998). or 80% might be acceptable as a passing per- One way of thinking about how this works formance. If you got the child to 85% or 90% or is to assume the correct answer is given by the maybe even 95%, you’d probably be pleased. product of the probabilities of each part. If the Getting the child to 99% or better would be even child can do the arithmetic part at a 90% level nicer. Certainly those are levels we’d like to see and the word translation part at an 80% level, if these children someday become professionals the chance of getting both right is 0.9 times 0.8, in engineering or medicine or other demanding which equals 0.72. Even with both at 90%, the disciplines. But let’s settle for 95%. Even at this probability works out as 0.81. That suggests the level it might be inappropriate to decide your child starting out at 99% and 99% should end up high-performing child is doing well enough. with an accuracy of about 0.98 on the combined Suppose you start asking questions like How problem. But that doesn’t happen. In cases like much change will you get from ten dollars if you these, children do far worse than the combined buy three toys costing two dollars each?, where the probabilities suggest they should. Why do we arithmetic needed for the answer is well within need overtraining for both speed and accuracy? this child’s competence. If you were asking lots One possibility is that when two reinforced of questions like this and the child hasn’t been classes of behavior come together, each has an taught to translate word questions so they can inhibiting effect on the other. If so, the function be dealt with as arithmetic, you might see the of the overtraining for speed and accuracy may child’s performance drop below passable levels. bring each class back up to the level it would So now you teach the skill to a profi ciency level have reached in the absence of the other. It of 95 percent or better and you try the test again. would also make them stronger, in the sense The child still doesn’t come close to 95 percent. of being more resistant to change than other This is a failure of adduction. The two com- classes that hadn’t been learned to a fl uency ponents, translating the word problem and criterion (Nevin, 1992). This is consistent with doing the arithmetic, must combine if the child is what we know about accuracy in other settings. to answer correctly, but here they mostly don’t. For example, correct responses and errors in Just putting the two pieces together isn’t good a matching or oddity problem will ordinarily enough to produce adduction. The child must differ in strength, and introducing reinforcers be taught to a level of fl uency, the “combination for other responses will typically reduce the of accuracy and speed that characterizes com- errors more than the correct responses (Cata- petent performance” (Binder, 1996, p. 164). The nia & Dobson, 1972). Paradoxically, instead of Chapter 17. Sources of Novel Behavior • 167 competing with or degrading the matching or be learned, the more they’ll learn, so anything oddity performance, introducing some other teachers can do to get them to put in more time concurrently reinforced response class makes will probably be a good thing. For example, I the performance more accurate. thought I might do no favors for my readers by Fluency is about the behavior of the stu- including chapter outlines, so I almost decided dent rather than the behavior of the teacher. It to omit them completely. They are gone from therefore has special relevance for educational the chapter headings, though they do appear in practices. Clearly it can’t be managed by teach- Appendix I. Try creating your own outline fi rst, ers working with large classes; for some well- and then see how it compares with what you defi ned skills, such as mastering basic arithmetic fi nd there. facts, computers may function as better teachers I did leave out lists of key words, because it than humans. Teaching programs built around should be easy enough for readers to generate fl uency have become increasingly sophisticated their own and to check them out against terms and we’re learning more about how it works in the glossary. I’ve been stingy with cross-ref- (Binder, 1996). The foundations on which it’s erences, too, on the assumption that searching based have been around for a long time. out related topics by using the index or other This is worth saying more than once, so resources is a good thing to do. But I’ll make an expect to see it again: for far too long, educa- exception here: I hope instructors asked to do the tional systems have focused on what the teacher things students should be doing for themselves does instead of what the student does. What the will direct them to the section on teaching in the student does is what the student learns. The chapter on applied behavior analysis. more time students spend with the material to Chapter 18 Behavior Synthesis

Novelty in an organism’s physiology, anatomy, or behavior arises mostly by the use of conserved processes in new combinations, at different times, and in different places and amounts, than by the invention of new processes. — Kirschner & Gerhart, 2005

Schedules don’t always operate in isolation. procrastination and preference for free choice. They can alternate, either with correlated stim- Here we’ll enlarge our tool kit and add to our uli ( multiple schedules) or without them (mixed taxonomy. schedules). The consequence of completing one An experimental analysis takes complex schedule can be the onset of another, either with behavior apart. Our taxonomy of behavior pro- correlated stimuli (chained schedules) or with- vides the behavioral units for our analysis. Once out them ( tandem schedules); in such contexts, we’ve fi nished an analysis by teasing the pieces one schedule can be the unit of behavior on apart, we can validate it by seeing if we can put which another operates ( higher-order schedules). them back together again. The reversal of an Schedules can operate at the same time, either analysis is a synthesis, as when, in chemistry, a for different responses (concurrent schedules) or compound is synthesized after an analysis deter- for the same response (conjoint schedules), and mines its elements and its structure. Here we’ll schedules operating concurrently can each pro- use various schedule combinations to synthesize duce other schedules (concurrent-chain sched- complex behavior. The success of our syntheses ules). Some notable schedule combinations are will tell us something about the adequacy of our summarized in Appendix IV, but these aren’t analyses. of interest just for their own sakes. Instead, they bear on general issues, such as discrimination learning (multiple schedules), conditioned or MULTIPLE AND MIXED conditional reinforcement (chained and sec- SCHEDULES: OBSERVING ond-order schedules) and choice (concurrent RESPONSES and concurrent-chain schedules). This probably seems daunting, especially In multiple schedules, two schedules alternate, if you’re not a reinforcement-schedules junkie each correlated with a different stimulus; we like me and some of my colleagues. But we speak of stimulus control when the performance won’t explore these complex schedules for their appropriate to each schedule occurs during the own sakes. A proper exploration would take corresponding stimulus. For example, with an FI far more than one chapter anyway. Instead, schedule operating for a pigeon’s pecks during we’ll use a few schedules as examples to show green and a VI schedule operating during red, how they can shed light on topics of more gen- we’d expect to see FI scallops in green alternat- eral interest. We’ve already seen how we can ing with roughly constant VI response rates in approach human issues like self-control and red. Reinforcement often alternates with extinc-

168 Chapter 18. Behavior Synthesis • 173

Figure 18–3 Reinforced responding showing spatial and temporal analogous to those seen in sensory sys- tems. Successive trials included FI reinforcement (RFT) or extinction (EXT) of a pigeon’s key pecks. In a spatial sequence (left), the top eight lamps in a vertical array of sixteen were correlated with RFT and the bottom eight were correlated with EXT. In a temporal sequence (right), six trials with one top lamp (RFT) alternated with one or three trials with one bottom lamp (EXT). The spatial sequence showed sustained contrast at the boundary be- tween RFT and EXT; the temporal sequence showed transient contrast, with RFT rates highest immediately after EXT and EXT rates lowest immediately after RFT. (Adapted from Catania & Gill, 1964, Figures 2 and 3) tion and the inverse lower activity after stimulus to a value of FR 240. In this situation, with only offset in sensory systems. the food produced by pecking, the pigeon eas- ily maintains an adequate daily ration. In fact, a pigeon can eat its daily diet in as little as four or CHAINED, TANDEM, AND fi ve minutes and each food delivery lasts only SECOND-ORDER SCHEDULES four seconds or so, including the time to get from key to food hopper, so the pigeon may be done Chained schedules have been used extensively for the day after just sixty reinforcers. At FR 240, to study conditioned or conditional reinforcers, rein- the pigeon must peck 14,400 times per day to forcers that acquire their capacity to reinforce earn this much food. It does so with a typical FR through their relation to other stimuli already performance in which postreinforcement pauses effective as reinforcers (Kelleher & Gollub, 1962). are followed by uninterrupted rapid peck- For example, a feeder light becomes a reinforcer ing. Our pigeon will earn enough food daily to only through its relation to food in the feeder, keep itself alive and healthy indefi nitely. Its life and a clicker becomes a reinforcer only through expectancy may be twenty years or more, and its relation to the reinforcers arranged by a pet with this kind of diet a veterinarian might even owner. Because the opportunity to engage in judge the bird to be slightly overweight, unless highly probable responses reinforces less proba- they live at sites where they’re handsomely fed ble responses only if the opportunity is signaled, by tourists, pigeons in the wild tend to main- the conditional reinforcing functions of stimuli tain themselves at 70% or less of free-feeding have something in common with their discrimi- weights, so maintenance at 80% of free-feeding native functions. weight is hardly a starvation diet. But pigeons can’t count very well, so we Extended Chains might wonder whether the pigeon is at a disad- vantage each time it goes through its 240 pecks. Assume we’ve gradually increased the size of an Let’s help it estimate how many pecks are left to FR schedule for a pigeon’s pecks on a white key 174 • Part V. Learning Without Words: Contingencies go by changing the key color after every forty key was always white, pecking at the start of pecks. Each ratio starts with a blue key for the the sequence wasn’t very different from pecking fi rst forty pecks; it turns green for the next forty, at the end, just before the reinforcer. Once the yellow for the next forty, orange for the next stimuli were added, pecking during the early forty, red for the next forty, and purple for the stimuli became less like pecking later. In blue, fi nal forty pecks, the last of which produces the for example, pecks never produced food. At reinforcer. The pecks per reinforcer remain the best they turned the key green, but pecks never same; only the key colors have changed. produced food during green either. In chained With no distinctive stimuli, the pigeon schedules, a stimulus supports less responding worked well enough, earning an adequate daily the further it is from the end of the sequence. ration. Have we helped by providing the colors? Even severe food deprivation may not counter- The stimuli have the surprising effect of slowing act this effect. the pigeon down. The pause at the start of the These differences occur with various sched- ratio, when the key is blue, lengthens. When the ules. Our example involved tandem and chained pigeon fi nally responds, the pecks that came in FR schedules, but Figure 18–4 illustrates similar rapid succession now occur sporadically. When effects with FI components. Comparable effects the key turns green after forty pecks, the pigeon of chained schedules occur with other organ- pauses again before starting the next forty. When isms besides the pigeon, with different kinds of the key was always white, the pigeon didn’t responses and different schedules, and with dif- pause at this point. In green, another forty pecks ferent kinds and orderings of stimuli (Kelleher and the key turns yellow; then another forty and & Gollub, 1962). Because these effects have such it turns orange. This time the pigeon is less likely generality, we may wonder whether they’re rel- to pause. Forty more pecks turn the key red and evant to human behavior. The things that people then forty more purple. The pigeon now quickly do, of course, depend on consequences more completes the last forty pecks and food is deliv- complex than food deliveries. Nevertheless, so ered. But then the key is blue again and another much of what we do involves sequential behav- long pause begins. The reinforcing effects of the ior we might wonder whether we sometimes stimulus changes in the chained schedules were operate under the strain of too many links in our mostly restricted to the late components, close to chains. Did you ever have trouble writing that the reinforcer. fi rst page of a paper as its due-date approached? The added colors didn’t help. This chain will The fi rst page is certainly the beginning of a never become a chunk. The pigeon takes much chain. The ability to formulate and achieve longer to earn each food delivery than it had long-range objectives is supposed to be a unique when the key remained white. In fact, though human characteristic. But if adding even a single the pigeon maintains an adequate diet at 240 link to a chain so devastates a pigeon’s behavior, pecks per food delivery when the key is always perhaps we should be alert for similar effects in white, we’d now be wise to watch carefully so our own behavior. its daily food intake doesn’t decrease so drasti- The constant ordering of the chained stimuli cally it starves to death. And all we did was add matters. The long pauses decrease markedly if some stimuli (Catania, Yohalem, & Silverman, stimulus order changes from one reinforcer to 1980). the next (Gollub, 1977). Analogous but inverse When we added the colors, we broke the effects also hold for schedules of punishment; 240 pecks down into six distinct units of forty relative to tandem schedules, chained schedules pecks each. We call these chained FR schedules: of punishment reduce responding mostly in the the separate stimuli correspond to the links of later components of the chain (Silverman, 1971). the chain. But the chain breaks the 240-peck An implication is that punishment after a deed sequence up into six FR components instead of is done probably has its greatest effect on the holding it together more cohesively. When the behavior that precedes getting caught and only Chapter 18. Behavior Synthesis • 175

Figure 18–4 Sample pigeon cumulative records from a 6-component chained FI 15-s schedule in which the key- color sequence remained constant over reinforcers and from a comparable 6-component tandem schedule with vertical lines as the stimulus. Records reset to baseline at each reinforcer; successive components are demarcated by pips. The chains produced half as many reinforcers as the tandems and included long pauses in early compo- nents and high rates in the last component compared to the relatively continuous tandem responding. minimal effects on the much earlier behavior example, a tone may function as a conditional that led up to the misdeed. reinforcer even if it’s followed by food only one time in twenty. Brief Stimuli In second-order schedules, the completion of one schedule is a behavioral unit reinforced Early experiments on conditional reinforcers according to another schedule, as when the sec- were conducted during extinction, after a history ond-order schedule FR 10 (DRL 5-sec) arranges of consistent pairings with a primary reinforcer a reinforcer for every tenth interresponse time (e.g., making tone a conditional reinforcer by fol- longer than fi ve seconds. This arrangement, lowing it with food and testing later with tone with a brief stimulus at the completion of each alone). Although this procedure countered the fi rst-order schedule, is one of several types of objection that responding might be maintained second-order schedules. Consider a schedule directly by the later primary reinforcer rather arranged for a pigeon’s pecks in which the peck than by the conditional reinforcer itself, it was that completes each 60-second fi xed interval also one in which the effectiveness of conditional produces a brief green light on the key and in reinforcers diminished rapidly once the primary which every tenth such interval is followed also reinforcer was removed. Convincing demon- by food; the notation for such schedules may strations of conditional reinforcers came only include the brief stimulus: FR 10 (FI 60-s: green). when reinforcement schedules were applied to Such schedules typically maintain FI scalloping their analysis: schedules were arranged not only within intervals even though most intervals end for the production of conditional reinforcers by without food. responses but also for the contingent relation In contrast to chained schedules, sec- between conditional and primary reinforcers ond-order schedules with brief stimuli can (Zimmerman, Hanford, & Brown, 1967). For amplify reinforced responding. For example, 176 • Part V. Learning Without Words: Contingencies

when a chimpanzee’s pushbutton presses were Second-order scheduling can also include reinforced with food according to an FR 4000 other kinds of operants, as when correct schedule, postreinforcement pauses ranged responses in matching-to-sample are reinforced from many minutes to hours. But when the light according to various schedules. In such contexts, accompanying food delivery came on briefl y lower levels of accuracy may be correlated with after every 400 responses, responding increased the times at which responding is least likely and typical postreinforcement pauses decreased to be reinforced. For example, errors are more to fi ve minutes or less. The light converted the likely early in FR runs or in the early portions simple FR 4000 schedule to a second-order of FI scallops than in responding occurring later schedule, FR 10 (FR 400: light) and amplifi ed and therefore closer to reinforcer deliveries the amount of behavior maintained by the food (M. C. P. Boren & Gollub, 1972; Thomas, 1979). reinforcers (Findley & Brady, 1965). The cumu- Classes defi ned sequentially may also imply lative records in Figure 18–5 illustrate these higher-order classes. For example, individual effects. The three FR 4000 sequences (unshaded) pecks are functional units, but within FR per- are each preceded by pauses of half an hour or formance the entire ratio may function as a unit. more; the three FR 400 x 10 sequences with brief The FR performance has a property consistent stimuli (shaded) include some early pauses, but with higher-order classes: as long as the higher- once they’re under way they’re completed in order class is reinforced, the subclasses within less than half an hour. it may also be maintained even though they are Variables such as the relation between the no longer reinforced. In other words, the fi rst brief stimuli and primary reinforcers determine peck of the fi xed ratio does not extinguish, even the effectiveness of second-order schedules though by itself it never produces the reinforcer. (Gollub, 1977). Both chained schedules and sec- ond-order schedules with brief stimuli involve conditional reinforcers, but where chained CONCURRENT SCHEDULES: schedules may attenuate responding, second- MATCHING AND MAXIMIZING order schedules may amplify it. Their opposite effects illustrate how critically schedule effects Any reinforced response is likely to occur in a depend on the detailed relations among stimuli, context of other behavior maintained by other responses and consequences (Malone, 1990, pp. consequences. The effect on one response of 294–296; Morse & Kelleher, 1977). reinforcement schedules operating for other

Figure 18–5 Sample chimpanzee cumulative records from FR 4000 during red and a second-order FR 400 x 10 schedule during green. During red the FR 4000 performance was preceded by very long pauses; respond- ing on the second-order schedule mostly began promptly during green. (Adapted from Findley & Brady, 1965, Figure 2). Chapter 18. Behavior Synthesis • 177 responses may therefore be of interest. Concur- incorporated a changeover delay, which prevents rent schedules are schedules simultaneously either response from being reinforced immedi- arranged for two or more responses. Consider ately after a changeover from the other. With an FR 25 schedule for a pigeon’s pecks on one a changeover delay, the pigeon distributes its key and an FR 50 schedule operating concur- responses to concurrent VI VI schedules roughly rently for pecks on a second key. Alone, either in proportion to the distribution of reinforcers schedule maintains responding, but when they they arrange (Herrnstein, 1961); for example, a operate concurrently, responding is likely to be pigeon is likely to peck a VI 30-s key (120 rein- maintained exclusively on the key with the FR forcers per hour) about twice as often as a con- 25 schedule. The outcome isn’t surprising. A current VI 60-s key (60 reinforcers per hour). reinforcer requires only twenty-fi ve pecks on the This phenomenon is suffi ciently general it’s fi rst key but fi fty on the second. been proposed as a general law of behavior, the Now consider concurrent interval schedules, matching law (Davison & McCarthy, 1988; Her- such as VI 30-s reinforcement of pecks on one rnstein, 1970), which states that relative rates of key and VI 60-s reinforcement of pecks on the responding match the relative rates of reinforce- other. In this case, the pigeon produces 120 rein- ment produced by that responding. The law even forcers per hour by pecking only the fi rst key or holds for concurrent ratio schedules, because sixty per hour by pecking only the second. By exclusive responding on one schedule means pecking both, however, it produces the reinforc- all the reinforcers will be delivered according to ers of both schedules, or 180 per hour. In this that schedule. Herrnstein’s account has also been case, responding is likely to be maintained on applied to responding maintained by a single both keys. Variables with small effects in sin- reinforcement schedule, on the assumption that gle-response schedules often have large effects other events besides the reinforcers arranged by in concurrent schedules, which are therefore the experimenter may have reinforcing effects useful for studying effects of reinforcement vari- even though we can’t identify them. To account ables (e.g., reinforcer duration: Catania, 1963a; for issues of scaling and bias, the law has been response force: Chung, 1965). Perhaps the most extended to the generalized matching law (Baum, general feature of behavior revealed by concur- 1974; Davison & McCarthy, 1988), but this is not rent performances is that increasing the rein- the place for an exposition of the mathematical forcement of one response reduces the rate of details of this and other extensions of quantita- others (Catania et al., 1988; Killeen, 1972; Rach- tive analyses. lin & Baum, 1972). This interaction depends on The matching law and the generalized law the reinforcers and not on competition between of which it’s a special case summarize perfor- responses for available time (Catania, 1963b). mances in a variety of schedules, but their status The response rate generated by the total rate of as a convenient description or as a fundamen- concurrent VI VI reinforcement is independent tal properties of behavior rests on whether they of how the reinforcers are distributed to the two can be derived from simpler processes (Catania, keys. It follows that the sum of the two rates will 1981; Hineline, 2001; Rachlin, 1971). For example, be constant, so increasing the reinforcement of consider how concurrent VI VI schedules oper- one necessarily reduces the rate of the other. ate when arranged for a pigeon’s pecks on two Although pigeons distribute their pecks keys. As the pigeon pecks one key, time passes to both keys with concurrent VI VI schedules, during which the VI schedule for the other key there’s a complication. If pecks on one key are may set up a reinforcer. A time comes when the immediately followed by a reinforced peck on reinforcement probability for changing over to the other, the reinforcer may act both responses, the other key exceeds that for continuing to peck so pecks on one key are maintained partly by the same key. If the pigeon emits the response reinforcers scheduled for the other. For this with the higher current reinforcement probabil- reason, concurrent VI procedures have often ity and this probability shifts from one key to the 178 • Part V. Learning Without Words: Contingencies

other as time passes, the pigeon will distribute its measures such as distributions of interresponse responses to both keys in concurrent VI VI sched- times, liked other measures called molar, are ules (Hinson & Staddon, 1981; Shimp, 1966). derived from behavior sampled over extended This has been called maximizing; with several periods of time. Biology has gone through responses available, maximizing means emitting similar controversies in its history, but it now the response with the maximum reinforcement appears that the molar biologists were wrong to probability. With unequal concurrent ratio sched- give up on the microbiologists. “In evolutionary ules, this is always the one with the smaller ratio, parlance, Evo Devo reveals that macroevolu- but with concurrent VI VI schedules the response tion is the product of microevolution writ large” with the maximum reinforcement probability (Carroll, 2005, p. 291). Why should we expect changes from moment to moment; concurrent VI behavior to be any different? VI performance has therefore been called momen- tary maximizing. Thus, momentary maximizing at Natural Foraging the molecular level may lead to matching at the molar level. Matching and maximizing may seem Concurrent-chains have been broadly applied contradictory alternatives, but they are measured to the synthesis of complex behavior. If the in different ways (Hineline, 2001). interpretation of complex behavior in a natural We can’t speak about matching without some habitat suggests it consists of several simpler sample of responses and reinforcers from which components, the interpretation can be tested by to estimate relative frequencies, but we can trying to assemble those components in a labora- speak about maximizing with a single response, tory setting. A successful synthesis supports the just by noting whether it was the one with the interpretation; an unsuccessful one may reveal maximum reinforcement probability. To some inadequacies in the assumptions about what extent, the issues involve the level of detail at was going on in the natural setting. which performances are analyzed. Furthermore, In the fi eld of behavioral ecology, this strategy matching and maximizing don’t exhaust the is illustrated by studies of natural foraging (Fan- possibilities. For example, other analyses have tino & Abarca, 1985; Kamil, Yoerg, & Clements, examined whether concurrent performances can 1988). In their foraging, animals in the wild travel be described as optimization (the organism pro- from one food patch to another, staying or mov- duces the highest possible overall reinforcement ing on to new ones depending on what they fi nd. rate), satisfi cing (it meets some minimal require- For example, a bird might fl y to a bush in which ment, such as a given food intake) or melioration the eggs of an edible insect have just hatched. As (it balances performance so as to produce equal it eats, it gradually depletes its prey, and when it reinforcement rates under different conditions); moves on depends on such factors as how much this is not an exhaustive list, and in their quanti- is left, how far it must go to fi nd another bush tative details these and other treatments (Killeen, and what the chances are of fi nding food there 1994; Mazur, 1991; Nevin & Grace, 2000) are (Wanchison, Tatham, & Hineline, 1988). more than we can handle here. Because analyses of natural foraging involve Whether generalized matching is a funda- switching among food sources as food availabil- mental process in some way dictating the details ity or other conditions progressively change, as of schedule performances or is a derivative of the in the last example, progressive schedules have moment-to-moment responding generated by been useful tools (Hackenberg, 1992; Neuman, reinforcement schedules has long been a source Ahearn, & Hineline, 2000). In a progressive of controversy. Is behavior built from the top schedule, some parameter of the schedule var- down or the bottom up? Those who argue for ies over successive reinforcers or blocks of rein- the former call theirs a molar approach (Baum, forcers. By analogy to the cost of switching over 2002), contrasting it with molecular analyses. from one food source to another, some proce- But this is a false dichotomy, because molecular dures provide an alternative response allowing Chapter 18. Behavior Synthesis • 179 the organism to reset the schedule to its starting concurrent-chain schedules, we’ve often omitted value. For example, a progressive FI schedule procedural details. This is perhaps inevitable. arranged for a pigeon’s pecks on one key might Unlike most of our other topics, reinforcement start with FI 30-s and add 15-s to the FI value schedules didn’t even exist as a systematic sub- with each reinforcer, but completing an FR on a ject matter until relatively recently (Ferster & second key might reset the progressive FI back Skinner, 1957; Skinner, 1956). One concern of this to 30-s. The FI value at which the pigeon resets subject matter has been maintaining behavior in the progressive schedule will vary with the size the steady state, and yet the change in behavior of the required ratio, so this schedule can be accompanying any transition from one schedule used to study how the pigeon trades off time (FI) to another is an instance of learning. and number (FR). We examined multiple and mixed, chained Some factors within natural ecological set- and tandem and second-order, and concurrent tings can be simulated within concurrent chains. and concurrent-chain schedules, but we haven’t For example, varying the schedules that operate exhausted the possibilities for combining the in initial links is analogous to varying the time basic schedules. For example, reinforcers can be and effort involved in traveling from one bush to arranged for completing either of two schedule another, and varying the schedules in terminal requirements (alternative schedules) or both of links is analogous to varying the availability or them (conjunctive schedules). In an alternative depletion of different food sources at different FI 30-s FR 50 schedule, either the fi rst response sites. Concurrent-chain schedules in the labora- after thirty seconds or the fi ftieth response is tory simulating those in natural habitats have reinforced, whichever comes fi rst. In a conjunc- revealed some properties of foraging. For exam- tive FI 100-s FR 20 schedule, a response isn’t ple, organisms are less selective in the food they reinforced until both one hundred seconds pass accept if they spend more travel time (more time and nineteen responses have already been emit- in initial links) between potential food sources; ted. In a progressive schedule , some parameter and if one food is preferred over another, the of a schedule changes systematically over suc- availability of the preferred food (the schedule cessive reinforcers or blocks of reinforcers. For operating during the terminal link in which that example, a ratio might increase by ten responses food is the reinforcer) is a primary determinant after every fi fth reinforcer (sometimes a second of the choice of food patches (as shown by ini- response is available that resets the progression tial-link responding). In other words, natural for- to some starting value according to some sched- aging may be treated as a set of concurrent-chain ule). Adjusting schedules vary as a function of contingencies, and properties of natural forag- some property of performance, as when a ratio ing, in turn, may suggest variables important to varies in proportion to the last postreinforce- concurrent-chain performances. ment pause, or as when shock delivery changes the RS interval of an avoidance schedule. A schedule in which time and number require- A SCHEDULE TAXONOMY ments interact is an interlocking schedule. For example, an interlocking FR FI schedule might Reinforcement schedules are tools applicable shorten the interval as a function of number of to a variety of behavioral phenomena relevant responses, or lengthen the ratio as a function to human concerns. Our examples have ranged of time. An example is winding a grandfather from causal relations between behavior and envi- clock, in which the reinforcer is the tension of ronment to self-control and freedom of choice. the fully wound spring; until the spring is com- Yet even so-called simple schedules aren’t sim- pletely run down, the number of turns required ple; the complexity of schedule effects has made to wind it increases as time passes. schedule analysis highly technical. In surveying For convenience of reference, the major multiple, chained, second-order, concurrent and schedule combinations are summarized in 180 • Part V. Learning Without Words: Contingencies

Appendix IV, which provides defi nitions, exam- plex behavior we may be able to use these tools ples and standard abbreviations. The formal to put the parts back together in a synthesis. designations may make some relations among For example, we can test our interpretation of the schedules more obvious. For example, mul- behavior in a natural habitat by trying to assem- tiple and mixed schedules both involve the alter- ble its components in the laboratory. We can’t nation of component schedules, and chained create a behavior synthesis without making and tandem schedules both involve sequences explicit our assumptions about the properties of in which completion of one schedule produces the behavior we’re trying to synthesize. For that another. In both pairs, the only distinction is reason, when we attempt synthesis we prob- whether the component schedules each oper- ably profi t more from our failures than our suc- ate during different stimuli or during a single cesses. In fact, it may be a general principle of stimulus. Similarly, the only distinction between scientifi c research that we learn the most when concurrent and conjoint schedules is whether our experiments produce data we didn’t expect. the two simultaneous schedules operate for dif- After all, what’s the point of doing experiments ferent responses or for a single response. if we always know exactly how they’re going to These schedule combinations are our tools. turn out? Once we’ve used schedules to analyze com- Part VI: LEARNING WITHOUT WORDS: CONDITIONING Chapter 19 Respondent Behavior: Conditioning

It seems obvious that the whole activity of the organism should conform to defi nite laws. — Ivan P. Pavlov

Historically, respondent conditioning so domi- Respondent conditioning is stimulus con- nated the fi eld of learning it provided the open- trol applied to stimulus presentations rather ing chapters of many learning texts and is still than to consequential contingencies. In other typically treated before operant learning. It’s also words, instead of signaling the consequences of been variably called classical or Pavlovian condi- responding, the stimulus signals that some other tioning. The language of conditioned refl exes stimulus will be presented. Pavlov’s conditioned has entered everyday vocabulary, although in salivary refl exes are the prototype example: its popular usage it’s often confused with oper- when a bell repeatedly signaled food to a hun- ant learning. Many still speak indiscriminately gry dog, salivation came to be elicited by the sig- about operant conditioning and respondent naling stimulus as well as by the food itself. conditioning; a case for some common underly- Ironically, Pavlov may never have used a bell ing processes can be made (Donahoe & Palmer, in his research; his rare mention of bells occurs 1994), but the persistence of that usage runs only in later work and there probably refers to the risk of confusing properties of environ- electrically operated devices. The Russian word ments that interact with behavior in very differ- used by Pavlov for bell, звонок (zvonok), also ent ways. For example, some behavior classes has buzzer and doorbell as synonyms; Pavlov can separately be either elicited or shaped, but didn’t use колокол (kolokol), another Russian respondent procedures don’t lend themselves to word for bell that doesn’t have these synonyms. shaping. Besides, Pavlov was scrupulous about stimulus The term conditioned came from the Russian consistency, and unlike buzzers or doorbells, a in the phrase for conditioned refl exes, условный handbell with a clapper is likely to vary from рефлекс (uslovnyi refl eks). The term should have one ring to the next. The ubiquitous references to been translated as conditional, because it was Pavlov’s bell may have originated with the com- applied to refl exes conditional on relations mon use of the dinner bell and salivation as an among environmental stimuli. As with oper- example in the writings of John B. Watson and ant phenomena, vocabulary has consequences: others. I visited Pavlov’s apartments in 1993, in the language of conditioning, it’s too easy to when St. Petersburg was still called Leningrad. talk about the organism being conditioned while The apartments are now a Pavlov museum, and neglecting the stimulus-stimulus contingencies the only bell was on his desk, presumably for that led to the changes in its behavior. summoning servants or assistants.

181 Part VII: LEARNING WITH WORDS: VERBAL BEHAVIOR Chapter 21 Social Learning

It is because our behavior is important to others that it eventually becomes important to us. — B. F. Skinner

One variety of selection operates on popula- explicitly taught, either in the informal context tions of organisms over successive generations. of interactions between my family members and It appears as phylogenic selection, Darwin’s treat- friends or in the formal context of the schools I ment of evolution as natural selection. Another attended. But the earliest types of social learn- operates on populations of responses within the ing must have been much simpler. At what lifetime of an organism. The effects of conse- point did organisms begin to learn to do things quences, especially in the shaping effects of rein- just by attending to what happened as others forcers, involves this variety, ontogenic selection. did them? The third variety also operates on behavior but We may think of learning by observation as involves more than one organism. The behav- commonplace. It occurs so often in our human ior an organism acquires within its lifetime is behavior. But it’s not clear how much of it goes eventually lost if not passed on to others. Once on even among nonhuman warm-blooded ver- social learning becomes possible, learned behav- tebrates (mammals and birds). Among primates, ior can survive the organism that learned it. It one example involved sweet potatoes fed to then continues in what others do, perhaps in the macaque monkeys on a Japanese reserve adja- behavior not only of descendants but even of cent to a beach (1959). The potatoes typically contemporaries who aren’t closely related. This became sandy, but one juvenile monkey discov- variety, cultural or memetic selection, involves the ered that rinsing them in the ocean removed the selection of behavior passed on from one organ- sand. The practice spread to other juveniles of ism to another, as when local languages or eth- the colony but wasn’t taken up by the adults. nic cuisines or religious practices spread within Human cultures offer many examples of the or across cultures. social selection of behavior as behavior is passed on from one individual to another. Some ways of raising children, getting and preparing food, KINDS OF SOCIAL building shelters and dealing with group mem- CONTINGENCIES bers and outsiders survive over generations. Some practices spread to other groups who Learning from others is especially signifi cant aren’t close genetic relatives of those who began in human behavior. A substantial part of what them. In contemporary Western culture, for I know I’ve learned from others, and some was example, ethnic foods are prepared and eaten by

198 Chapter 21. Social Learning • 199 many who aren’t members of their originating mates the receptive must be distinguished from ethnic groups. the unreceptive; and so on. In many organisms, Some socially transmitted behavior survives such properties are correlated with anatomical because of its consequences. In early human features (e.g., colorful plumage in birds); often, history, the person who learned from someone behavior is the crucial dimension (e.g., courtship else how to make stone tools or fi re or garments dances in birds). was probably also more likely to survive long Releasers and fi xed action patterns provide enough to pass the behavior on to someone else many examples of effects of social stimuli. In than one who couldn’t learn that way. Suppose some cases, the behavior serving as a social stim- some patterns of child rearing work so adults are ulus and the behavior produced by that stimulus likely to rear their own children the same way are topographically similar, as when one bird’s their parents reared them; let’s call these self-rep- takeoff triggers the fl ight of other birds, or as licating patterns of child rearing. Whenever a when a few galloping steers set off a stampede. self-replicating pattern happens to be used by Such cases may superfi cially look like imitation, some parents, for whatever reason, it will likely but they’re limited to a narrow range of response appear again in the next generation; other pat- classes (e.g., contagious yawning: Provine, 2012) terns will come and go. Little by little over many and must be distinguished from it. generations, the self-replicating patterns will Discriminating the behavior of other organ- displace the others and become traditional. Once isms, whether of one’s own or other species, has such patterns get going, they’re likely to survive selective advantages. If one antelope in a herd for a very long time, perhaps even in the face of limps just a little, a lion that notices the limp may powerful countervailing contingencies such as be more likely to make a capture. The antelope discriminatory practices. that can tell the difference between a lion that Darwinian evolution and shaping depend hasn’t eaten for a while and another that’s just on variable populations on which selection can fi nished may be more likely to move away in a operate. Similar constraints exist at the level of safe direction. A predator that can distinguish cultural selection. For example, cultural practices whether it’s been noticed by its prey has a dis- favoring ethnic diversity may survive those that tinct advantage over one that can’t; an advan- don’t simply because they allow such variability. tage also accrues to a prey that can distinguish It was no help to Hitler that some Jews who left whether it’s been noticed by its predator. Over Europe because of Nazi policies included scien- an extensive phylogenic history, attention to tists like Albert Einstein. At every level of selec- the behavior of prey was a dimension on which tion, we usually look at how selection operates natural selection operated in the evolution of at the level of the survival of individuals rather predators, just as attention to the behavior of than the survival of the group, though the need predators was a dimension on which natural for that constraint has sometimes been ques- selection operated in the evolution of prey. Fol- tioned (D. S. Wilson & Sober, 1994). lowing from such selection, discriminations of social behavior have become so sophisticated Learning about Others they sometimes override other types of discrim- inations. In many situations, the discriminative stimuli Discriminations of the behavior of others are provided by others are more important than at the heart of the concept of intentionality (Den- those provided by inanimate objects and events. nett, 1987): I say I understand someone’s inten- For example, parental investment in offspring tions when my discriminations of that person’s may be wasted if parents can’t discriminate past and current behavior allow me to act appro- between their own offspring and the offspring priately toward that person’s future behavior. of others; potential mates must be distinguished In fact, if discriminating my own behavior is a from potential competitors, and among potential special case of discriminating the behavior of 200 • Part VII. Learning With Words: Verbal Behavior

others, it can be argued this topic encompasses why it’s hard to get rid of rats by poisoning. One all the phenomena considered under the rubric rat has learned about some novel foods from of intentionality. Our judgments of the inten- another, in the sense that combinations of foods tions of others are social judgments (Bem, 1967). and social stimuli change the effectiveness of the It takes no special assumptions about selective foods as reinforcers or as aversive stimuli. contingencies to see they could shape well-pre- Observational learning has been compel- pared capacities for social discriminations both lingly demonstrated with rhesus monkeys within and across species. (Mineka et al., 1984). Monkeys in the wild show Social discriminations within species have fear of snakes by screaming and other agitation many functions: within dominance hierarchies; and by snake avoidance. Even if their parents in the defense of territory; as isolating mecha- fear snakes, monkeys reared in the laboratory nisms maintaining the integrity of a group who’ve had no experience with snakes don’t; for against intrusions from outsiders; in the distri- example, if food is on the other side of a con- bution of limited resources among group mem- tainer with a snake in it, they’ll reach across bers. Within species living as social groups, like the snake for the food. But if laboratory-reared most primates, individuals learn what sorts of monkeys briefl y observe a parent behaving behavior to expect of those with whom they fearfully toward snakes, they too become fear- have extensive contact. ful. Their fear is intense and persistent; it’s still there if tested three months later. What they’ve Learning from Others learned is based solely on observing a parent’s behavior toward a snake. But there’s a phylo- It’s one thing to learn about other organisms. It’s genic component, because such learning is more another to learn something from them. Some- likely with snakes or snakelike objects than with times the behavior of one organism allows other types of stimuli. These kinds of learning another to act based on stimuli available only may play a critical role in the onset and mainte- to the fi rst, as when a vocal call from one mon- nance of post-traumatic stress disorders (PTSD) key allows another to escape from a predator it and other human anxiety disorders (Mineka & hadn’t seen. Warning calls are well-documented Zinbarg, 2006). in bird behavior. In monkeys, predator calls can Observational learning is a prerequisite for vary with kinds of predators, and the response other varieties of learning and social interaction. to the call can depend on who the caller is and For example, joint attention, in which a child who the listener is (Seyfarth, Cheney, & Marler, looks where a caregiver is looking or pointing, 1980a). depends on the child fi rst learning to attend to Observational Learning. Learning based on the caregiver’s looking and pointing (Dube et observing the behavior of another organism is al., 2004). Observational learning is probably called observational or vicarious learning (Zentall best treated as a higher-order variety of behav- & Levine, 1972). Sometimes what seems to be ior. Different skills must come together appro- observational learning involves simpler pro- priately for observational learning to work; in cesses. For example, food preferences in rats human observational learning it’s also likely are learned in social contexts. When rats come there’s a large verbal component. together they sniff and lick each other, so by smell and taste each rat becomes familiar with Imitation. The crucial difference between the food the other has eaten (Galef & Stein, 1985). observational learning and imitation is that in If the food is novel and the other rat is healthy, imitation the observer’s behavior corresponds to the fi rst rat later prefers that novel food over that of the observed organism. Imitation doesn’t other novel foods, but if the other rat is sick, the imply the imitating organism has learned about fi rst rat avoids that novel food. In other words, contingencies, so not all imitations are advanta- this is a socially mediated taste aversion; that’s geous. For example, a coyote seeing another coy- Chapter 21. Social Learning • 201 ote step into a trap would do well not to imitate defi ning it are complicated. For example, if you that behavior; a hatchling bird that hasn’t yet touch the top of your head when it’s modeled grown its fl ight feathers would do well not to in the game of Simon says, you see but don’t feel follow its parents when they fl y from its treetop the leader’s hand and you feel but don’t see your nest. own hand. Someone else’s seen hand and your Following may sometimes be imitative. After own felt hand involve different stimulus dimen- food-deprived pigeons learned to eat from an sions, so how did you learn to imitate? experimenter’s hand, the hand then approached You can learn such correspondences by and "pecked" the key, producing food; the behaving in front of a mirror. Humans and some pigeons followed the hand and began to peck the primates can learn these correspondences with- key (Neuringer & Neuringer, 1974). This proce- out explicit training. For example, if a chimpan- dure often worked more rapidly than shaping of zee experienced with mirrors has a spot painted the key peck. Under natural conditions, young on its eyebrow while asleep, it will touch the animals may learn to behave like their parents spot the next time it sees its face in a mirror. simply by following them to sources of food. Some monkeys pass this test and others don’t, We call responding imitative when one organ- but the differences may depend not on cognitive ism duplicates the behavior modeled by another. capacities but on behavior interfering with look- Some imitations may be limited to duplicating ing in mirrors (Rajala et al., 2010). For example, only specifi c instances explicitly taught, while species of monkeys differ in gaze aversion , i.e., others may involve correspondences between looking away if another monkey looks at them; the behavior of model and observer even in novel thus, when some monkeys look in a mirror and instances, when it’s called generalized imitation; see themselves looking back, they’ll look away in the latter, imitative responding is a class that before seeing the spots on their eyebrows. may be differentially reinforced; in other words, With other organisms, such as pigeons, these it’s a higher-order class (D. M. Baer, Peterson, correspondences can be taught. First, a pigeon’s & Sherman, 1967). Suppose a child learns to pecks at blue cardboard dots pasted on its body imitate behavior modeled by a puppet: jump- were shaped. Next, discriminations among blue ing, skipping, clapping, etc. If consequences dots refl ected in a mirror were taught by pre- maintain imitation of all but the clapping, the senting the dots behind holes in one wall only child will usually continue to imitate clapping when the pigeon faced a mirror on the opposite along with the others even though this imita- wall; each dot was gone by the time the pigeon tion is never reinforced. If imitation of clapping turned around, but only pecks at the hole where doesn’t extinguish, it’s a member of the general- it had appeared were reinforced. When another ized class. With generalized imitation, the child blue dot was attached to the pigeon’s breast will also produce novel imitations if the puppet while it wore a short bib allowing it to see the does something it never modeled before, such as dot in the mirror but not by looking down, the stamping its foot; in fact, responses occasioned pigeon pecked down toward the dot on its body by novel stimuli defi ne generalization, as when even though it saw the dot only in the mirror a pigeon pecks when it fi rst sees yellow or blue (Epstein, Lanza, & Skinner, 1981). Did monkeys after a history of pecking only during green. who failed the mirror test simply lack relevant Because modeling of a new response can produce experience with mirrors? novel behavior only if a child’s imitation has generalized, it can supplement shaping, as in Social Origins of Language. Another way to teaching skills to children with autism or other learn from other organisms is through verbal developmental issues (degli Espinosa, 2011). behavior: you can be told about contingencies Generalized imitation is consistent with what instead of observing them. But verbal behavior we understand about the contingencies creating can’t have started that way, because descriptions higher-order classes, but the correspondences of contingencies require sentences and the ear- Chapter 22 Words as Stimuli and Responses

In an American school if you ask for the salt in good French, you get an A. In France, you get the salt. — B. F. Skinner

Our talking and listening and reading and writ- The language of meaning is another compli- ing are all behavior, but everyday usages get in cation. Dictionaries don’t contain meanings of the way when we talk about them. Consider the defi ned words; they just contain other words. common term word . We seldom bother to dis- We speak metaphorically when we say words tinguish spoken words from written ones. Yet contain meanings and that we convey these mean- speaking a word isn’t the same as writing it, and ings to others through language. The metaphor speaking and writing usually occur in different of words as containers for meanings has been circumstances. Even worse, we often speak of around for a long time, and yet the stored pat- using words, as if they were things instead of terns corresponding to a recorded voice or the behavior. I don’t say I use a reach when I pick patterns of pigment on a page corresponding to something up, but I might say I use a word a handwritten message have no meaning unless when I say something (Skinner, 1957). someone listens to the recording or reads the We also speak of words as if directed toward note; the meaning isn’t waiting to be released events or objects. We say words or sentences from the computer or the ink. If language trans- refer to, deal with, speak of, call attention to mits anything, it’s verbal behavior itself; in lis- or are about things. The language of reference tening and reading, our own behavior re-creates implicitly includes a direction from verbal some features of the behavior of the speakers behavior to environment. Everyday language and the writers constituting our verbal commu- doesn’t include words emphasizing the other nity. We share our verbal behavior; it is, above direction. As with treating vision as light enter- all, social behavior. ing the eye instead of as something emanating Our fi rst task is classifying verbal behavior. from the eye and contacting things seen, getting But our taxonomy must be functional, not struc- the direction right makes a difference. Has our tural. A grammatical classifi cation of words in a everyday language prejudiced us about how sentence doesn’t tell us how it was produced or verbal behavior works? We hardly ever say what consequences it had for the one who pro- nouns are uttered in the presence of relevant duced it. The same holds for phonological classi- objects or sentences are occasioned by relevant fi cations of the sounds in someone’s speech. You events. Instead, we say words refer to objects can say “I’m thirsty” or “Please pass the water” or sentences are about events. These usages are and get the water either way. They’re members appropriate to the equivalences relating differ- of the same verbal operant class, but linguists ent classes of verbal behavior, but in analyses of will usually put them in different grammatical the behavior of speakers and listeners or readers categories. Functional accounts of verbal behav- and writers they can be misleading. ior examine what verbal responses do. As with

205 206 • Part VII. Learning With Words: Verbal Behavior nonverbal behavior, structural and functional both listener behavior shaped by its effects on accounts can complement each other. Unfortu- the speaker’s behavior, and speaker behavior nately, those of verbal behavior were often pitted shaped by its effects on the listener’s behavior. against each other as if they were incompatible These reciprocities defi ne verbal behavior. Ver- instead of complementary (Catania, 1972, 1973b; bal behavior is shaped and maintained by the Chomsky, 1959; Skinner, 1957). practices of a verbal community. When we defi ne verbal behavior this way, by its function, we distinguish it from language VERBAL BEHAVIOR: NOT A (Skinner, 1957, p. 461). Languages are defi ned SYNONYM FOR LANGUAGE by structure and not function. The defi nitions, spellings and pronunciations in dictionaries and By itself, verbal behavior doesn’t do things. It’s the rules in grammar books describe standard effective through the mediation of other people. structures of verbal units in a language. In so The mode isn’t critical; you can tell someone doing, they summarize structural properties of what you want when shopping by asking for the practices of a verbal community. The verbal it or pointing or in writing, as when you place behavior of speakers occurs in the context of an order online. But mediation by others char- those practices, but the maintaining practices, acterizes all social behavior, so we must add language, must not be confused with what they another proviso. The social contingencies shap- maintain, which is verbal behavior. ing verbal behavior don’t just create and main- tain the conditions for speaking. They also create verbal repertories with a special property: in CORRESPONDENCES BETWEEN the ordinary give and take of everyday talk, as SPOKEN AND WRITTEN speakers become listeners and listeners become CLASSES speakers, the behavior of the listener reinforces the behavior of the speaker. Some nonhuman Our verbal communities shape correspondences behavior may minimally qualify, as when a between things and their names, between words horse is taught to turn in response to a touch of and their defi nitions, between what we did and the reins to its neck; the turns then reinforce the what we said we did, between what we prom- trainer’s behavior. But the horse and trainer dif- ised and what we accomplished, and so on. The fer crucially from a child and an adult parent or conditions maintaining correspondences deter- teacher. For the latter only, verbal contingencies mine how they work in our verbal behavior. become reciprocal: the child learns to ask as well Our fi rst examples will consist of formal verbal as answer and to say thank you as well as you’re relations between spoken and written classes, welcome. Thus, in some respects all verbal cul- not because they’re the most fundamental but tures are mutual reinforcement societies (Skin- because they’re familiar and involve well- ner, 1957, pp. 224–226). defi ned correspondences. They illustrate how to Consider a simple verbal exchange: A says, talk about words antecedents and consequences Hi; B replies, Hi, how are you?; A continues, I’m instead of as references and their meanings. In fi ne, thanks. And you?; B answers Okay; and so on. other words, they help us talk about words as The fi rst reply of B is both a consequence of A’s behavior. greeting and an occasion for A’s continuation, The term verbal is general and applies to lan- but so is A’s response with respect to B’s anteced- guage in any modality; we distinguish it from ent and consequent verbal behavior. If you have vocal, which is specifi c to spoken language. We any doubt about whether consequences main- could extend our account to other modalities, tain verbal behavior here, just imagine A’s or such as the gestural modality of sign language B’s verbal behavior if the verbal behavior of the or the tactile modality of Braille, but for conve- other simply ceased. Verbal behavior involves nience we’ll mostly restrict our attention to spo- Chapter 22. Words as Stimuli and Responses • 207 ken and written classes of verbal behavior. Also, cords, tongue, lips and so on. These produce it’s sometimes worth distinguishing spoken from sounds but aren’t themselves sounds. How then vocal, because not all voiced sounds qualify as does the child know what to do to produce the speech. But these technical issues won’t affect sounds heard as “mama” or “dada” by the par- our basic story. ents? Correspondences between verbal stimuli The speech units called phonemes are more and verbal responses in formal verbal relations easily defi ned by articulations (positions of the are implicit in the colloquial vocabulary: we say tongue, etc.) than by acoustic properties (Lane, words are the same whether they’re heard or 1965; Liberman, 1982). The interactions of artic- spoken, seen or written, or, translating, whether ulation and sound are complex; for example, they’re auditory or visual stimuli or responses. many English consonants (e.g., p, b, d) can’t be An elementary verbal function is the replication produced unless accompanied by a vowel, and of verbal behavior: we repeat what others say or their acoustic properties vary as a function of copy what others write. Thus, our cases include context (e.g., both the l and k sounds are different the replication of verbal behavior in all four in lick and in kill). Echoic behavior isn’t defi ned combinations of spoken or written stimuli and by acoustic correspondences; it’s defi ned by spoken or written responses: spoken to spoken the correspondences of phonetic units. (echoic behavior), written to written (transcrip- Voices differ in many respects: an adult voice tion), written to spoken (textual behavior), and is deeper than a child’s; a woman’s voice dif- spoken to written (dictation-taking). fers from a man’s; people speak with varying It’s easy to forget most of these relations are regional dialects. If a young boy from a small arbitrary. A spoken letter has no visual shape New England town repeats what a grandmother and a written letter has no sound. The relations from Atlanta just said, their utterances will dif- between visual and spoken forms come about fer acoustically in many ways. But differences in only after a very long history. What’s important vocal quality and regional dialect are irrelevant here is that saying the letter sets the occasion for to whether the boy’s behavior is echoic; the cri- writing it down. Saying the letter can’t possibly terion for echoic behavior is the correspondence look like the letter, but given an appropriate ver- of verbal units like phonemes and words. That’s bal history it can become a recipe for writing it why a parrot’s duplication of human sound pat- down. terns doesn’t qualify as echoic: its duplications are acoustic rather than phonetic. For example, Echoic Behavior if a child’s lisp of an s is reproduced by a par- rot, the parrot will make the lisped th sound Imitation of properties of vocal stimuli appears but a parent would ordinarily use the unlisped relatively early in an infant’s acquisition of s instead. A voice synthesized by a computer speech. We call this class of verbal relations often sounds unrealistic because the computer echoic. When a parent says “mama” and the doesn’t produce its approximations to speech child repeats “mama,” the child’s response is the way a human vocal tract does. echoic to the extent (i) it’s occasioned by the Echoic units can vary in size from individual parent’s utterance and (ii) the phonemes of the speech sounds to extended phrases or sentences. child’s utterance have a one-to-one correspon- A variety of verbal phenomena, such as speech dence to those of the parents. Even though the errors (Fromkin, 1971), can help us decide what stimulus and the response have common prop- these units are. Echoic relations can appear in erties, this verbal relation isn’t simple. Hearing a poetic rhyming and alliteration (Skinner, 1972; sound isn’t the same as making the vocal adjust- B. H. Smith, 1968). For adult speakers, units of ments producing the sound. The stimulus is a echoic behavior may be whole words or phrases. complex sound pattern. The response consists The echoic production of extended phrases or of the coordinated articulations of lungs, vocal sentences occurs in dramatics, as when an actor 208 • Part VII. Learning With Words: Verbal Behavior repeats lines whispered by a prompter, and on the differences when they make the sounds ritual occasions, as when a bride and groom themselves and some of the sounds may become repeat a marriage vow spoken by a member of reinforcers (Friedlander, 1968). the clergy. Echoic units aren’t defi ned by size; At fi rst their babbling includes undifferen- they’re defi ned by the correspondences into tiated sounds, but over time they retain native which they enter. speech sounds in their spontaneous vocaliza- Echoic behavior doesn’t simply accompany tions while other kinds gradually disappear. It the acquisition of language and then vanish; was once thought early infant sounds included it persists in mature speakers. For example, I all possible human speech sounds, but instead might repeat the name of someone to whom I’ve infants progress from intonations and stress pat- just been introduced. But whether a speaker has terns to vowels and then consonants, starting understood what’s been echoed is independent with the easiest ones (Vihman, 1996). Eventually of whether the speaker has echoed successfully. their babbling evolves to self-repetitions ( echo- Meaning doesn’t enter into the defi nition of lalic speech; e.g., “ma-ma-ma-ma-ma”) and then echoic behavior; we must deal with it in other repetitions of the speech of others (echoic speech). ways. Vocalizations can be reinforced (Poul- Some verbal responses are only partially son, 1984), and the vocalizations of infants are echoic, as when I say something incorporating engendered and maintained by what they hear just a couple of words from a question I’ve been themselves saying; without these auditory con- asked. In his design of a system to tap latent ver- sequences (as in cases of hearing impairment), bal behavior, Skinner (1936) took advantage of the behavior doesn’t develop. As we know from partial echoics. To the participants, his verbal sum- recordings, our own voices sound different to mator was supposed to test speech detection in us than they do to others, because we hear our noise. After general instructions, they were asked own voices via bone conduction as well as from to “Listen to the phonograph until you fi nd your- sound traveling through the air. This adds to self saying something; then say it aloud.” Their the case that echoic behavior is defi ned by cor- stimuli were repeated infl ection patterns (e.g., oo- respondences of learned phonetic units rather ah-ee, oo-ah-ee, oo-ah-ee) and not words; nonethe- than by the matching of acoustic properties. less their verbal responses were typically words. Rare individuals may, through accidents of The infl ection patterns of the words matched anatomy, hear themselves as others hear them. those in the stimuli but also refl ected whatever They have the advantage of knowing how well verbal behavior was currently strong in the lis- they’ve imitated other voices. I fi rst heard how tener’s verbal repertoire for other reasons, so they different my own voice sounded to me than to worked like an auditory analog of the Rorschach others when a high school friend used a hidden inkblots used in personality testing. tape recorder to record a group conversation. After listening to the tape, one of our group said The Development of Echoic Behavior. Echoic he didn’t hear much of a difference. He was good behavior depends at least in part on the shap- at doing vocal impressions. Given he sounded ing of articulations by their vocal consequences roughly the same to himself as to others, if he (Risley, 1977; Skinner, 1957, p. 58). Before their talked so he sounded like Humphrey Bogart to own vocalizations begin to be differentiated, himself then he’d sound like Humphrey Bogart infants discriminate among many aspects of the to us too. I’ve occasionally asked in class about speech of those around them (Eimas et al., 1971); how people hear their own voices; over the for example, they readily discriminate between years I’ve encountered fewer than half a dozen sounds of their native language and those from who said they heard themselves as others heard an unfamiliar foreign language but not between them. But each was good at doing impressions. sounds from two unfamiliar foreign languages Perhaps native speech sounds become rein- (Mehler et al., 1988). This means they can hear forcing relative to sounds of nonnative lan- Chapter 22. Words as Stimuli and Responses • 209 guages simply because they often accompany boundaries between the projection areas for dif- activities of the infant’s caregivers (DeCasper ferent fi ngers that can be visualized with appro- & Fifer, 1980). An articulation producing some- priate stains. Adjacent cells on one fi nger are thing sounding like what mommy says may be likely to be stimulated at the same time, whereas reinforced automatically by this correspondence those on a neighboring fi nger, though projected between the infant’s and the parent’s utterances. close by on the cortex, are likely often to be stim- The differentiation of phonemic structure may ulated separately. These differences between then follow from the overlapping contingencies correlated and uncorrelated fi ring may provide into which different speech sounds enter. the basis for a boundary. Moreover, the number of compartments is determined during develop- Categorical Perception of Phonemes. A child ment by the number of units at the periphery raised exclusively in an English-speaking envi- (e.g., the separate rays in the nose of the star- ronment will discriminate between the conso- nosed mole: Catania, K. C., & Kaas, 1997). nants b and p, but one raised exclusively in an This has been an ontogenic account of the Arabic-speaking environment, which doesn’t natural selection of phonetic units of verbal include separate b and p consonants, won’t. Fur- behavior. The selection is based on correspon- thermore, if the b is gradually changed to a p dences between the language already available for the English speaker, the switch from calling in a verbal community and the vocalizations the sound a b to calling it a p occurs discretely; of an infant language learner, so it provides a listeners don’t report an intermediate conso- means for transmitting verbal behavior at the nant that’s a cross between them. This type of cultural or memetic level, in this case from one transition is called categorical perception , in that generation of speakers to the next. There are responses to stimuli varying along some contin- ways it can fail: motor disorders may constrain uum don’t also vary continuously but instead articulation; hearing may be impaired; relevant fall into two distinct categories (Harnad, 2003). consequences may be absent, perhaps because Another fact about acquisition of phonemes of neglect or abuse by caregivers or because is that discriminations of speech sounds learned neurological defi cits have reduced the effective- easily at an early age may be hard to learn later ness of social stimuli. But once vocal articula- (Werker, 1989). For example, the distinction tions have been shaped and the child can hear between spoken r and l in English doesn’t exist their consequences, the minimum conditions in Japanese and is much more easily learned by for the ontogenic selection of vocal units are in a Japanese child than an adult Japanese speaker. place. In contrast, reinforcement of nonhuman Infants master simple articulations before mov- vocalizations is typically limited to rate; it’s dif- ing on to more complex ones. If the vocal units fi cult to modify topography (K. Hayes & Hayes, differentiated during babbling are different from 1951; Lane, 1961). A lot may follow from the one language to another, perhaps an adult learn- shaping of speech: “The human species took a ing a new language fi nds it diffi cult to master the crucial step forward when its vocal musculature new phonemes in part because the prerequisite came under operant control in the production simpler articulations were never differentiated. of speech sounds. Indeed, it is possible that all The environment may be able to sculpt lan- the distinctive achievements of the species can guage units in various ways. Developing ner- be traced to that one genetic change” (Skinner, vous systems divide into functional regions 1986, p. 117). sometimes called compartments (Irvine & Raus- kolb, 2001; Kiecker & Lumsden, 2009). The for- Transcription mation of boundaries may also be signifi cant in the development of behavioral units like pho- Verbal stimuli and responses can correspond nemes (Kim & Bao, 2008). For example, we know when both are written. In such cases, the behav- cortical tactile receptor areas in primates include ior is called transcription . For example, you might 210 • Part VII. Learning With Words: Verbal Behavior copy a number from a telephone book or an printed might produce an accurate copy, but author and title in preparing a bibliography. It the copying would not be verbal. The behav- doesn’t matter whether you produce the copy ioral units in the two kinds of copying matter. by writing it or typing it or dragging and drop- The critical features of the calligrapher’s copy- ping it. Just as we distinguish vocal articulations ing are geometrical properties of letters in the and the sounds they produce in echoic behavior, text and marks produced by the calligrapher’s we also distinguish the movements involved in strokes; the critical features of transcription are producing words from the looks of the words in the verbal units—letters, words and phrases— transcription. Just as echoic behavior depends on in the original text and its copy. Visually, the correspondences of verbal and not acoustic prop- calligrapher’s copy might look more like the erties, transcription depends on correspondences original than a handwritten copy by a speaker of verbal and not visual properties. A handwrit- of the language, but only the latter counts as ten sentence looks different from the printed text transcription. An A may be transcribed as an A from which it was transcribed (the script letters or an a or an , but a mere picture of an A run together but the printed ones don’t). Even so, doesn’t qualify. writing the sentence qualifi es as transcription if Except for their respective spoken and the script sentence replicates the printed one in written modes, echoic behavior and transcrip- spelling, word order and punctuation. tion are formally similar. Children ordinarily Just as units of echoic behavior can vary acquire echoic behavior early even without spe- from individual phonemes to entire phrases or cifi c instruction, but they take some time doing sentences, units of transcription can vary from so. They don’t ordinarily acquire transcrip- individual characters to extended passages, tion unless it’s explicitly taught. Nevertheless, depending on the circumstances in which the the relative ease with which one or the other is behavior occurs. A child learns to copy single learned provides no grounds for assuming one letters before learning to copy whole words. is simpler than the other. In doing so, the child learns correspondences Pure transcription is probably rare. It occurs between arbitrary visual forms, such as printed only when transcription is unaccompanied by and script a in upper- and lowercase. No visual other behavior, such as responding to a text in property is exclusively common to all forms of reading for meaning. For example, if a skilled the letter a. Consider the different forms of the typist accurately transcribes a letter while not several following English letters: responding verbally to it in other ways, say while listening to a conversation elsewhere in the offi ce, the typist afterwards may be unable to describe what was in the letter. This would be a case of relatively pure transcription: the typist was transcribing and doing nothing else. Responding to the text in other ways, as in Such differences within the forms of a single reading or understanding it, is doing something letter are sometimes greater than those between more that may combine with the transcription different letters, as in the following groups of let- but doesn’t count as part of it. When we’re tran- ters with properties in common: scribing, we’re usually doing a lot more besides. As with echoic behavior, meaning isn’t part of hnm – ODQ – bdpq – MNVW our defi nition of transcription. The variations of form explain why tran- Textual Behavior scription differs from copying in the pictorial sense. A skilled Asian calligrapher unfamiliar When a written verbal stimulus sets the occasion with the European alphabet in which a text is for a corresponding spoken verbal response, the Chapter 22. Words as Stimuli and Responses • 211 behavior is textual. You might say aloud what’s you’re reading this book!). Such experiences on a menu or read a bedtime story to a child. In are evidence for the importance of these dis- textual behavior, the arbitrary correspondence tinctions. Reading for understanding includes between verbal stimuli and responses is more other behavior along with or instead of vocal obvious than in either echoic behavior or tran- or subvocal speech, so it’s more—probably a lot scription, because the stimuli and responses are more—than textual behavior. in different modes. A written word is a visual stimulus; it has no sound. A spoken word is an Dictation-Taking auditory stimulus; it has no shape. Yet these cor- respondences are so familiar we rarely notice the Just as a written stimulus can set the occasion arbitrary nature of the relations between verbal for a spoken response, a spoken stimulus can shapes and sounds. set the occasion for a written one. This is called As with transcription, textual behavior is dictation-taking. We’re concerned here with the usually taught explicitly, and some controver- listener who takes the dictation, not the speaker sies about teaching it are based on assumptions who dictates. For example, you might take notes about the units appropriate to various stages at a lecture. The units of dictation-taking are typi- of instruction, such as whether the teaching of cally entire words or phrases, but individual let- reading should begin with individual letters, ters may also serve (e.g., as when children are syllables or whole words (Gleitman & Rozin, taught the written alphabet or an unusual name 1973). is spelled out for a stenographer). As with tran- As with the other formal classes, we must scription, the form can vary: shorthand, script distinguish textual behavior from other kinds of and typing all qualify. responses to written verbal stimuli. For example, Some of the special properties of dicta- if a sign says STOP, reading the word aloud is tion-taking follow from the relatively permanent textual but stopping isn’t. In the mature reader, record produced in a written text. In addition, textual responses become less important than occasions for dictation-taking are limited rela- other kinds of responses to written verbal stim- tive to textual behavior, because, unlike our uli. Vocal responses diminish in magnitude, vocal apparatus, writing implements aren’t become subvocal and perhaps disappear com- parts of our anatomy. Perhaps for this reason, pletely as a child becomes a profi cient reader. we aren’t much tempted to pursue its covert Reading is behavior but it’s far more than textual manifestations; we’re more likely to speak of behavior; textual responses are at best one part subvocal reading than of submanual writing of reading. For example, a father reading a bed- or typing. Even so, textual behavior and dicta- time story aloud to a child might fi nish a page tion-taking are formally similar and either can and suddenly realize he doesn’t know what just be accompanied by other kinds of verbal behav- happened in the story, even though the child ior occasioned by verbal stimuli. does. His example, without understanding, is a pure case of textual behavior. Relations Among the Classes The colloquial vocabulary doesn’t distin- guish between reading when it’s nothing more This account of formal verbal classes has been than saying the words on a page and reading limited to spoken and written stimuli and when it’s the kind we call reading for under- responses. We could extend it to other modes, standing. Most of us have occasionally found such as writing and reading Braille or produc- ourselves partway through a page unable to say ing and reading sign language or sending and what was in the last couple of paragraphs. We receiving Morse code. Sometimes we can ignore were at that time presumably engaging in some the distinction between verbal stimuli and ver- reduced form of textual behavior and absolutely bal responses, but not always. For example, nothing else (I assume it never happens while though we speak of letters and words without 212 • Part VII. Learning With Words: Verbal Behavior regard to whether they’re written or spoken, the written. Thus, saying what has been written or mode matters in teaching. A teacher who has writing what has been said are instances of rep- only taught a child to name letters written on a lication even though they carry across different chalkboard shouldn’t expect the child to write modalities. These relations extend the effects of the letters when they’re spoken. The stimuli in verbal stimuli over time and space as the verbal the former are the responses in the latter, and behavior produced by some is passed on to oth- vice versa. Until they become equivalent, they’re ers. Furthermore, replications allow instructions just one part of “knowing the alphabet.” to be followed in the original speaker’s or writ- To teach reading and writing is to teach er’s absence, later and elsewhere, as when we equivalences between spoken and written modes repeat to ourselves the details of a task someone of verbal behavior. Our everyday vocabulary has asked us to complete or write notes on which obscures these distinctions, perhaps because we can later act. Replication must come fi rst, but equivalences so come easily to us, or perhaps once in place powerful contingencies maintain it. because language instruction is designed to We’ve emphasized classes defi ned by verbal eliminate them, or perhaps both. In any case, the modes, but other formal relations may be dif- relations are summarized in Figure 22–1. ferentiated even within a mode. For example, a student who has learned only to translate from English to German may have diffi culty when THE REPLICATION OF VERBAL asked to translate in the other direction. The BEHAVIOR problems are compounded when both spoken and written languages are involved. Language All the formal classes involve the replication of instruction usually recognizes these distinctions; verbal behavior in spoken or written form. Dar- a course in conversational French is expected to win discussed parallels between the evolution of emphasize the spoken mode, whereas one in species and the evolution of languages (Darwin, scientifi c Russian is expected to emphasize the 1871). Under the pressures of modernization written mode. some languages have become extinct and many Each mode, spoken and written, has special others are threatened. The replication of verbal characteristics. Spoken verbal behavior var- behavior isn’t mere reproduction. Based on a long ies more freely in stress, rhythm and intensity history in which we’ve learned relations among than written verbal behavior, but it’s also more spoken and written stimuli and responses, we transient. Some verbal properties are indepen- say words are the same whether spoken or dent of mode. For the immediate consequence

Figure 22–1 Relations among the four formal classes (S, stimulus; R, response). Similar relations can be es- tablished for any pair of verbal modes (e.g., printed text and the gestures of American Sign Language, or spoken behavior and Braille characters). Chapter 22. Words as Stimuli and Responses • 213 of being able to dial a phone number, it hardly musical West Side Story , so it’s been used here as matters whether you hear the number or see it an illustration. on a cellphone screen, or whether you repeat it Some musicians play or sing by ear and oth- aloud to yourself or write it down. Some conse- ers prefer to do so from a score. Some musicians quences may affect how you look for it next time can sing or play a heard melody but many who (e.g., how hard was it to fi nd it?), but they’re not can do so can’t write the music down. The musi- essentially verbal. Thus, once equivalences are cal behavior analogous to echoic behavior is in place distinguishing among the formal classes common among musicians, but it’s variable with diminishes in importance. respect to the analogs of dictation-taking, tran- scription and textual behavior, with some pro- Parallels in Music fi cient in all three and others profi cient in only one or two. But we may need the distinctions for studying Some talented musicians manage without certain topics, such as the language pathologies any musical scores at all on either the stimulus called aphasias , in which defi cits are character- or the response side, so in reproducing music ized by the verbal classes affected (Sidman, they’re limited to echoic music making. If they’d 1971). Some aphasias affect only one of the four been correspondingly limited in spoken and possible relations illustrated in Figure 22–1, so written verbal behavior they’d have been called we can’t always assume equivalences. But we illiterate. But the honed skills of sophisticated don’t often run into aphasias. Music provides a musicians together with contemporary record- more familiar example. In music, the mastery of ing technologies outweigh the formal limits on a subset of the possible formal relations among how they can replicate music, so distinctions like musical stimuli and musical responses doesn’t those we’re considering here simply don’t mat- guarantee the others will be carried along. Fig- ter to their audiences. ure 22–2 substitutes singing and reading or writ- In verbal behavior, a speaker’s response is ing music for the verbal classes of Figure 22–1. a listener’s stimulus and a writer’s response is The word behavior has the same stress patterns as a reader’s stimulus, and a speaker or writer at the word Maria in the song by that name in the one time becomes a listener or reader at another.

Figure 22–2 Relations among four formal classes of musical behavior. They involve singing given a heard song or a musical score, or writing the music given a heard song or a score (S, stimulus; R, response). “Behavior!” is here sung to the melody of the song “Maria” from West Side Story. 214 • Part VII. Learning With Words: Verbal Behavior

Music supplements our other verbal examples Ronald Reagan (Sacks, 1985). Psychotics with- because it’s more obvious in these cases that out affect responded only to the speech content mastery of some formal relations doesn’t guar- whereas aphasics responded only to its affect; antee the integrity of others. They’re separate only those responsive to both dimensions found classes of behavior that become integrated only the speech persuasive. If affective vocal func- given appropriate contingencies. tions are like those of other emotional displays, Verbal behavior and music differ in other they might have provided a vocal substrate on ways (Masataka, 2007). Music is more closely which other functions of verbal behavior were related to tone of voice than to other properties built. Emotional behavior predates language, of verbal behavior. The functional distinction and other evolving functions of verbal behavior between tone of voice and substantive proper- may have been overlaid on it later while prop- ties of vocal discourse is illustrated by the dif- erties like tone of voice retained their emotional ferent reactions of two audiences to a speech by functions. Chapter 23 Antecedents and Consequences of Words

…the relation between a name and that which it names is fundamentally asymmetrical. — Horne & Lowe

The formal classes involve verbal responses and often confl icting contingencies. Many dif- occasioned by verbal stimuli. They’re character- ferent responses are brought under the control ized by one-to-one correspondences of verbal of a given stimulus word, and many different units. In transcription, for example, each word of stimulus words are placed in control of a single a text has a unique equivalent in the transcribed response” (Skinner, 1957, p. 74). Skinner wasn’t version. But we learn many verbal relations that writing about M&Ms. His reinforcements aren’t don’t involve such formal correspondences, edibles but rather the simple consequences of and these relations interact in powerful ways, day-to-day interactions, like a nod or a change as when combinations of basic arithmetic intra- in expression; some consequences are verbal, verbals such as two plus two equals four or three like acknowledgments or other replies. Reading times fi ve equals fi fteen combine in our solution a word in a certain way because of its context of a mathematics problem. Skinner called such might sometimes qualify too. Small-scale conse- relations intraverbal (Skinner, 1957). They come quences following from listening or reading can in three varieties: through membership in com- accumulate over time; these are Skinner’s hun- mon thematic or semantic classes, as when hear- dreds of thousands of reinforcements. ing cow and barn makes us just a bit more likely We encounter many words groups in com- to say farm; through effects of verbal stimuli set- mon situations, so they may come to vary ting the occasion for specifi c verbal responses, together as members of verbal classes. Such as when we’re likely to say blue if someone says classes are said to be thematically or semanti- red, white and...; and through the temporal inte- cally related . For example, transportation words gration of verbal units into larger sequences, include car, bus, plane, train, as well as groups as when we say One picture is worth a thousand within groups, like vehicle, automobile, taxi, words or A word to the wise is suffi cient (additional sedan within the class car, and the classes can intraverbal behavior is involved in deriving the be further differentiated by makes and mod- implication that A picture to the wise is redun- els. The complex interlocking of such classes is dant). evident in literature and poetry, and many ver- bal games appeal to thematic classes (Botticelli, Wordsmith, MadLibs). Free association provides INTRAVERBALS: THEMATIC further examples of thematic relations. A verbal CLASSES response may be occasioned by combined the- matic and formal properties of a verbal stimulus. The intraverbal relations in any adult repertoire For example, the echoic heard-word rhyme may are the result of hundreds of thousands of rein- make a slight contribution toward my saying “I forcements under a great variety of inconsistent heard that word” instead of “I heard what you

215 216 • Part VII. Learning With Words: Verbal Behavior said.” Thematic and formal classes are ubiqui- The same applies to chronologies, geogra- tous, so the available examples are as vast as our phies and much of our everyday knowledge. No extended human verbal environments. one living saw Washington crossing the Dela- We might say thematic classes consist of ware. I might argue I saw the scene in a painting, words with common meanings, but to do so we’d but even if I’d recognized Washington, could I have to defi ne meanings. But meanings aren’t have known the setting was the Delaware River properties of words; they’re properties of our without a label telling me so? We rarely learn responses to them. For example, if you repeat a historical details by experiencing them. Instead, familiar word like your own name over and over, given names or dates, we learn to say when or in you may fi nd it loses its meaning; some response what order events occurred. “Most of the ‘facts’ to it drops out after several repetitions. A red of history are acquired and retained as intraver- traffi c light means stop and a green one means bal responses” (Skinner, 1957, p. 72). go, but when I stop on red and go on green, I’m In the chaining of intraverbal behavior, one responding to red and to green and not to their verbal stimulus sets the occasion for another meanings. Written words in a language no one verbal response. The relation between stimu- understands have no meaning, and when we say lus and response is arbitrary; there are no sys- a word has many meanings (as in bat in a belfry tematic correspondences between them. Free and bat at a baseball game), the meanings change association is an example (Galton, 1879). The only in the sense that we respond differently to immediate consequences of free associations the word in its different contexts. are usually minimal, and any given verbal In these examples, words function as both stimulus may occasion a variety of different stimuli and responses, so they can’t be identifi ed responses, so like the verbal summator the with just the response terms of operants or just the procedure is assumed to tap verbal responses stimulus terms of discriminated operants. Mean- of relatively high probability in the speaker’s ing is sometimes a convenient term for referring verbal repertory. You’re asked to report “any- to verbal classes functioning as either stimuli or thing that comes to mind,” and the word-asso- responses within verbal contingencies. ciation experiment reveals some of the relations between verbal responses and verbal stimuli created in your everyday listening and reading. INTRAVERBAL CHAINS …educational reinforcement sets up many We learn to recite the alphabet, to count, to give different intraverbal operants involving answers to arithmetic problems, to provide the cardinal numbers. Four is part of the readings of poems, to defi ne terms and to state occasion for fi ve in learning to count, for facts. The letter sequence in the alphabet is no six in learning to count by twos, for one in more orderly than the one on a computer key- learning the value of , and so on. On the board: some similar letters of the alphabet are other hand, many different verbal stimuli close together and others are widely separated come to control the response four, e.g., (e.g., M, N but D, T), some voiced consonants one, two, three... or two times two makes.... precede and some follow the voiceless (e.g., B, Many different connections between P but F, V), and so on. Despite its arbitrary char- verbal responses and verbal stimuli are acter, we learn the alphabet because so much established when different passages are is ordered according to it: dictionaries, tele- memorized and different “facts” acquired. phone books, indexes. We’re less able to recite The word-association experiment shows the order of letters on a computer keyboard the results. (Skinner, 1957, pp. 73–74) because we don’t have to behave with respect to keyboards the way we do with respect to alpha- Word associations were the basis for some betized lists. attempts to measure meaning (Galton 1879). If Chapater 23. Antecedents and Consequences of Words • 217 listeners produced more varied lists in response of Silence, assuming in each case the words to one word than another, the fi rst word was have become well learned. Some short verbal said to be more meaningful than the second. sequences can be learned very quickly, but when Words that occasioned overlapping lists of asso- they’re as extended as these they may take some ciates were assumed to be closely related. For time. At the beginning, completing the sequence example, common associations to both infant will depend a lot on chaining, perhaps in com- and baby might include crib and bottle but prob- bination with prompting or reference to a writ- ably neither would be responses to guitar. The ten copy. But even early on, some parts begin to different degrees of overlap among the associ- hang together. When I get as far as To be or not ates are consistent with what we already know: to I don’t need to consider what I’ve said so far infant and baby are closer in meaning than either before I continue with be. Whether completing of is to guitar. the line occasions That is the question will tell me In its simplest forms, intraverbal behavior whether this part of the soliloquy is still chained has been a major focus of research on human or I’ve begun to integrate it into a larger chunk. verbal learning, perhaps because of the ease Throughout my learning, the soliloquy will be with which verbal materials can be manipulated a mix, partly chained and partly chunked. And as stimuli. The classic experiments of Ebbing- even for a seasoned Shakespearean actor, much haus (1885/1964) were specifi cally concerned less me, the soliloquy may remain a segmented with the learning of arbitrary verbal combina- work where major boundaries separate chunks tions. Paired-associates learning (learning word that are chained together. Ay, there’s the rub. pairs) and serial learning (learning ordered lists, The case for chunks was extended from non- as in learning to count) represent basic cases of verbal to verbal behavior by Lashley (1951), who intraverbal behavior. Intraverbal relations are a described cases from language and music that component of standard educational practice, as made such accounts implausible. For example, when a child mastering the multiplication table in answering the argument that each movement gives 42 as the response to 6 x 7. But the response serves as a unique stimulus for the next, Lashley is strictly intraverbal only if it doesn’t depend on considered the complex sequence of movements other intervening arithmetic behavior, such as required to pronounce the sounds of the word adding six sevens, counting by sixes or counting “right” in proper order. The order isn’t given by the boxes in a 6-by-7 rectangle. the sounds themselves, because they can occur in a variety of orders and combinations (e.g., in the opposite order, as in “tire”). If the sound INTRAVERBAL CHUNKS sequence can’t be based on responses uniquely occasioned by stimuli it must depend on a larger It’s appropriate to speak of intraverbal chains organization. Lashley extended the case from only when successive parts of an utterance serve sequences of sounds within words to sequences as discriminative stimuli for later parts. When of words within sentences: extended utterances function as independent verbal units, as in rapidly emitted sequences, The word “right,” for example, is a noun, the language of chains doesn’t apply. By anal- adjective, adverb, and verb, and has four ogy with similar units in the analysis of remem- spellings and at least ten meanings. In such bering, they’re intraverbal chunks. Maxims like a sentence as “The millwright on my right “Haste makes waste” and “He who hesitates is thinks it right that some conventional rite lost,” best regarded as verbal units in their own should symbolize the right of every man right, are intraverbal chunks. to write as he pleases,” word arrangement Some examples of large intraverbal chunks is obviously not due to any direct associa- are Hamlet’s soliloquy, the Pledge of Allegiance, tions of the word “right” itself with other and the lyrics of Simon and Garfunkel’s Sounds words, but to meanings which are deter- 218 • Part VII. Learning With Words: Verbal Behavior

mined by some broader relations. (Lashley, may be hard to specify. The experimenter des- 1951, pp. 115–116) ignates responses as right and wrong. When the learner responds, the experimenter may say Lashley used music to point out that the “Right” or “Wrong” or the learner may just be sheer rapidity of some sequences constrains how shown the correct item. These procedures are they might be generated: said to provide feedback or knowledge of results. It’s tempting to assume telling or showing a The fi nger strokes of a musician may reach learner a response was correct is a reinforcer, sixteen per second in passages which call but feedback might function only as an instruc- for a defi nite and changing order of suc- tion: telling the learner a response was correct is cessive fi nger movements. The succession like saying, “Respond the same way next time.” of movements is too quick even for visual Thus, speaking of the reinforcing properties of reaction time. In rapid sight reading it is being correct and the punishing properties of impossible to read the individual notes being incorrect may be misleading. of an arpeggio. The notes must be seen in Verbal learning materials have ranged from groups…. (Lashley, 1951, p. 123) simple items like numbers and letters through nonsense syllables and words to such complex When a skilled typist rapidly types the, each materials as sentences and extended texts in letter can’t be a discriminative stimulus for the either written or vocal modes. Written stimuli next stroke, fi rst because the typist will be exe- have the advantage that they’re uniform, easily cuting the next stroke even before last typed described and can be presented at well-defi ned letter exists on the page, and second because rates and durations. An experimenter who as discriminative stimuli these letters can’t be reads a list to different learners might change unique if they can be followed by hitting the the infl ection or loudness of words over suc- space bar or any of many other keys depending cessive readings. Similarly, the choice of writ- on what the typist is typing (e.g., the or these or ten or vocal responses varies with experimental then or thermometer). requirements. Learners can usually respond more quickly vocally than in writing, but the Verbal Learning vocal response must be recorded whereas the written response is its own record. The area traditionally called verbal learning is Verbal-learning procedures experimentally concerned with what happens as we learn word realized the associationist principles developed sequences, word combinations and word con- by philosophers like David Hume and James texts. Its literature is extensive. Here we briefl y Mill, who advocated that human thought is examine four major classes of verbal learning: based on the association of ideas. Ideas were serial learning, paired-associates learning, free said to become connected or associated through recall, and verbal discrimination, including similarity or common elements or other dimen- verbal recognition as a special case. In typical sions, but especially through contiguity in space procedures, verbal stimuli are presented to the or time. Later, with the beginnings of modern learner and then verbal responses are recorded. chemistry, analogies were drawn between the We take a lot for granted in such arrangements. formation of associations and the chemical com- Describing them simply as verbal discrimina- binations of atoms into molecules. Hermann tive stimuli and differentiated verbal responses Ebbinghaus, a German investigator, in effect omits signifi cant features. For example, the the founder of research on remembering, used learner’s performance is rarely generated by dif- himself as the experimental subject. With mea- ferential consequences; instead, it’s usually cre- suring the formation of arbitrary associations as ated by giving instructions. The consequences his goal, he invented the nonsense syllable as an Chapater 23. Antecedents and Consequences of Words • 219 item that hadn’t acquired verbal functions and serial learning by an American researcher, Mary therefore wouldn’t be contaminated by existing Calkins (1894, 1896). Associations can sometimes associations (Ebbinghaus, 1885/1964). be learned on a single trial, but the properties of As its name implies, serial learning is the the association may be more important than how learning of a verbal sequence, as when a child quickly it can be formed. For example, is it sym- learns to count or recite the days of the week. metrical? Once the stimulus item consistently For example, suppose we try to teach some- occasions the response item, will the response one the fi rst ten presidents of the United States item occasion the stimulus item? The naming through serial recall. We show each name briefl y of written letters is a paired-associates task with and the learner tries to recite the entire list in written stimuli and vocal responses; if a child proper order only after all ten have appeared; who hasn’t yet started to write learns to say “A” we repeat the list until the learner gives one or when shown a written A, we wouldn’t expect the more correct repetitions. Once the list is learned, child to write an A in response to the spoken let- we might ask the learner questions about the ter (Polson et al., 1997). The issue has practical positions of names in the list, like “Who was the implications. In learning Russian, for example, ninth president?” (Harrison), “Which president learning to give the equivalent English word in was Tyler?” (tenth) or “Which president came response to a word in Russian doesn’t guarantee before Van Buren?” (Jackson). A well-learned you’ll be able to give the corresponding word in serial list, the alphabet, illustrates the roles of Russian when the English word is the stimulus context and position. We can recite the alphabet item. You’d be well-advised to learn the symme- and probably can quickly say which letter pre- try explicitly by studying the vocabulary in both cedes or follows any other, but without count- directions (e.g., English to Russian and Russian ing how easy is it to give the 11th letter or the to English). Such symmetries are prerequisites numerical position of S? for equivalence classes as well as the defi ning Paired-associates learning generates cor- characteristics of bilingual skill (Kolers, 1966). respondences between items on two lists, as In free recall, the learner is asked to name, in learning the dates of historical events or in any order, items presented earlier, as when the equivalents in one language of words in you’re asked to say what was on a misplaced another. Each verbal stimulus sets the occa- shopping list or what questions were on a test. sion for a verbal response. For example, if the In verbal learning studies, the list consists of ver- pairs to be learned are titles and authors, they bal items, but the procedure is like recall of non- might consist of: Don Quixote–Cervantes; Can- verbal stimuli, as when a witness names those dide–Voltaire; Moby Dick–Melville; Catch 22– present at the scene of a crime. Another com- Heller; The Odyssey–Homer; and so on. Each monplace example is when someone asks us to title appears alone for a few seconds. Whatever name the people we met at some event. the learner’s response, the author is then added. Verbal discrimination is simply discrimination The full set is repeated, usually with the order along some verbal dimension of stimuli, as when changed, until the learner correctly names each a child learns to distinguish vowels from conso- author before the name appears. Questions can nants or nouns from verbs or grammatical from then be asked about effects of changing items or ungrammatical sentences. Pairs of items are reversing stimuli and responses. Paired-associ- presented and the learner chooses one item of ates experiments, however, don’t typically use each pair by naming it, pointing to it or perhaps familiar items and pairings; they’re more likely pressing a left or right button. One special case is to use nonsense syllables or arbitrary combina- that between new and old items, as when a stu- tions. dent in a language course distinguishes words Paired-associates procedures were devel- already learned from new ones. This is verbal rec- oped independently of Ebbinghaus’s work on ognition. After an initial list, the learner is given 220 • Part VII. Learning With Words: Verbal Behavior another one with both new items and items The main difference between serial recall and from the fi rst list. The learner must identify the free recall is in the instructions to the learner. In items from the fi rst list. Except that it uses ver- serial recall, the learner is instructed to name the bal materials, verbal recognition is like everyday items in their original order; in free recall, the instances of recognition, as in recognizing places learner isn’t told order is important, or perhaps you’ve visited or friends in a crowd. is specifi cally instructed to ignore order. But this Each procedure seems superfi cially simple, doesn’t mean items freely recalled are recalled but while mastering any of these tasks, the in arbitrary orders. In fact, the order of items human learner inevitably does a lot more. The recalled usually differs systematically from the historical transition from serial and paired- original list order. associates experiments to those of free recall In the phenomenon called clustering, groups occurred in part to allow the learner to demon- of items occur together based on properties strate other things learned about lists besides the independent of their location within a list. For orders of specifi c items in them. These various example, if a list includes a mixture of color verbal-learning procedures provided baselines names, animal names and plant names along for studying factors infl uencing human learning. with words in other categories, the names within For example, the fi nding that spaced practice is each group will tend to be recalled together. In a typically more effective than massed practice phenomenon called intrusions, group items that (Underwood, 1961) is often cited in support of didn’t appear in the list are sometimes recalled distributing study evenly throughout a semester along with those that did appear. For example, instead of cramming it in at the end, just before if one word group on a list included the word exams. The fi nding is so familiar it’s surprising it wigwam and several other Native American was once regarded as counterintuitive. words but not the word tepee, some learners Data from serial learning and other proce- may report tepee even though it wasn’t on the dures typically include serial-position effects. list. The effect is sometimes so powerful that Learners are most likely to recall items at the even after feedback some learners remain con- beginning and at the end of lists; thereby illus- vinced the intruding word appeared on the list. trating two principles: primacy and recency. The As easily as this, learners can be led to say they principle of primacy states that the fi rst items of remember something that in fact hadn’t hap- a list are more likely to be recalled than later pened. ones; the principle of recency states that the most Verbal learning procedures usually repeat recent items, i.e., those at the end, are more likely a list until the learner meets some criterion of to be recalled than earlier ones. But these princi- correct responding. This is appropriate when ples don’t explain serial-position effects; they’re we’re interested in the relative diffi culty of dif- just names summarizing what learners do. The ferent types of lists. But when we’re interested important dimensions of free recall are in the in teaching, we can design lists so that, through learner and not the list. Learners behave with a progression of changes, they’ll effi ciently pro- respect to what they’re learning. For example, in duce a verbal discrimination. In other words, free recall learners often rehearse recent items by verbal discriminations are well suited to fading repeating them vocally or subvocally. Learners procedures. The following progression of items have more opportunities to rehearse early words is an illustration; the correct item of each pair is than later ones, so rehearsal may contribute to indicated by bold italics (adapted from Goldia- the primacy effect. But opportunities to rehearse mond, 1966): near the end of lists is limited, so it’s unlikely rehearsal is implicated in recency effects. And if A – B you followed the instruction to go through the B – D list quickly, you probably didn’t have time for B – V rehearsal anyway. O – B Chapater 23. Antecedents and Consequences of Words • 221

FS – BF or uh-huh can reinforce verbal classes such as JB – EL plural nouns or the substantive content of con- EAB – VOX versations (Greenspoon, 1955). One general ABE – JANE consequences of verbal behavior is that through MARY – BERT it a speaker changes the behavior of a listener. BILL – CONNIE Words are ways to get people to do things. DAISY MAE – LI’L ABNER The consequences that reinforce human ver- ROBERT – MARIA bal behavior are many and varied. Sometimes ANNA – BOB they’re nonverbal—someone comes when called; BOY – GIRL sometimes they’re verbal—someone answers a DANNY BOY – BETTY ANN question. Sometimes they’re highly reliable— BYRON – BETH Thank you is typically followed by You’re wel- BETSY – WILLIAM come; sometimes they’re not—not all requests DOROTHY – GEORGE are granted. Sometimes they’re very specifi c— KENNETH – ANTONIA answers to questions, taking appropriate action. JOHN – ELIZABETH But the shaping of verbal behavior, whether by natural or artifi cial contingencies, often involves Early in the list, discrimination is based only generalized reinforcers. Furthermore, the ten- on the letter B. At the transition from letters to dency to speak may depend on some conse- words, a gender difference is introduced, with quences while what get said depends on others. the male item always paired with the letter B. One diffi culty with experiments on verbal shap- Later B is included in both items and in sub- ing was they sometimes attempted to modify the sequent pairs B is removed so the correct item content of verbal behavior with the same conse- depends only on gender. The gender discrimi- quences that were supposed to keep the speaker nation in the fi nal pair also includes a reversal talking (Azrin et al., 1961; Greenspoon, 1955; of the letter discrimination created at the begin- Rosenfeld & Baer, 1970). Verbal behavior is main- ning. The fading from pictures to written items tained by varied consequences operating in the is an effective method for teaching children early context of interacting natural contingencies. They reading skills (Gleitman & Rozin, 1973). can be thought of as complex nested schedules of reinforcement. It’s no wonder we talk a lot.

THE CONSEQUENCES OF The Mand VERBAL OPERANTS: MANDING One obvious consequence of verbal behavior Verbal behavior has consequences. As with all is illustrated when we’re given something we operant behavior, these consequences affect asked for. If a child says milk and gets a glass of later verbal responding. In a simple two-person milk, we can say the milk is likely to reinforce conversation, each person provides an audience the verbal response. The response needn’t occur for the other. Audiences vary. We speak into in the presence of the reinforcer. For example, a phones, write messages or address groups of child may ask for milk even if milk isn’t pres- people. Often the consequences for the speaker ent. Verbal responses that specify their rein- are simply what listeners say later. It doesn’t forcers have been called mands (Skinner, 1957); take a laboratory experiment to show a listen- demands and commands, for example, specify er’s response can maintain a speaker’s talk. We what the listener is supposed to do. usually stop talking to people who don’t react An analogue from animal research may be to what we say. To this extent we can say the useful. Assume a rat in a chamber with one listener’s responses reinforce the speaker’s ver- lever producing food and another produc- bal behavior. Simple consequences such as yes ing water. If the rat presses the fi rst lever only 222 • Part VII. Learning With Words: Verbal Behavior when food-deprived and the second only when are specifi ed by the verbal responses naming water-deprived, we could argue the presses them in other circumstances; in other words, respectively are food requests and water manding is a higher-order verbal operant. This requests. Though it would be irrelevant to our woozle mand isn’t a name but functions like argument, we could make the analogy look one. No such class existed in the rat example. more convincing by arranging signs that lit up Within the class of mands, some subclasses when either lever was pressed, saying Please give specify stimuli—May I have an apple?—and some me food and Please give me water. The rat’s vocab- specify the listener’s behavior—Please wait for ulary is limited to two levers, but the relations me; others called questions specify the listener’s between the presses and their consequences are verbal behavior—What’s your name? or Did you just like those relating human verbal requests to see where I left my keys? These may be further their consequences. subdivided. For example, we speak of a prompt Yet this account isn’t quite satisfactory. Imag- when the appropriate verbal response is already ine a child who sees a new toy, learns it’s called a known to the speaker, as in giving a hint to a woozle and then asks for a woozle even though child who is unable to solve a riddle, and a probe asking for woozles could never have been rein- when it isn’t known, as in a police interroga- forced before. As a category of verbal behav- tion. In everyday discourse, we also distinguish ior, the mand can’t consist of many separate mands based on the contingencies they signal; response classes corresponding to each of the for example, bribes specify the consequences of many mandable consequences. Rather, it must compliance whereas threats specify the conse- be a single response class in which reinforcers quences of noncompliance. Chapter 24 Contact of Verbal Behavior with the Environment

Facts do not cease to exist because they are ignored. —Aldous Huxley

Verbal behavior would never have evolved had woman) and others are restricted to relatively it made contact only with other verbal behavior. narrow circumstances (e.g., your name or mine). At some point, it must contact environmental We can take the wealth of available tacts as a events. We speak of this contact as tacting; a tact remarkable feature of human language, but we is a verbal response occasioned by a discrimi- mustn’t let it obscure the simple relation defi n- native stimulus (Skinner, 1957). For example, if ing a tact: it’s precisely the same as the relation a child learns to say “apple” in the presence of between a stimulus and the response it occasions an apple, the child is said to tact the apple. The in a three-term contingency. tact doesn’t involve any new process; it’s just a It’s useful to consider how we might teach name for stimulus control as it enters into verbal tacts to a nonhuman organism; the exercise behavior. forces us to be explicit about their properties. Imagine a food-deprived pigeon in a chamber with a window on which we can project different TACTING colors. Three keys are next to the window. When it’s red, left-key pecks produce food; when it’s Superfi cially they may seem alike, but tacting blue, middle-key pecks do so; when it’s green, differs from naming much as textual behavior right-key pecks do so. The pigeon will eventu- differs from reading. Tacting is sometimes a ally peck left during red, middle during blue component of naming. Unlike naming, however, and right during green. We can call the pigeon’s it can only occur in the presence of the tacted performance the tacting of red, blue and green. stimulus. We can name an absent object but we The stimuli are highly specifi c, so we wouldn’t can’t tact it. We make the distinction because expect the pigeon to respond to red roses or fl ags our responses to past events aren’t determined or sunsets as it does to the red window. But the directly by those events; instead, they’re deter- generality or specifi city of the relevant stimuli mined indirectly by our previous behavior with isn’t a criterion for a tact relation. If we doubt respect to them. We’ll return to naming later. whether the pecking will generalize to red in An unlimited number of tacts is available other contexts, we can just call the pigeon’s left to mature speakers. We tact objects (chairs and peck a tact of this red window. tables, cellphones and books), living things What of the consequences of the pecks? The (fl owers and trees, birds and insects), weather pigeon would stop if it wasn’t food-deprived or conditions (rain and snow, sun and clouds), if food wasn’t a consequence. But even human activities (walking and running, working and tacting depends on consequences. We don’t playing) and innumerable other features of the go around tacting everything we see. Conse- environment. Some tacts are general (e.g., man, quences aren’t criteria for tacting. A tact may

223 Chapter 24. Contact of Verbal Behavior with the Environment • 229 buttressed, and stand or fall. On the other hand, accessible dimension, up-down, from raised systems of metaphors needn’t be consistent. For spirits and the heights of cloud nine to low- example, the metaphor of discussion as war may ered expectations and the depths of depression. only occasionally make contact with that of dis- Hardly any dimension can be more abstract cussion as exploration, in which issues are gone than time, but through metaphor it becomes a over in depth and at different levels and from differ- concrete spatial one: tomorrow versus yesterday ent approaches, the ground is covered, the speakers becomes front versus back. We’re so used to say- map out their territories and make direct or round- ing our past is behind us and our future is before about arguments (Lakoff & Johnson, 1980). us it’s hard to imagine taking an about-face so Metaphor is pervasive. Children learn it the future is in back and the past lies ahead; our readily and adults can’t ignore it (Winner, 1979). time line turns with us. Reaction times are often shorter for metaphori- Some similar extensions across cultures sug- cal than for literal usages (Foss, 1988). Metaphor gest common contingencies in the human verbal isn’t just the stuff of poetry; it’s a fundamental histories that shaped them. The word psychol- aspect of verbal behavior, an inevitable product ogy comes from the Greek psukhein, to breathe, of tacting in complex environments. If there isn’t and psukhe, breath. When the psyche was said a word for it, the verbal community invents one. to leave the body of a mortally wounded war- Much of our technical vocabulary evolved meta- rior in the Homeric epics, it was understood phorically from everyday sources, and much to mean the warrior’s breath and not his soul. of the everyday language applying to our own A parallel exists between this word and spirit, behavior arose through metaphorical extensions. from the Latin spirare, another word meaning Mind was once a verb (“Mind what I say!”), and to breathe, but no evidence links these Greek many esoteric cognitive or mentalist terms have and Latin words etymologically. A similar rela- mundane origins (Skinner, 1989b), as in compre- tion between air and spirit exists in the Latin hension from prehendere, Latin for to grasp. These anima, originally a breath of air but later soul or etymologies demonstrate concrete sources of spirit in words like animate and animosity, and our fundamental concepts. The creative aspect in the Greek atmos, vapor or air as entering into of metaphor, in other words, is in making the the English atmosphere but breath or soul in the abstract substantial, specifi c and solid or, to mix related Sanskrit atman. In all human cultures air more metaphors, in bringing it down to earth. is invisible but can be felt in breath and wind. Usages for mind continue to evolve. Although Air and breath are indispensable to life itself, it would once have been an unusual usage, some so it should be no surprise the corresponding online dictionaries now list brain as a synonym. words have so often been inextricably inter- But I fi nd it more interesting and more consis- twined. tent with a behavioral view of the world that We haven’t explicitly defi ned metaphor. some have moved in the other direction, persua- As in other cases, our failure to do so doesn’t sively arguing that instead of looking inward imply the term is meaningless. We don’t need any adequate concept of mind must include the to specify a stimulus to identify discriminative extended environment along with the organism relations, and we don’t need to specify verbal (Killeen, 2004; Noë, 2009; Rachlin, 2014). classes to identify tact relations between words The phenomenon of metaphor tempts us and events . We speak of classes of responses to talk about abstract properties captured by as operants and classes of stimuli as discrimi- words. Yet its most important feature is it allows nated operants or concepts. Words can function us to deal with the abstract in terms of the con- either as responses or as stimuli, so meaning can crete (Lakoff, 1987; Lakoff & Johnson, 1999). For work as shorthand for the relation of words and example, the language of abstract dimensions events within thematically related verbal classes like good-bad or happy-sad becomes a more (Lowenkron, 2004). 230 • Part VII. Learning With Words: Verbal Behavior THE LANGUAGE OF PRIVATE words from others and in teaching them to us EVENTS they had access only to the public correlates. If we can be mistaken even about the location of a toothache, what assurance do we have our other A crucial extension of the tact is to private reports of private events are reliable? Skinner events. Tacted stimuli are sometimes accessible (1963) makes the point by citing some students only to the speaker, as when we say we have a who watched a pigeon in a class demonstration headache. Such tacts depend on the verbal com- and then attributed what they saw mostly to the munity for their origins and maintenance. The pigeon’s expectations: problem is how the verbal community can cre- They were describing what they would have ate and maintain these responses without access expected, felt, and hoped for under similar cir- to the stimuli. A parent can teach a child color cumstances. But they were able to do so only names because the parent can see the colors the because a verbal community had brought rel- child sees and therefore can respond differen- evant terms under the control of certain stimuli, tially to the child’s correct and incorrect color and this had been done when the community naming. So many varied consequences follow had access only to the kinds of public informa- from color naming it usually doesn’t matter tion available to the students in the demon- whether the parent teaches the color names stration. Whatever the students knew about explicitly or simply allows them to be learned themselves which permitted them to infer com- through casual day-to-day interactions. parable events in the pigeon must have been With private events, however, the vocabu- learned from a verbal community which saw no lary can be taught only through extensions from more of their behavior than they had seen of the tacting events to which the verbal community pigeon’s. (Skinner, 1963, p. 955) has access. The child may learn to report pain Some time ago my colleague, Philip Hine- because the parents have access to overt manifes- line, collected students’ reports of their feelings tations such as an event that caused injury or the under various emotional circumstances. I saw child’s crying or facial expression; if the child has an opportunity to extend his results when snow learned the names of body parts, the two kinds of disrupted the exam schedule for a course. I’d verbal responses may be extended to the tact of told the students I wouldn’t give an exam on the pain in a specifi c place (Skinner, 1945b). fi rst day after a snow closure, but nevertheless I A toothache is a discriminable event, but arrived that day with papers and exam booklets someone with toothache has a different kind under my arm. As I distributed the papers face of access to it than the dentist called on to treat down I instructed the students not to turn them it. Both respond to the unsound tooth, but one over until everyone had copies. When I allowed feels it and the other looks at it and probes it with them to start, it read: “This is not an exam. instruments. They make different contact with Describe your feelings when you saw me start to the tooth just as a seeing person and a sightless hand out the exam booklets.” I asked them not person make different contact with a geometric to put their names on their papers, to preserve solid if one is teaching its name to the other; the their anonymity when I collected the papers and seeing person does so by sight and the sightless shared some passages with the class. one by touch. One kind of contact isn’t necessar- Despite the instructions, most of them ily more reliable than the other. For example, in described not feelings but the situation (“There referred pain a bad tooth in the lower jaw may be wasn’t supposed to be an exam today” or “I reported as a toothache in the upper jaw. In this wasn’t prepared for a pop quiz”) or wrote case, the dentist is a better judge than the patient. a question about the situation (“I wondered We probably think of private events like our what’s going on”). The reports of feelings were feelings and thoughts as ones to which we have almost exclusively limited to names (“I felt ner- privileged access. But we learned the relevant vous” or “Fear!” or “I felt anxious”). A very Chapter 24. Contact of Verbal Behavior with the Environment • 231 few reported physiological events (“My heart saying “I have a headache,” it isn’t clear whether raced”), and among those most were metaphori- the verbal response tacted some private event or cal (“My heart fell to the fl oor”). Had I asked just allowed the speaker to escape from unwanted again after a bit of discussion, I’m sure “Relief” company. On such grounds, some have argued would also have been high on the list. But the private events have no place in a science of behav- point is that when we talk about our feelings ior (Baum, 2011; Catania, 2011b). But like the very often what we say is more a commentary language of public events, that of private events about our environment than about something depends on the public practices of a verbal com- happening inside us. munity (Day, 1969; Skinner, 1945b; Wittgenstein, This cross-section of results came up over 1953). The implication is we know what’s public several classroom variations, even when the better than we know what’s private, because the instructions specifi cally encouraged descrip- verbal community can teach us the language of tions of feelings rather than descriptions of the public events more consistently than it can teach situation. But maybe no surprise, because few of us the language of private events. How did it teach us have been explicitly taught how to describe us to say we understand something, or we’re in our feelings or what goes on in our own bod- love, or we’re uncertain, or we’re happy or sad? ies. There are opportunities here. Given what we We know from examples like breast self-exami- know about teaching breast self-examination , nation that sometimes we need to be taught even shouldn’t we be doing better at teaching people about our own bodies, so shouldn’t we be simi- to detect symptoms preceding a heart attack and larly concerned about how we’re taught about our discriminating them from heartburn and sore own feelings and states of mind? muscles? How many other medical conditions Verbal behavior doesn’t always require exist for which patients might contribute to their stimuli to be simultaneously available to both own diagnoses by learning to discriminate prop- speaker and listener. Some important conse- erties of their own bodies? quences of verbal behavior occur when the Private events can include behavior as well speaker tacts an event unavailable to the lis- as physiology. Once we learn to tact properties tener. If I’m telling someone over the phone of the public behavior of others, we may come about something I’ve found at a website, the to tact the same properties of our own behav- other person can’t see what I’m seeing but may ior, whether public or not. If one person works be able to do something about it by switching hard at something with little compensation and to the same site I’m looking at. The relation another does so only with substantial compen- between tacting a public event and tacting a sation, we usually assume the task was more private one is the same as that between tact- important to the fi rst person than the second. ing when both speaker and listener have access The same observations of our own behavior may to what is tacted and tacting when only the lead us to say what’s more or less important to speaker has such access. The argument that dual us (Bem, 1967). What we say about our beliefs access is possible only in the website case can or about the causes of our actions may follow be addressed by recalling the different kinds of more directly from our discriminations of our access in our dental example. The language of own public behavior than from anything pri- private events has its own special diffi culties, vate (Kiesler, Nisbett, & Zanna, 1969). This isn’t but we needn’t invent new categories of verbal to deny private events. It’s instead a cautionary behavior to deal with it. note: the language of private events can distract Skinner (1945, pp. 131–133) suggested “at us from the public causes of behavior. least four ways in which a reinforcing commu- The main problem with tacting private events nity with no access to a private stimulus may is that the verbal community doesn’t have the nec- generate verbal behavior with respect to it.” The essary access to shape and maintain it. For exam- verbal community may differentially respond to ple, when someone leaves a social gathering on reports of private events based on (i) common 232 • Part VII. Learning With Words: Verbal Behavior

public accompaniments, (ii) collateral behav- verbal community can shape such vocabularies, ior, (iii) shared properties of public and private which sometimes appear as components of cog- events, as in metaphorical extension, and (iv) nitive accounts of behavior, we move toward generalization from public to private behavior an experimental analysis not only of the child’s along the dimension of response magnitude. verbal behavior but also of the verbal behavior For example, a child’s report of pain (i) may fol- of those who study it. Theories of mind are to be low a public event like a fall or a cut; (ii) it may found not in the heads of children but rather in the be accompanied by crying. Furthermore, (iii) behavior of the theorists proposing them. Mental a report may derive from properties of public vocabularies may not serve us well as guides objects producing certain kinds of pain, such as for experimental analyses of the verbal behav- sharp edges producing sharp pains; and, fi nally, ior of children, but when they appear in the (iv) a report of talking to oneself can generalize behavior of researchers we shouldn’t neglect from cases when the talking is overt to those them as potential objects of such analyses. when it’s no longer of public magnitude. These four categories give listeners a basis for differen- tially reinforcing verbal behavior occasioned by NAMING AS A HIGHER-ORDER private events. Skinner excluded only the prop- CLASS erty of privacy itself from these contingencies. Developmental accounts are often expressed What about naming? It may include tacting as in mental language, as when a child’s vocabu- a component, but our everyday vocabulary lary of remembering, forgetting and never hav- extends to so many cases any formal defi nition ing known is said to depend on the child’s theory will probably be unsatisfactory in one way or of mind (Wellman, 1990). Children usually learn another. Let’s again consider the pigeon whose remember before forget, and knew before never pecks tact red and green and blue lights. It’s a knew. They ordinarily learn through natural con- good bet you’d object to any argument suggest- tingencies in the context of being asked ques- ing the pigeon was naming those colors or the tions and answering them. To fi nd out more we different key pecks were the pigeon’s names can see what it takes to teach the words explicitly for the colors. And you’d be right to object. But as tacts. Suppose one day a child is taught tacts the reason isn’t that we do something different for some objects. The next day the child is asked when we name colors; instead, it’s that in nam- to tact those and some new ones. The child’s ing we do something like what the pigeon does tacting is sometimes successful and sometimes but we also do a lot more. not. Now we can differentially reinforce three In a few cases tacts are clearly very different different responses: given successful tacting, I from names. For example, hello is occasioned remembered; given an original object and unsuc- by special social situations: we’re introduced to cessful tacting, I forgot; and given a new object someone, or someone we haven’t seen recently and necessarily unsuccessful tacting, I don’t comes up to us, so we say hello. It’s diffi cult to know. Developmental programs that look only say exactly where to draw the line between situ- at when different verbal responses are usually ations that qualify and those that don’t, but these learned by children tell us little about the contin- sorts of situations occasion the hello as surely as gencies that shaped them. a dog occasions dog or a cat cat. We wouldn’t call As programs for teaching the tacts I remem- it a name, but hello qualifi es as a tact. Another ber, I forgot or I never knew, our procedures example is thank you. To the extent we say thank make the public correlates explicit. It would you only given certain social situations involving be misleading to seek the controlling variables someone having done something for you, thank for these reports inside the child, because they you too qualifi es as a tact, but here again we were based on public events available to the wouldn’t call it a name. Tacts, in other words, verbal community. By demonstrating how a are examples of stimulus control in which the Chapter 24. Contact of Verbal Behavior with the Environment • 233 response is verbal; they’re not defi ned by prop- Along with tacting, another prerequisite for erties of the stimuli occasioning them. Naming naming is echoic behavior. The child is otherwise involves something more. unlikely to produce a new word spontaneously. Naming is a higher-order class involving Echoic behavior depends on the replication of arbitrary stimulus classes (the things or events the phonetic components of verbal stimuli rather we name) and corresponding arbitrary verbal than of their acoustic properties, so they’re not topographies (the words serving as names); mere imitations. They require a verbal history their relationship is bidirectional, as when we (Camões-Costa, Erjavec, & Horne, 2011; Horne can name something seen or look for some- & Erjavec, 2007). But given the bidirectional- thing named. Tacting is at the heart of naming, ity, what about equivalence classes? They seem but tacting alone isn’t enough. Consider what prerequisites for naming and other higher-order a child must do before we’re satisfi ed the child verbal classes, but they may instead have nam- knows the name of something, like shoe as a ing as their prerequisites (Horne, Hughes, & name for one of those things you put on your Lowe, 2006; Horne, Lowe, & Harris, 2007; Horne, feet. Not only do we expect the child to say shoe Lowe, & Randle, 2004; Lowe et al., 2002; Lowe, when we hold up a shoe and say What’s this? We Horne, & Hughes, 2005; Randell & Remington, also expect the child to look around when we 1999) say Where’s the shoe? and then point to the shoe Naming has still another prerequisite besides if it’s in sight. tacting and echoic behavior. If a child already Superfi cially this looks like an equivalence does some naming but hasn’t yet learned the relation, or at least a symmetry relation, because name glove, we might hold one up and say This is the stimulus object and response word in the fi rst a glove; the child might say glove and point at it. A part of the example seem to switch places in the little later the child may point to the glove when second. But that’s assuming too much, because, we ask Where’s the glove? or say glove when we unlike a pigeon’s pecks on samples and com- hold it up and ask What’s this? The child may do parisons, a seen object can’t be exchanged with a this even never having answered these questions point at it and a heard word can’t be exchanged before with respect to gloves. Once the child has with a spoken one (Horne & Lowe, 1996, p. 234). come this far, it won’t be long before the child Furthermore, a point won’t be effective unless is rapidly acquiring new names: socks and hats, the child looks not at the caregiver but at where tables and chairs, cats and dogs, and so on. the caregiver is pointing. Thus, joint attention is The prerequisites for doing so include at least another critical prerequisite for naming (Dube et three components: (i) listener behavior, in looking al., 2004; Tomasello & Farrar, 1986). It’s a compo- for things and pointing based on what someone nent of the behavior of the listener. has said; (ii) echoic behavior, in repeating names Starting by about six months of age, before when they’re spoken; and (ii) tacting, in saying they’ve begun to tact, children begin to master the names given the objects (Horne & Lowe, listener behavior (Gurteen, Horne, & Erjavec, 1996). The child’s responses to the questions 2011). Between the ages of one and six years, about the glove are novel behavior and emerge they increase their vocabularies at an average as members of a higher-order class, naming. The rate of fi ve to eight words per day; by six, a various social and nonsocial consequences of child is likely to have a productive vocabulary naming are individually small, but collectively of thousands of words (K. R. Wagner, 1985). they sustain a lot of behavior. Soon the child Many function words (e.g., more, bye-bye, allgone) may initiate naming instead of waiting for it to are included along with common nouns (e.g., be initiated by others. It’s common for a child mama, milk, chair) in the early vocabulary. What who has learned some naming to turn the ques- common features must the learning of each new tioning around. On encountering something that word include to allow the child to learn so many doesn’t have a name yet, the child points and new words so quickly? asks What’s this? 234 • Part VII. Learning With Words: Verbal Behavior

Naming is generated from the ordinary (Kuczaj, 1977). Among young children, regular interactions between children and their caregiv- but nonstandard past-tense forms are common: ers. Once it’s available as a higher-order class goed, comed, breaked. But they don’t always start within a child’s repertory, it allows for expan- with regular forms. Often, the child fi rst learns sions of vocabulary in which the introduction the standard but irregular forms as individual of new words in functional relations (such as words: went, came, broke. After some standard tacting) involves those words in a range of other regular past-tense forms are learned, the irregu- emergent functions. These include but are by no lar forms are displaced by regular but nonstan- means limited to intraverbal behavior, echoic dard ones, even though the irregular ones were behavior and orienting toward or pointing at part of the child’s vocabulary for some time. named objects. We’re learning more about the Months or years later, the standard irregular verbal functions that constitute naming, the forms reappear and become permanent parts kinds of contingencies in the natural environ- of the child’s verbal behavior. The progression ments of children that shape and maintain those from standard irregular to nonstandard regular functions, and the role of naming in enlarging and back to standard irregular past-tense forms and amplifying other functions of verbal behav- is consistent with a progression from mastery of ior. All of this puts us in a better position to individual words through stereotyped interme- extend our analyses to applications, as in teach- diate classes to fl uent language. ing naming to children on the autism spectrum The acquisition of grammatical structure has (degli Espinosa, 2011). been central to debates over structural linguistic accounts and functional behavioral ones. Just as Language Development anatomical features of birds and bats determine the different ways they fl y, special characteris- Children who grow up in Italy speak Italian tics of our species may determine the structure and those who grow up in Brazil speak Por- of human language and how it develops. But tuguese. They learn the specifi c vocabularies even if it should be shown that human language and grammars of their native languages, and is limited in its structural properties or in how most become fl uent long before they begin its structure develops, the fi nding would have formal education. Although there are many little bearing on the functional questions raised descriptions of language development in chil- in the analysis of verbal behavior. An anatomical dren, there’s still much to learn about its criti- analogy is relevant: an account of the different cal factors (R. Brown, 1973; Moerk, 1992). Like properties of fl ight in birds and bats doesn’t bear vocabulary, grammatical skills progress rapidly on where or when they take off or land; so too between the ages of one and six years, as the an account of language structure doesn’t bear on child’s syntax moves from single-word utter- when we speak or what we talk about. ances to those of two words or more, sometimes If the signifi cant consequences of verbal described as roughly telegraphic (e.g., stove hot, behavior range from direct outcomes like get- daddy go car), and then to constructions more ting something we asked for to indirect ones like and more closely approximating the syntax hearing a comment bearing on something we of adult speech. Collecting data on language just said, these will all contribute to the shaping development in children poses problems of of verbal behavior. If these consequences include translation as well as other technical challenges. correspondences between sounds we’ve heard For example, a young child’s more probably has and sounds we’ve produced ourselves, as in the the sense of give me rather than of expressing a ontogenic shaping of echoic behavior, it’s not quantity (Moore & Frye, 1986). too great a leap to extend such correspondences Children must learn irregular constructions, from those phonetic properties to the semantic like go, come and break, as well as regular ones and syntactic properties of our verbal behavior. Chapter 25 Verbal Behavior Conditional on Verbal Behavior

Etymology is the archaeology of thought — B. F. Skinner, 1989

Verbal behavior, like any other event, can be to fi nish off the intraverbal sequence by saying tacted. No new kinds of relations are involved, blue. The verbal stimulus Red, white and... sets the but verbal behavior gets complicated when it’s occasion for the concluding word whether I say built on other verbal behavior. Verbal behav- it or you fi nish it for me. ior that depends on other verbal behavior and Skinner called a second category relational modifi es the effects of that other verbal behavior autoclitics. These involve verbal units that can’t on an audience is called autoclitic (Skinner, 1957). stand alone because they must be coordinated We rarely utter mands or tacts or other verbal with other verbal behavior. An example is the units by themselves, and even when we utter plural –s; we hardly ever say it all by itself in them in combination they seldom appear as a the presence of multiple objects or events. This simple succession of units. Instead, they’re artic- category doesn’t depend on discriminations of ulated with each other in complex ways. Skinner one’s own behavior. Instead, it’s based on the introduced two types of autoclitics to deal with ways multiple determinants of a verbal response these properties of extended verbal behavior. can come together, as when species and number The fi rst, a descriptive autoclitic, involves the and action and other properties combine so we tacting or discrimination of one’s own verbal say things like the shaggy dogs run or the little cat behavior. It tells the listener something about walks. Both autoclitic categories involve verbal the rest of what a speaker is saying, as when behavior conditional on other verbal behavior, we prefi x a comment with a qualifi er such as I but they operate in ways that make them funda- just saw that… or I’ve just been told that…. These mentally different. autoclitics modify the effect on the listener of the rest of the sentence by letting the listener know the source of what follows. They may also DESCRIPTIVE AUTOCLITICS: have instructional functions that operate on the DISCRIMINATING OUR OWN speaker as well as the listener (Hübner, Austin, VERBAL BEHAVIOR & Miguel, 2008). Intraverbals seem superfi cially like descrip- Many verbal responses tact the conditions under tive autoclitics but don’t qualify. They also which other verbal behavior is emitted and depend on and may modify the effects of other thereby modify the responses of the listener. verbal behavior, but they don’t require discrimi- Consider the phrases I doubt and I’m sure in “I nations of one’s own behavior. For example, if doubt the coffee is ready” and “I’m sure the cof- you say Red, white and… I don’t have to discrimi- fee is ready.” Each modifi es how the listener is nate anything about my own verbal behavior likely to act on the statement that the coffee is

235 240 • Part VII. Learning With Words: Verbal Behavior tional control from the language of tacting, if the details of verbal contingencies matter. especially in the interactions of the chimpanzees For example, judgments of adequacy of the dis- Austin and Sherman (Savage-Rumbaugh, Rum- crimination between requests such as Get juice baugh, & Boysen, 1978). The syntheses of com- from kitchen or Take juice to kitchen, or their ape plex interactions in such studies have special language equivalents, depend on contextual value because they force all of the assumptions features of the requests such as whether juice is about what counts as verbal to be made explicit always present when the requests are made; if it (Epstein, Lanza, & Skinner, 1980). You can’t tell a is, then the discrimination is probably based on pigeon or a chimpanzee what to do; instead, you the difference between get from and take to, but must shape all the components to be integrated if juice is present only given the request to take into the fi nal performance. it to the kitchen, then the discrimination may The performances of nonhuman organisms depend only on whether juice is present when a will become ever more sophisticated as human request has juice in it. investigators provide them with ever more Once some feature of human language was sophisticated environments. Differences like the demonstrated in the behavior of a chimpanzee or extent of operant control over the vocal appa- a pigeon or any other nonhuman organism, that ratus will remain; some will be more obvious feature could no longer be regarded as uniquely than others. Kanzi, an eight-year-old bonobo human. Too often attention then turned to the monkey (Pan paniscus), appeared capable of defi nition of language instead of the experimen- sentence comprehension beyond the reach of its tal analysis of its properties. Given these debates, relative, the chimpanzee (Pan troglodytes), and we can’t say whether chimpanzees are capable perhaps comparable to that of a two-year-old of language; the answer depends too much on human child (Savage-Rumbaugh et al., 1993). how we defi ne language. We certainly can say, We’ve already seen how verbal communities however, that their behavior includes many of are crucial in shaping and maintaining human the relations between stimuli and responses that verbal behavior, so we shouldn’t be surprised fall into our taxonomy of verbal behavior. Part VIII: WHEN VERBAL AND NON-VERBAL BEHAVIOR INTERACT Chapter 26 Verbal Governance

Never tell people how to do things. Tell them what to do, and they will surprise you with their ingenuity. — George S. Patton

The May 1950 issue of Astounding Science Fic- ics lost its appeal. Probably just as well. But tion featured a purportedly nonfi ction article by when I took Schoenfeld’s undergraduate course L. Ron Hubbard entitled “Dianetics: A new sci- and he talked about hypnosis as total control by ence of the mind.” It set out an account of pain- verbal stimuli, I was prepared to listen. One way ful engrams acquired early in life that disrupt to think about hypnosis is that it narrows the lis- us in the present; revisiting them can help make tener’s attention to the hypnotist’s voice; all else us “clear.” This was the kernel of what Hub- is excluded. Some people seem more susceptible bard later turned into his controversial religion, to hypnotism than others. Scientology, of Tom Cruise fame. I read it when To be functional, verbal behavior must do in high school and it seemed worth trying, so I things, but it can’t do things by itself. It’s effec- persuaded some friends to participate. We got tive only through the mediation of other people. together in my room, I had one of them lie down, Some nonhuman behavior may minimally qual- and I proceeded with what was essentially a ify, as when a horse is taught to turn in response hypnosis session. Following Dianetics guide- to a touch of the reins to its neck (the turns then lines, I fi rst had him relax and get sleepy. Then I reinforce the trainer’s behavior). The directional- sent him back to his childhood. After some prob- ity is like that for hypnotist and subject or for offi - ing he described walking down a path and then cers and those under their command, but in their jumping up on a wooden fence where a protrud- cases control is limited to certain domains. The ing nail cut his thigh. When I brought him back directionality makes the example of the horse he didn’t recall the event, but when he checked and trainer differ crucially from that of a child his leg it had a corresponding scar. Later his par- and an adult teacher. For the latter only, ver- ents confi rmed such an incident had really hap- bal contingencies soon become reciprocal, and pened. I thought I was on to something, but my these reciprocities defi ne verbal behavior. Verbal other friends decided against serving as subjects. behavior is maintained by the practices of verbal I hypnotized a couple of friends during high communities, and here we’ll consider some con- school and college, but for some reason Dianet- sequences following from these practices.

241 248 • Part VIII. When Verbal and Non-Verbal Behavior Interact

I could provide sample utterances and the time course of the shaping. But at least I discovered that the bit of attention I gave when he uttered those fi rst-person words was reinforcing enough to change his talk. I had heard the literature on verbal shaping was controversial, but my doubts vanished once I had him talking to me about him- self. And verbal shaping would come in handy later.

Effects of Shaped Verbal Behavior on Other Behavior

In one experiment (Catania, Matthews, & Shi- moff, 1982), students’ presses on two buttons Figure 26–1 An apparatus for studying human occasionally produced points exchangeable for behavior maintained by reinforcement schedules. money; a representative apparatus is shown in Left-button presses were reinforced according to VR Figure 26–1. When a light above the left button during yellow, and right according to VI during blue. lit, a random-ratio (RR) schedule operated for Details in text. pressing that button; when one above the right button lit, a random-interval (RI) schedule oper- Sometimes guessing press fast was accompanied ated for its presses. Between alternations of the by fast pressing and press slow by slow pressing, two schedules, during red, students fi lled out but sometimes the guesses were accompanied guess sheets, including sentences to be com- by equal rates on the two buttons or rates differ- pleted for the left and right button of the form ing in the opposite direction. the way to produce points with this button is Verbal shaping involves treating successive to…. Their guesses were shaped with differen- verbal responses as varying along semantic tial points worth money. In shaping guesses, dimensions, but judging which ones are closer an experimenter assigned points to each guess, or farther from the behavior to be shaped can be writing point values next to each one and pass- tricky. It’s easy to pick out words related to fast ing the sheet to the student through a partition. or slow and time or number, but a student who When the guess shaped for one button comes up with a guess like four fast then three slow was press quickly and for the other press slowly, might just vary those two numbers on all trials response rates on the buttons changed in cor- thereafter. The possibilities for varying the num- responding directions, without regard to sched- bers are unlimited, so the student may be caught ule contingencies. Thus, shaping press slowly in a verbal trap, so that our further attempts to for the left button and press quickly for the right shape simpler guesses are unsuccessful. Fur- button produced relatively low RR rates and thermore, it matters whether what we shape is a relatively high RI rates, opposite to those usu- description of behavior or a description of con- ally produced by these schedule contingencies. tingencies operating for the behavior. What the students said about their responding Figure 26–2 shows data from a variation on more powerfully determined what they did than these procedures (Catania, Shimoff, & Matthews, the consequences of their responding; students 1989). Again, an RR schedule operated for left- who slowed down on the RR button lost points button presses (L) and an RI for right-button they’d have earned if they’d responded faster. In presses (R). This time the guesses shaped were a procedure in which students were told what descriptions of contingencies instead of descrip- to guess, however, correspondences between tions of performances. The sentences to be com- guesses and response rates were inconsistent. pleted for left and right were of the form the Chapter 26. Verbal Governance • 249

Figure 26–2 Left (L) and right (R) rates over cycles of multiple random-interval (RR) random-ratio (RI) sched- ules of points delivered for button pressing. Shading shows points for verbal behavior (guesses) during verbal shaping. Unconnected points represent an interruption between sessions. Details in text. (Adapted from Catania, Shimoff , et al., 1989, Figure 1) computer will let your press produce a point depend- instead said that because points in both were ing on…. The guesses successfully shaped were unpredictable, point deliveries were unaffected variations of number of presses for the left button by presses. Responding showed rate differ- contingency and variations of time intervals for ences appropriate to the reports. In other words, the right button contingency. The progress of what you do in an environment may depend shaping, shown by the shaded areas, was com- on whether you describe what you do there or plete when guesses were earning the maximum instead just describe how it works without say- of eighteen points per guess period. The shaped ing what you might do. contingency descriptions were accurate, but they Verbal shaping was quickly effective when weren’t accompanied by substantial differences we switched from targeting contingency descrip- in the RR and RI rates of button pressing. tions to targeting performance descriptions Whether descriptions of contingencies pro- (dashed vertical line); it produced variations on duce corresponding changes in performance press fast for left RR button guesses and varia- depends on other verbal behavior. For example, tions on press slow for right RI button guesses. one student, correctly identifying two schedules Rates of button pressing diverged as perfor- as RR and RI (though not in technical language), mance guesses became more consistent over went on to say point deliveries increased with cycles. Unlike procedures that merely sample higher RR rates but not with higher RI rates. verbal reports, this one identifi es the direction Another, also correctly describing the schedules, of effect in the relation between verbal and non- 250 • Part VIII. When Verbal and Non-Verbal Behavior Interact verbal behavior, because we know which came fi rst. The verbal behavior changed during shap- ing; then came the change in response rates. In different circumstances, of course, the direction could go the other way, as when students whose pressing is fast or slow accurately describe their own behavior.

The Development of Correspondences

Correspondences between shaped verbal behavior and relevant nonverbal behavior come about because verbal communities reinforce consistencies relating what we say to what we do. Research with children shows these effects start early. An experiment with four- to six- year-olds arranged reinforcing consequences consisting of the successive lighting of lamps in an eight-lamp column; when the whole column was lit, the child earned a present, such as a box later exchangeable for pictures for the child’s scrapbook. When the top window showed a star, presses there worked according to a ran- dom-ratio (RR) schedule; when the bottom one showed a tree, presses there worked according Figure 26–3 Setup for verbal shaping with children. With a star projected on the top screen, presses to a random-interval (RI) schedule. As is typi- there are reinforced according to an RR schedule; cal for verbal humans, the schedules didn’t pro- with a tree projected on the bottom one, presses duce consistently different response rates even there are reinforced according to an RI schedule. at this age. The child occasionally talked with a This child is pressing the star on the top screen. Garfi eld hand puppet, who appeared from time Garfi eld, a hand puppet, at the left, can diff erentially to time through a small curtained aperture (Cat- react to the child’s talk. Details in text. (Adapted from ania, Lowe, & Horne, 1990). Figure 26–3 illus- Catania, Lowe, et al., 1990) trates the setup. After some sessions, Garfi eld appeared was therefore usually more effective to shape between schedule components and began talk- alternatives such as press a lot without stopping ing to the child: e.g., This game looks like fun. and press and wait. Can you teach me how it works? The reinforcers Performance was more variable than with used to shape what the child said about the per- adults, but successful shaping of verbal behav- formance including lights in the column and ior was typically accompanied by corresponding enthusiastic reactions from Garfi eld: e.g., That’s rates of pressing. As with adults, shaped verbal what I’ll try when I play the game. The child was behavior produced corresponding rate changes never told what to say or what to do, but the even when they were opposed to the usual verbal shaping procedure had to be adapted to effects of schedule contingencies, as when the each child’s vocabulary. Fast and slow typically child who came to say With the star you press and mean different things to children than to adult wait responded slowly during star even if the experimenters. For example, children will usu- schedule was RR. Figure 26–4 shows reversals ally say fast or slow for how quickly their hands of response rates that accompanied reversals of move rather than for how often they press. It one child’s verbal behavior. Chapter 27 Prejudice as Verbally Governed Discrimination

The very ink with which all history is written is merely fl uid prejudice. — Mark Twain

Before he sings “You have to be carefully taught” man. No surprise if you generalize from all your in the musical South Pacifi c, Cable says “It’s not past interactions with men and with women to born in you.” The show debuted in 1949; the this new instance. This is generalization of dis- song was controversial. It was protested in some criminatory behavior. places because it was said to justify interracial But discrimination isn’t in itself the problem. marriage and implicitly threaten the American For example, differential behavior based on gen- way of life. But saying it’s not born in you gets der has its origins in a long phylogenic history, it right. You do have to be taught “to hate all the as in the sexual selection of potential mates by people your relatives hate.” women. Should we then include this example as Whether it’s based on race, gender, ethnic- prejudice or as a member of an existing class? ity or any other dimension, discrimination in its Perhaps it’s true that men are more likely than societal sense is a product of interacting with oth- women to behave violently. If so, does it matter ers. If your behavior has different consequences how you learned it? depending on the reactions of those you inter- The problem parallels that of defi ning oper- act with, you’ll learn to discriminate based on ant classes. I tend to favor the language of those reactions. Being taught is something else. contingency-shaped versus verbally governed If you’ve never met any members of some group discrimination over discrimination versus prej- and social contingencies shape your verbal udice, perhaps because it fi nesses the question. behavior, you may behave differently toward the As a working solution, I’ll treat all prejudice as fi rst ones you meet, without ever having inter- verbally governed discrimination but with the acted with any others like them. Your behavior proviso that, based on different histories, not will be based only on what you’ve heard or what all verbally governed discriminations qualify you fi nd yourself saying. When that happens, the as prejudices. That’s because the term prejudice outcome may look like discrimination, but it’s a is usually regarded as a pejorative. It’s usually different kind. We must distinguish between risky to introduce colloquial terms into technical contingency-shaped discrimination, shaped by dif- usages. The issues are complex, and at this stage ferential consequences, and verbally governed of our analysis maybe clarifying the contingen- discrimination, which sometimes qualifi es as prej- cies is all we should hope for. udice. Verbal behavior can create prejudgment. It But what do we say when you’ve learned may lead you to behave differentially toward to discriminate based on some dimension, like others though you haven’t participated in non- gender, and then you continue the practice on verbal contingencies involving them. Push it far encountering someone you’ve never met before? enough and you might be ready to favor some Maybe when you meet a woman for the fi rst time children over others in the classroom or to pass you treat her differently than if she had been a over some people for advancement in the work-

254 Chapter 27. Prejudice as Verbally Governed Discrimination • 255 place or to incite violence that targets only those materials can enhance acquisition (Fields et al., in certain groups. It can get more complicated, of 2012), but they can do the opposite if the rela- course, when discrimination and prejudice com- tions run up against incompatible classes that bine as multiple causes in specifi c instances. But are already well-established. In the Catholic and that doesn’t mean the distinction can be ignored. Protestant communities of Northern Ireland If discrimination and prejudice arise in different these stimuli were familiar and already seen as ways, they must be dealt with differently when belonging to one or another of two incompat- they create problems. ible classes. Existing classes established over an extensive social history may be diffi cult to override with the relatively weak contingencies BUILDING UP AND BREAKING available in a laboratory. DOWN EQUIVALENCES To the extent such classes are created by nonverbal contingencies, they illustrate con- The Catholic Nationalists and Protestant Union- tingency-shaped discrimination. To the extent ists of Northern Ireland differ in their preferred they’re created by verbal environments, they colors, names, symbols and sports. Through the illustrate verbally governed discrimination. inevitable contingencies and contiguities in their Either way, they make the point that we mustn’t environments, these elements will have come limit our attention to what it takes to create together in their respective stimulus classes, equivalence classes; it’s also important to know one made up of the Catholic stimuli and the what it takes to break them down. Equivalence other made up of the Protestant ones. Equiva- relations can sometimes change rapidly, as lence-class procedures can be used to probe the when you discover the players or management properties of these two classes (McGlinchey & of a sports team for which you’ve been rooting Keenan, 1997). Catholic and Protestant learners have behaved unethically or have taken a politi- easily mastered new equivalence classes involv- cal position with which you strongly disagree. ing only neutral stimuli but had trouble master- One way to break down boundaries is to ing new ones in which Protestant and Catholic mix together groups that would otherwise have stimuli were linked through a neutral node. For remained separate, so they’re more likely to example, if C in a CN match for one class con- learn about each other through both verbal and sisted of a Catholic stimulus such as Irish green nonverbal contingencies, as when President and P in an NP match consisted of a Protestant Truman desegregated the military after World stimulus such as an image of marching Orange- War II. Another is by introducing new classes men, so the class always included both Protes- incompatible with the old ones, as in creating tant and Catholic stimuli, acquisition was slow schoolroom working groups that cross racial or if successful at all; learners typically failed CP ethnic or other lines, or as in representing diver- transitivity tests, even though both C and P had sity in the casting of dramas and comedies and been linked to the neutral N. These effects were even advertisements. stronger in older than in younger children and But either route can produce fragile out- were often locked in by age eleven or so. These comes. Existing prejudices can enhance rather outcomes were somewhat less likely among than reduce divisions across ethnic or racial or learners who attended universities with a sub- religious lines when groups come together, espe- stantial mix of Catholics and Protestants in the cially if the prejudices lead to confl ict. Desegre- student body. gation can be rolled back and pronouncements Equivalence classes are typically established in the media can support boundaries instead of in the laboratory with arbitrary stimuli, such as helping to break them down. Geography and visual shapes or nonsense syllables. Meaningful economics can lead to de facto segregation. 256 • Part VIII. When Verbal and Non-Verbal Behavior Interact

Even relatively benign preferences can create when continuous changes in the acoustic for- problems. For example, those who try to create mants that determine whether a phoneme is a p integrated neighborhoods must contend with or a b are heard as one or the other, with those in the fi nding that when people move into a neigh- the region of the abrupt transition from one to the borhood, a preference for having at least one other rarely if ever reported as blends of the two. or two neighbors matching your own racial or Racial differences are often discussed in the ethnic identity usually leads to segregated areas language of black and white. Skin color varies (Schelling, 1971; Tauber, 1965). This happens within both African and Caucasian popula- even among those actively seeking diversity. tions; not only do the distributions overlap, but Unintended consequences hide around every they’ve been blended together by a complex corner and integration calls for careful planning. history of miscegenation and interracial mar- It’s probably important to note that we’re riage. There was a time when it was widely held not here concerned with reasoned judgments. A that even one drop of black blood was enough white author wrote of a black home intruder who to make someone black; some still hold to that intimidated her but left without harming her view. Based on the assertion that one drop made or her infant son (Seiffert, 2004). She felt guilty so much difference, Malcolm X said “That must (“the pangs of her liberal conscience”) because be mighty powerful blood!” (Etzioni, 2003, p. 30). thereafter she felt unable to respond to young Verbal dividing lines can’t be justifi ed when pop- black men without prejudice. She was victim ulations vary continuously along dimensions of the natural consequences of nonverbal con- like skin color. Once populations have blended tingencies. Consciousness raising can’t remove and overlapped, there’s no good way to decide effects like these, though it may ameliorate where the dividing line should be. Subdividing them. Profi ling invites similar concerns, because populations into even more categories, as when it too may pit natural contingencies against ver- some are singled out as mulattos, only creates bal governance. Pulling over some cars but not added complications. Racial identifi cations and others based on the race of the driver is inap- race itself are verbal categories. The more we propriate if done to fulfi ll some racial quota. But learn about the human genome and our shared if the surveillance video of a crime scene clearly phylogeny, the clearer it becomes that they are displays the race of the driver of a getaway car, verbal constructs rather than biological entities. it would be counterproductive not to disclose It remains to be seen whether racial lines will the information to those in pursuit. If both count become further blurred with increases in the as profi ling, the implication is that profi ling is number of interracial families. But even if that sometimes fully justifi ed, and the problem is happens, racial differences will sometimes be drawing the line between what’s justifi ed and correlated with economic or class or ethnic or reli- what isn’t. But then what should the criteria be gious ones, so those who fi nd it useful to exploit for where its drawn? differences will no doubt still seek opportunities to create prejudice (cf. Coates, 2015). Differences can become further exaggerated when self-main- FROM DICHOTOMIES TO taining social groups like megachurches and CONTINUA: RACE AND fringe political parties reduce their interactions GENDER with others. The effects of verbal shaping are especially likely to be amplifi ed within groups of Naming itself is a verbal category that can con- individuals who mainly talk to each other. tribute to the creation of boundaries separating Other dimensions along which verbal bound- groups. On examination, many dichotomous aries can override continuities are gender and verbal classifi cations break down continua into sexual identity. Individuals can vary in anatomy, two or more discrete categories. This is analo- hormonal balance, cultural history and other gous to categorical perception of consonants, as sexual characteristics. Distinctions have been Chapter 27. Prejudice as Verbally Governed Discrimination • 257 drawn between lesbians, gays, bisexuals and differences among my black coworkers. I think transgender individuals, and between those and that mattered especially because discrimina- others sometimes called straight. But the origins tions within a racial or ethnic class are likely of such different categories may rest in part with to correspond with those already established cultural practices contributing to designations outside the class, and can therefore create new of gender. For example, long-standing obstetric equivalences cutting across existing lines. This traditions have affected the treatment of infants person is easy to get along with but that one born as hermaphrodites or with ambiguous isn’t; this person is outgoing but that one’s genitalia (Dreger, 2015). Obstetricians who have reserved; this person likes puns but that one delivered these infants have too often resolved doesn’t. Once I was discriminating along those the ambiguities surgically, sometimes without other dimensions, discrimination based on race even informing the parents. An infant may then became irrelevant. You can’t get to know things mature with the hormonal balance of a male and like this about people and still say “they all look the genitalia of a female, or vice versa. If con- alike to me.” tinua of anatomy and of hormonal balance are The same kinds of arguments can be made compromised, especially for those in the middle with respect to another class of classes, as of one or another of these continua instead of when in American culture we talk about peo- near the more culturally accepted male or female ple as being in the lower or the upper or the extremes, then different combinations can yield middle class. To the extent that people in dif- different sexual identities. ferent socioeconomic classes tend to mix more Though it’s diffi cult to justify where lines with those in their own class than with those in should be drawn, here again it makes sense to others, much happens at the level of verbally acknowledge the full spectrum of gender poten- governed discrimination rather than discrimi- tial instead of arbitrarily cutting it up with prej- nation grounded in personal interactions. Here udicial labeling. But dichotomies can also be again we see continua arbitrarily divided, in locked in by environmental constraints, such as this case in a trichotomy, or, on other grounds, gendered bathroom facilities. Unisex bathrooms as in determining poverty levels by income are perhaps one step toward a solution, but the rather than access to resources. It’s diffi cult prejudices originate in verbal behavior and can’t to see how these divisions can be addressed if be addressed adequately merely by such verbal they aren’t accommodated by educational insti- means as writing laws about them. tutions. For some, those institutions have been the only route for escaping from the constraints Discriminating Within and Between of class; whether they continue to play that role Classes remains to be seen. Probably H. G. Wells was right: “Human history becomes more and more When I worked summers in Harlem Hospi- a race between education and catastrophe” tal during my student days, the people there (Wells, 1920, Chapter 41) gave me lots of support as I learned how to do my job. The work of most exceeded what was required of them. A few worked exactly to the ATTITUDES, INTENTIONS, AND limits of what their job description called for ATTRIBUTIONS and no further, and a few cut corners whenever possible. There was also a small cohort of those When we talk about other people, we sometimes who resented my presence and found ways to describe what they do by citing their attitudes let me know. But though I rarely use the adjec- or intentions. A child might be said to have a tive, the behavior of nurses and other medical respectful attitude or a confrontational one; we staff was often awesome. I can say in retrospect might call one person laid back and another that I was learning discriminations based on uptight. We might say we didn’t mean to do Chapter 28 Verbal Function: Coordinations among Classes

Unless social interaction is to break down, the lie must always be the exception. —Bolinger, 1973

In its full complexity, verbal behavior involves behavior is what’s occasioned by these stimuli. interactions among different processes. Just as The listener’s responses to verbal stimuli can be a taxonomy of basis processes is required for as varied as the responses to any other events. analyzing nonverbal behavior, we need one Some nonverbal responses occasioned by verbal for verbal behavior. It includes classes different stimuli are obvious enough they don’t require from those in the everyday vocabulary. Textual special consideration. Whether the critical stim- behavior isn’t equivalent to reading, though it ulus is a red light, a traffi c offi cer’s outstretched may be its precursor. Transcription isn’t equiva- hand, the word stop or a tree fallen across the lent to pictorial copying, but depends on estab- road, stepping on the brakes illustrates stimulus lished units of written verbal behavior. Tacting control. There’s no need to distinguish among isn’t equivalent to naming or referring, and yet such stimuli even though some are products of as stimulus control of verbal behavior it’s where verbal behavior and others aren’t. verbal behavior is anchored to environments. Listeners aren’t passive; they often behave The effectiveness of verbal behavior depends on verbally along with the speaker, saying things to the coordinations of these and other elementary themselves, planning replies and so on. Some- components. By themselves they aren’t even times we act as our own listeners or readers, as particularly verbal, but in fundamental ways when we carefully attend to what we’re saying our verbal behavior is built on them. or think aloud or read something we’ve just Verbal contingencies extend how we act on written. Products of our own behavior can serve our environments. Verbal behavior is typically as discriminative stimuli and occasion our own shaped by social contingencies, but nonsocial further behavior (Skinner, 1989a). contingencies can also be effective, as when an As with nonverbal stimuli, not all responses engineer’s calculations lead to changes in the to verbal stimuli are operant. For example, if specifi cations of a project. Such cases clearly a spoken word is paired with a stimulus that depend on an extensive social history of behaving elicits autonomic responses (e.g., shock), the verbally, but interactions between writing and word may come itself to elicit these responses. reading are functionally similar whether writer This phenomenon, sometimes called semantic and reader are the same or different individuals. conditioning (Riess, 1946), is a verbal equiva- lent of the respondent conditioning of nonver- bal responses. Responding generated by these THE LISTENER’S OR READER’S procedures generalizes across semantic as well BEHAVIOR as phonological dimensions of verbal stimuli. For example, if electric shock is paired with a The speaker’s verbal behavior provides dis- vehicle word, such as truck, the conditioned gal- criminative stimuli for the listener; the listener’s vanic skin response is more likely to generalize

263 264 • Part VIII. When Verbal and Non-Verbal Behavior Interact to other vehicle words, such as car or bus, than and responses, but rather by the integrity of dif- to words simply having some letters in common ferent sorts of relations. We say someone under- with the original word, such as duck. Semantic stands something said when the person repeats conditioning may contribute to instructional what has been said not because another person effects such as a child’s adherence to warnings said it but for the same reasons the other per- about a hot stove. son said it (Skinner, 1968, p. 139). Such behavior Multiple causes may confl ict with rather than implies consistent relations among verbal and support each other. If words are printed in dif- nonverbal responses. ferent colors, it’s diffi cult to tact the colors rap- idly if the words themselves are incompatible color names, as when the word red is printed in FROM ACTION TO ACTING green (Stroop, 1935)); we read words and don’t TO LITERATURE: CREATING ordinarily attend to physical properties such as WORLDS the color in which they’re printed. This relation between stimuli and the verbal responses they We spend time telling stories to each other and occasion clarifi es some logical paradoxes of the much teaching has been devoted to what makes language of reference. For example, consider a narrative work, as in creative writing courses. the statement, This statement is false. If it’s true, Some issues are structural. Stories start out with a then it must be false; if it’s false, then it must be setting, but then something happens, and fi nally true. Clearly it can’t be true and false at the same things are resolved; this has been described as time. This is a paradox of logic but not of verbal an “and—but—then” or “and—but—therefore” behavior. One verbal response can tact another template (Olson, 2015). It has some resemblance verbal response, but it can’t tact itself. In other to our three-term ABC contingency. But what is words, the statement is not a response to itself, it about good narrative structure that keeps the but It can’t be true and false at the same time might reader or listener attentive throughout a story? be a response to it. This is a question about behavior and function In dealing with the formal relations, we (Hineline, 2016). We have in this paragraph an argued that the symmetry of stimulus and “and” and a “but,” but we’re still some way from response relations favors a vocabulary words fi lling in the “then” or “therefore” (cf. Hineline, in general rather than in terms of words in spe- 2016). cifi c vocal or written modalities. Similar corre- As we move from watching an actual inci- spondences exist in relations between tacts and dent to watching it acted in a play or a fi lm and environmental events. These correspondences then to reading the script for the actual incident may be important when we speak of meaning, and then to reading a description of it in a story, because the language of meaning is indepen- the common feature holding the cases together dent of whether words function as stimuli or as must lie in consistencies of stimulus control over responses. This may be the most important way verbal and nonverbal behavior. We can under- in which equivalence classes enter into verbal stand at least some of what happens when we behavior. watch a play or fi lm or television drama, because When a listener repeats what a speaker has in those cases we’re responding to stimuli with said and we say the listener understood the a reasonable resemblance to what we’d see and speaker, we aren’t usually satisfi ed in calling the hear if we were observing similar actual events. relation echoic. The several relations between Part of the history of the movies is its progress rain as a verbal stimulus and rain as a verbal toward ever more realistic presentations, in the response are among the criteria for talking about transitions from silent fi lms to sound, from black naming, but more is involved here than naming. and white to color, from fl at to 3D, and on ever Presumably we judge understanding or mean- larger screens. But those presentations are pro- ing not by any single relation between stimuli duced for us. What happens when instead we Part X: CONCLUSION Chapter 31 Applied Behavior Analysis

There are no such things as applied sciences, only applications of science. — Louis Pasteur

When I started out as a student in the branch of patients. Goldiamond designed treatments for experimental psychology that became behavior stuttering and tics (Flanagan et al., 1958); Bijou analysis, it seemed obvious the basics we were (1966) and Bijou and Baer (1961) demonstrated learning would sooner or later have real-world the feasibility of applications for children from applications. But in retrospect plenty was going both normally developing and challenged popu- on even then. Skinner (1968) was developing lations. Gilbert (1978) set the stage for behavior teaching machines and soon Keller (1968) was analysis in the workplace. contemplating a system called the Personalized These were human applications, but mean- System of Instruction. The fi eld became more while Brady (2007) trained chimpanzees so their specialized, so what began as its only journal behavior could be monitored while they safely in 1958, the Journal of the Experimental Analysis rode the fi rst suborbital fl ights in preparation of Behavior (JEAB), spun off its companion, the for human ventures into space, and Verhave Journal of Applied Behavior Analysis (JABA), in (1966) produced a system in which pigeons out- 1968. Until then, JEAB published both basic and performed humans in maintaining quality con- applied research and large proportion of con- trol in an assembly line. Animal applications tributors published in both areas, which were now include clicker training for pets (Pryor, regarded as parts of a single fi eld. 1999), dogs who detect drugs or explosives in Ferster worked with children, then called airports (M. Williams & Johnston, 2002), and psychotic, showing their behavior could be pouched rats who fi nd unexploded landmines modifi ed by simple consequences (Ferster & (Poling et al., 2011), among many others. Behav- DeMyer, 1961); bringing them out of their shells ioral medicine has been extended to breast self- to make contact with their environments laid the examination (Pennypacker & Iwata, 1990), and groundwork for behavior analytic services for techniques for highlighting where a radiologist’s children on the autism spectrum. Lindsley (1956, eyes linger longest hold promise for enhanc- 1992) worked with psychiatric patients and ing the reading of diagnostic images (Kundel, developed celeration charts to track Precision Nodine, & Krupinski, 1990). Teaching. Dews (1958) explored the sensitivity Charlie Ferster once summed up behavior to drugs of behavior maintained by reinforce- analysis by saying we solve problems by chang- ment schedules and in so doing paved the way ing the environment. Skinner managed his own for research in psychopharmacology and behav- environments in many ways, building a better ioral toxicology. Ayllon and Azrin (1968) devel- crib for his younger daughter, arranging a space oped token economies for use with psychiatric devoted exclusively to his scholarly writing, and

295 296 • Part X. Conclusion dealing with problems of aging (Skinner, 1945a, THE AUTISM SPECTRUM AND 1981a, 1983, 1999; Skinner & Vaughan, 1983). OTHER DEVELOPMENTAL Others added to personal applications, espe- cially regarding the stimulus control exerted by CHALLENGES consistent environments. For example, attend to and reinforce the behavior you value rather than I begin with a whirlwind tour of the area of taking a partner for granted; go somewhere else application that continues to dominate applied to discuss family issues or politics rather than behavior analysis (ABA) both among its prac- letting disputes intrude into the bedroom; and titioners and on the international stage. Defi - (still pretty good advice) do unto others as you nitions of autism and medical insights into its would have others do unto you. sources remain in fl ux and treatment fads come A further step is to record your own behav- and go. Vaccination is not the culprit, but envi- ior and to study the variables that affect it. In ronmental pollutants and bacterial loadings some courses I’ve had students conduct self- have not been written off as suspects (e.g., Yong, experiments along the lines recommended by 2016). The treatment called ABA is data-based Neuringer (1981). Students who looked at study and surpasses other approaches in its outcomes. data like pages read often found their assump- It therefore has advantages in obtaining recog- tions about their own study practices were nition by some third-party payers of medical wrong. A student who thought herself a night benefi ts. The prevention or cure of autism won’t person discovered she worked more effi ciently make ABA therapies obsolete, because a large in the morning. A student who thought he did population would still need remediation of the better with music in the background discov- behavioral complications that arise throughout ered he worked more effi ciently in silence, but the development of children who fall along the another whose music masked noisy neighbors autism spectrum. And even if autism had never did better with the music on. I was dubious existed, ABA treatment would remain appropri- whether it was worth the trouble for one stu- ate for early intervention in other kinds of intel- dent to collect data on how much she studied lectual challenge, such as Downs Syndrome, and when she worked alone and how much when for coping with self-injury and other problem- her husband was around. Of course she got atic behavior. less done when he was there, but the payoff The natural consequences of everyday inter- came when she shared her data with him and actions between parent and child build social he resolved to work at interrupting her study behavior. If they’re missing because the child less often. Instead of making assumptions about rarely makes eye contact and shies away when your own behavior, it often helps to check it out touched, the scaffolding for more sophisticated by collecting data. behavior won’t be in place. The child who Some of the early promise of behavior analy- doesn’t make eye contact won’t see the smiles sis is being fulfi lled. What was speculation has that ubiquitously work as reinforcers for the become real. I hesitate to stop at this point. Even behavior of other children; the behavior of the my incomplete roster of what was there from the child for whom touch is aversive won’t be rein- formative days suggests no single chapter can forced by hugs. So much else is built on these do justice to the range of its applications (Fisher, reinforcers that even if their absence is the only Piazza, & Roane, 2011). Some topics were defi cit for a child on the autism spectrum, their discussed in other chapters: e.g., breast self- cascading effects on the child’s developmental examination; functional analysis of self-injury; trajectory accumulate over time, especially in treatment of aggression in dogs. What follows the evolution of social interactions. What would here is merely an idiosyncratic sampling. otherwise have been shaped by natural contin- Chapter 31. Applied Behavior Analysis • 297 gencies must then be created by artifi cial ones, the toy as a reinforcing consequence would work like reinforcing eye contact with edibles. Then on the entire sequence only if the child found it comes the monumental task of teaching hierar- quickly. This is differential reinforcement: given chies of missing skills while also teaching par- rapid responding, the reinforcer is effective as ents and other caregivers how to maintain new far back as the start of the sequence; given slow behavior once it’s been created through shap- responding it’s effective only for responses near ing. Along the way disruptive behavior must the end. Thus, faster sequences will be selected also be managed. The work is daunting, but it’s by their consequences. This strengthening of well repaid when it’s managed well enough that rapid responding takes time, which may explain children starting out with multiple behavior why hyperactivity often takes a while to develop problems grow to be high-functioning young and can develop separately in different environ- adults. ments. In any case, hyperactivity is an appropriate name for behavior in which rapid sequences ATTENTION DEFICIT have displaced more leisurely ones. Of course, HYPERACTIVITY DISORDER if an ADHD delay gradient declines steeply AND THE DELAY GRADIENT enough that only a single response can ever be within reach of the reinforcer, rapid sequences Children diagnosed with attention-defi cit hyper- can’t be differentially reinforced, so hyperactiv- activity disorder (ADHD) show one or more of ity won’t emerge. The point is that gradients dif- a spectrum of behavioral symptoms (Sagvolden fering in steepness can produce different degrees et al., 2005). They may be hyperactive in school of hyperactivity. or home settings; they may be inattentive to The onset of a discriminative stimulus may instructions or school materials; and they may be followed by a response-produced reinforcer behave impulsively in situations where other after a shorter or a longer delay. With reinforcers children exhibit self-control. Some children delivered in its presence the stimulus becomes a show all these symptoms while others show conditional reinforcer, but its potency depends only one or another. Each may be understood as on this delay. If the gradient is steep, stimulus derived from delay-of-reinforcement gradients onsets followed only a long time later by rein- that decrease more steeply than ordinarily ones, forcers may be weak conditional reinforcers, so so reinforcers that don’t follow responses closely they’ll be less likely to be looked at or listened have less impact on the behavior of the ADHD to. Attention defi cit is an appropriate name for child than would otherwise be the case. what’s missing when a child doesn’t much look Delay-of-reinforcement gradients describe or listen. Its magnitude and the balance between how the effects of reinforcers change as a func- hyperactivity and attention defi cit may depend tion of the time separating them from preceding on the steepness of the ADHD gradients. responses. If an average gradient in some con- The child for whom the delay gradient is text declines slowly enough over time that rein- unusually steep will be less able to deal with or forcers are still somewhat effective even if they tolerate longer delays, which is perhaps why the follow ten or twelve seconds after a response, an attention of such children is so easily captured by ADHD gradient may decrease so rapidly that computer games, which typically provide very reinforcers are ineffective unless they follow rapid feedback. In reinforcing the behavior of responses by no more than fi ve seconds or so. such children the reinforcers must be delivered Consider crossing a room to fi nd a toy. A promptly. This also has implications for impul- child might do this quickly or slowly. Given sivity. Forgoing a small immediate reinforcer an average gradient, starting up the sequence for a later larger one is described as self-con- would participate in the effects of the reinforcer trol, but whether this will happen depends on either way. With the ADHD gradient, however, the steepness of the delay gradient. The steeper 298 • Part X. Conclusion the gradient the less potent the larger reinforc- or other environments outside the laboratory. ers that follow longer delays, and so the greater Sometimes behavior is recorded in checklists or the likelihood of impulsive behavior, behavior charts or with counters or other devices. In such strengthened by more immediate reinforcers. In cases, observers must have well defi ned crite- other words, these three difference manifesta- ria for what counts as an instance of relevant tions of ADHD, hyperactivity, attention defi cit behavior and what doesn’t; without such crite- and impulsivity, seem all to be derivatives of a ria, data collected at one time by one observer single variable, the steepness of the delay gra- may not be comparable to behavior collected at dient. If so, individual differences in steepness another time or by a different observer. Consis- could account for variations in the dominance tency across time and observers is referred to as of one or another of these symptoms in different reliability. The measures must also be relevant to children (Catania, 2005a, 2005b). the behavior of concern. For example, the time The case for an interpretation of ADHD as a child spends with an open book may not be a product of delay gradients in Sagvolden et well correlated with the time the child spends al. (2005) is also strengthened by comparisons reading: the relevance or appropriateness of of the schedule-maintained behavior of Wistar measures is referred to as validity. Recording Kyoto rats (WKY) and a closely related strain of instances of countable behavior is often more spontaneously hyperactive rats (SHR). The SHR productive than recording time spent engaging rats are a useful model for ADHD in children, in the behavior. For example, there’s a big differ- not only because their behavior is affected by the ence between completing fi ve arithmetic prob- same drugs used to treat hyperactivity in chil- lems in one minute and completing only one, dren but also because differences between the but someone looking only at time spent doing delay gradients of the WKY and the SHR rats math wouldn’t see it. Interventions must also be can be directly assessed. acceptable to those to whom they are directed, Parents and teachers may fi nd some compo- so they must also pass tests of social validity, as nents of the ADHD syndrome hard to manage, in assessments of user satisfaction. but if ADHD behavior is a manifestation of steep These issues are of special signifi cance in delay gradients in one tail of a continuous dis- education, but schools may not track what stu- tribution of human variation, maybe individuals dents learn in behaviorally signifi cant ways. with such gradients are especially well suited for Doing so isn’t just a matter of measuring with certain activities or occupations. If so, treating reliability and validity. An early lesson learned such behavior as a disorder may be inappropri- by behavior analysts was that general impres- ate. Hyperactivity may sometimes be inconve- sions of how behavior is changing over time nient to others, but in some settings it could turn can’t be trusted. You must collect data. As with out to be a considerable asset. cumulative records, important properties of the data must be made easily visible and the recorded units must be unambiguous (Johnston MEASUREMENT AND & Pennypacker, 1993). METHODS IN APPLIED One method for recording learning is in cel- SETTINGS eration charts (Lindsley, 1992). In a standard cel- eration chart, the horizontal or x-axis (typically Working outside the laboratory presented meth- time or number of opportunities for behavior) is odological and ethical challenges (D. M. Baer, the usual linear scale, but the vertical or y-axis Wolf, & Risley, 1968; Bailey & Burch, 2005; Gold- (the behavior of interest, usually expressed as a iamond, 1974). The cumulative record made rate) is scaled logarithmically. In a logarithmic it easy to see moment-to-moment changes in or log scale, a given interval is equivalent to a behavior, but it isn’t always feasible to arrange constant ratio rather than a constant difference. such recording in schools or treatment centers For example, on a linear scale the space between Chapter 31. Applied Behavior Analysis • 299

5 and 10 equals that between 25 and 30 or any tiple-treatment designs, in which different other pair with a difference of 5; on a log scale, interventions are compared by presenting them however, the space between 5 and 10 equals that alternately or in some other systematic way. between 40 and 80 or any other pair where one is Applied methods are constrained by the behav- twice the other. Here’s a linear sequence of seven ior targeted for change and other details. New numbers: 10, 20, 30, 40, 50, 60, 70. Here’s a log ones will no doubt be devised for special cases, one: 2, 4, 8, 16, 32, 64, 128. In the linear sequence so here again a sampling must take the place of the middle one is halfway between the highest an exhaustive list. and lowest, in the log sequence it’s closer to the highest than the lowest. Learning often goes from small early incre- BEHAVIOR ANALYSIS AND THE ments to big ones later. The advantage of log BOTTOM LINE scales for examining changes in behavior over time, like a student’s rate of solving math prob- Educational settings aren’t the only ones that lems, is that equal percent changes over equal need insulation from fads like the crusade against times produce lines of constant slope. On a linear reinforcers under the guise of purported hidden scale, early gains would look trivial compared to costs of reward. In business, motivational speak- later big ones and if the scale went all the way ers argue for building self-esteem, as if it could to mastery they’d be lost at the bottom. The log serve as a replacement for achievement rather scale of celeration charts solves that problem. than just being one of its products. Differential Such charts are useful in assessing acquisition attention to competence is too often displaced of academic skills and fl uency (Binder, 1996; K. by noncontingent praise: Good job! On examina- R. Johnson & Layng, 1992). Another important tion, useful interventions in business that seem consideration is determining minimal units of to have involved generalized attention turn out learning, sometimes called learn units (Greer & to involve contingent consequences (Parsons, McDonough, 1999). These measurement innova- 1974). Contingencies are obscured by the lan- tions make the effects of educational interven- guage of reward, which too often fails to specify tions quantifi able. what has been rewarded. Some purported prob- Applications involve different methodologi- lems with reinforcement come about because cal constraints than do laboratory experiments. contingencies are mixed with verbal interven- For example, laboratory research usually can go tions, and reinforcement can be obscured when back and forth between procedures freely, but if human verbal and nonverbal behavior interact an intervention gets rid of self-injurious behav- (Shimoff & Catania, 1998). ior, it may not be appropriate to go back to the As documented by Daniels (2009), conse- original conditions just for the sake of showing quences presumed reinforcing in the workplace the outcome was produced by the intervention may instead have damaging effects. Designat- and not something else. Behavior analysis grew ing an employee of the month leaves out every- out of research with individual organisms, so one else, and at the end of the month that lucky its most signifi cant applied methodologies are employee returns to the same old hunt for a single-case designs (Roane et al., 2011). These parking space anyway. Promotion policies include: reversal designs, in cases where it is based on timing or seniority rather than on per- appropriate to alternate between an interven- formance fail to target competence. Meeting set tion and its absence; multiple-baseline designs, goals might have the consequence that the goal- in which an intervention is introduced at differ- posts are shifted the next time around. Getting ent times for different participants in the study; a good appraisal periodically may be fi ne, but changing-criterion designs, which can assess what maintains the employee’s behavior in the whether changes in behavior follow changes times between? Evenly shared raises or bonuses in the magnitude of an intervention; and mul- give as much to those who have done the least 300 • Part X. Conclusion as to those who have worked the hardest. What the functions of a drug if you deliver it at doses are the contingencies and where are the reinforc- that overwhelm physiological systems; and (iii) ers in these examples? Business provides many when you do drug experiments you can’t draw examples in which what superfi cially seem rein- conclusions from single doses; you must always forcing practices go seriously wrong. Business examine dose-response curves, because the effects practices based on behavior analysis are likely of a drug at one dose will probably be differ- to benefi t not only employees and management ent, and maybe even opposite in direction, from but also the bottom line. what the drug does at other doses. These principles provided useful guide- lines for testing the effects of drugs on baseline BEHAVIORAL PHARMACOLOGY responding maintained by schedules of rein- forcement (Kelleher, Goldberg, & Krasnegor, Behavioral pharmacology, often also called 1975). The research soon showed that early psychopharmacology, began when Peter Dews classifi cations of psychoactive drugs were too (1970) visited Skinner’s pigeon laboratory and simplistic. It was inappropriate to call a drug a recognized that the performances maintained stimulant if it raised response rates maintained by reinforcement schedules were desirable base- by some schedules but not those maintained lines for studying the effects of drugs on behav- by others. Aside from the use of behavioral ior. The effects of some psychoactive drugs pharmacology to differentiate among classes of depended on whether behavior was main- drugs that were behaviorally active, the work tained by interval or ratio schedules, so operant showed that behavior was often sensitive to research could open a window on mechanisms drugs at very low doses. A valuable spinoff was of drug action and offer a methodology for drug the extension of these methodologies to behav- screening (Weiss & Laties, 1969). Meanwhile, ioral toxicology (Weiss, 1983), which has provided techniques were being developed to allow the data on the effects of pollutants in air and soil self-administration of drugs by monkeys, cre- and water; these data have been essential pre- ating opportunities for the study of addictions requisites for setting guidelines for the control of (Schuster & Thompson, 1969; T. Thompson & such substances. Schuster, 1964; Wurster et al., 1977). I got into the picture when, as my fi rst job after my postdoctoral work, I took a position BEHAVIORAL ECONOMICS at the Smith Kline and French Laboratories in Philadelphia; SK&F arranged my auditing of a Behavioral economics began when a few behav- pharmacology course at the medical school of ior analysts recognized that performances and the University of Pennsylvania. But I’d already concepts derived from reinforcement schedules gotten a nontraditional introduction to the fi eld were relevant to properties of behavior studied from Peter Dews, who often visited the Harvard by economists (Rachlin et al., 1976). One point pigeon labs while I was working there. A major of contact was that behavior analysts and econo- point of his was that we should think of drugs mists had mutual interests in choice. Some econ- as stimuli, differing from other stimuli mainly in omists have since adopted behavioral economics their route of administration (Schuster & Balster, as theirs, but economic behavior shares proper- 1977; T. Thompson & Pickens, 1971). He also ties with other varieties of behavior, so behavior offered three basic principles of pharmacology: analysts can still study them profi tably (Bickel, (i) no drug has a single action; all drugs have side Green, & Vuchinich, 1995). effects, and what one investigator might see as a A familiar feature of economic theory is sup- bothersome side effect might be regarded as a ply and demand . It’s obvious enough that an main effect by another; (ii) enough of anything experimenter can arrange a supply of some rein- will block anything; you can’t learn much about forcer, like food, and that the demand for food Chapter 31. Applied Behavior Analysis • 301 can be altered by motivational operations such when the reinforcers share properties, as when as deprivation and satiation. The experimenter different foods contain common ingredients. can manipulate cost of food as a commodity by Behavioral economics has given substantial changing response requirements. For example, attention to discounting (L. Green & Myerson, a reinforcer produced by a fi xed ratio of 100 2004). If you can choose between $100 now and responses can be thought of as twice as expen- $100 next year, you’ll no doubt choose the imme- sive as one produced by a fi xed ratio of 50. diate $100. But what about $100 now versus $15 Not so obvious is the difference between next year? If that’s not enough, how large must situations in which (i) the only food reinforcers the delayed amount be for you to prefer it? We available are those earned within a session and typically discount delayed consequences rela- those in which (ii) earnings are supplemented tive to immediate ones. Studies of human dis- by additional feeding outside the session. The counting typically examine verbal judgments, former is analogous to what economists call a but such procedures have analogs in nonhuman closed economy and the latter to what they call an studies that pit small immediate consequences open economy. Comparisons depend on experi- against larger delayed ones. mental parameters, such as schedule values and Studies of discounting and other economic type of schedule, but some experiments that pro- analogs examine changes in the value of conse- duce increases in response rate with increases of quences: $100 delayed for a year is said to have a reinforcement rate in open economies produce far lower value than $100 available immediately. opposite results with closed economies. But interpretations limited to changes in value Another economic concept is elasticity . When may leave out the behavior. That may make it the demand for a commodity changes substan- easier to communicate with those in econom- tially with changes in price, its demand is said ics and other disciplines, but the advantages of to be elastic; to the extent that demand doesn’t behavioral treatments and interpretations may be change fl exibly with changes in price, it’s said lost or unrecognized when their main strengths to be inelastic. For example, demand for leisure are obscured by verbal transformations. items one can easily do without is highly elastic Furthermore, verbal estimates may differ relative to demand for essentials such as electric- from choices that are products of actual contin- ity or fuel. In experiments arranged as operant gencies. What people say they’ll choose given analogues of elasticity, the behavior maintained hypothetical alternatives may not agree with by reinforcers can vary differently given changes what they choose when confronted with real in cost if the alternatives differ a lot in elastic- alternatives. Whether or not verbal mediation is ity. For example, if a food-deprived organism is involved, we can’t assume choices like these or also dependent on opiates, a doubling of a ratio economic behavior in general are based on ratio- requirement arranged for food reinforcers won’t nal decisions (Kahneman, 2011). Instead, we have the same effect as a doubling of an equal must examine the contingencies. For example, ratio requirement arranged for drug reinforcers rationality isn’t involved when small immediate (Hursh, 1991). reinforcers are selected over large delayed ones. One determinant of elasticity is substitutabil- Words matter, of course, but contingencies are ity, i.e., whether some reinforcers will substitute the bottom line in the analysis of behavior. for others. For example, if coffee costs rise, the effect on demand for coffee will be different in a setting where other sources of caffeine such as TEACHING: THE UNFULFILLED tea are easily available than in a setting with no PROMISE other caffeine source. Substitutability is relevant in experiments that involve different responses When I was a Columbia undergraduate, educa- maintained by different reinforcers, especially tional applications were in the air. Skinner had 302 • Part X. Conclusion given a lecture on teaching machines at Bar- those accomplishments have been unrecognized nard College. The power of computers wasn’t or recognized and forgotten. A prominent exam- yet available, so books with special designs or ple occurred when behavior analytic programs machines that could successively present ques- achieved the most substantial teaching effects tions and feedback were alternatives to what in early research on Project Follow Through, later became programmed instruction (B. H. the largest educational project in United States Barrett, 2002; Skinner, 1968). It would be a while history up to that time (Bushell, 1978). Alterna- before those efforts paid dividends in the pre- tive approaches produced less student progress cision teaching of Ogden Lindsley (Potts, Esh- in academic skills, but funding and implemen- leman, & Cooper, 1993), Fred Keller’s paced tation nevertheless favored programs target- instruction (F. S. Keller, 1968; Twyman, 1998), ing different objectives, such as self-esteem or and other innovations (Goldiamond & Thomp- social skills. Behavior analysts know as well as son, 2002). anyone how to teach children with intellectual I’d been an only child for thirteen years when challenges. Shouldn’t those skills transfer to I acquired a brother. Robert was about eight by the teaching of children at large? Education has the time I took a seminar on teaching taught by sometimes turned to those who know how to Fred Keller. As my course project in the seminar teach things to computers, but shouldn’t it look I took on teaching Robert some algebra, includ- instead to those who know how to teach things ing the distributive law: a(b+c) = ab+ac. One thing to real children? I learned was that teaching the law in as many A principal who had studied behavior analy- ways as possible mattered: solving specifi c num- sis discovered that students spent only minutes ber problems, working with unknowns, saying per week doing math in math classes, so he ran what the law meant. By the time the semester an informal experiment. He asked teachers to ended he understood the distributive law, in the increase the time students spent doing math by sense of being able to work with it in different giving them more math problems to work on ways. I can’t say with any certainty whether the in class. It was unrealistic to ask the teachers project had any long-term effects, but his subse- to grade the extra work, so just the usual math quent career has consistently involved numbers papers were graded and the additional work and mathematics and computer programs. wasn’t even returned to the students. After a When I left Columbia for the graduate pro- while these students began getting better grades gram at Harvard, one of my main interests was on exams even though they hadn’t gotten feed- working on programmed instruction. But soon back on the added work. Just spending more I became caught up in the pigeon laboratory, so time doing math improved math learning. the interest was deferred until years later when Educators must attend to the student’s I began teaching my own courses. When I even- behavior rather than the teacher’s. There’s no tually arrived at UMBC I was fortunate my col- evidence for signifi cant effects of individual league Eliot Shimoff had already been applying learning styles (Pashler et al., 2008), so every stu- what we knew to education. We collaborated dent should be able to learn not just in one way in both laboratory and classroom, so the line but in several different ways: by rote, by under- between teaching and research was sometimes standing, by discovery, and so on, as appropriate blurred (Shimoff & Catania, 1995, 2001). That to different subject matters. But teachers must blurring is as it should be, for what’s the point of also assure that contingencies don’t become research if you don’t learn from your colleagues aversive; absenteeism is one index of whether and teach them in turn about what you’ve school environments provide adequate reinforc- learned? ers. Keller’s Personalized System of Instruction Applied behavior analysis has again and (PSI) allowed each student to work at her own again documented what it can do, yet too often pace, but its structure confl icted with traditional Chapter 31. Applied Behavior Analysis • 303 course formats and created administrative diffi - Effective methods for teaching based on culties (F. S. Keller, 1968, 2009). After fl ourishing behavior analytic principles have been around briefl y, paced instruction waned; whether there for decades (Greer & Ross, 2004; K. R. Johnson & will be a revival remains to be seen (Eyre, 2007). Layng, 1992; Lindsley, 1992; Skinner, 1968). It’s For far too long, educational systems have shameful that in contemporary western culture focused on what the teacher does instead of they are so little recognized, much less imple- what the student does. What the student does mented. Selection by reinforcement is the core is what the student learns. The student is not a of our science, but too many in psychology and vessel into which knowledge can be poured. If education and related disciplines understand no contingencies are placed on student behavior, it superfi cially at best (Catania, 2013b). Teach- it matters little whether a teacher changes from ers must attend to the structure of what they lectures to presentation software or from large teach, but they must also know the functions of class sections to small ones. Too many proposed their interactions with their students. Above all, educational innovations involve little more than they must attend to their students’ behavior. In changing the curriculum, but the problems of the best of all educational worlds the shaping education won’t be solved merely by changing should be reciprocal. course content. The more time students spend In those days when programmed instruc- with the material to be learned, the more they’ll tion seemed to offer so much promise, some learn, so anything teachers can do to get them to concentrated on teaching one or another specifi c put in more time will usually be a good thing. skill. The targets ranged from reading and basic I had a colleague in biology who discov- arithmetic through university-level courses. ered classroom clickers, which allow students’ But some had a different dream: an educational answers to questions to be tallied on an instruc- system constructed from simple units like let- tor console. He said at fi rst his students asked ters and numbers to the reading of words and a lot of questions and got many wrong, but he sentences, and from there to whole paragraphs, gradually tailored his lectures to deal with what not simply at the level of reciting the contents hadn’t worked well. They became so polished but instead answering questions and expanding his students hardly needed to ask questions any on them. Critical thinking would be introduced more. He was of course dealing with the prac- early on. From there the next step would be ticalities of large classes. But I worried that by pages and sections and eventually whole chap- that point he was doing more while his students ters. These could be contained in a convenient were doing less, so maybe he was doing too teaching device called a book; today we might much for them. prefer a tablet that can hold many such devices). I was surprised when a graduate student Students who reach this point have been weaned complained I hadn’t provided outlines of the from their teachers. They’ve learned how to topics discussed in class. When I pointed out learn; they provide examples of higher-order that every chapter already had an outline, the classes writ large. I hope some of my readers student wasn’t satisfi ed. This student was hav- qualify. Perhaps this is what Skinner intended ing the same problem as those educators who when he wrote his oft-quoted sentence, “Educa- think the teacher’s behavior matters more than tion is what survives when what has been learnt the student’s. Students who produce their own has been forgotten” (Skinner, 1964/1980). outlines learn more than those who depend on It’s diffi cult to assign the authorship of the outlines produced by their teachers. Overheads claim that Darwinian selection is so simple any- in class and online outlines are popular, but stu- one can misunderstand it. The claim also holds dents who depend on them are having things true for Skinner’s selection by consequences. done for them rather than learning to do things Darwinian selection remains a foundation for themselves. That’s not what education is about. biology, despite tweaks and reassessments, 304 • Part X. Conclusion but it would be diffi cult to sustain the case that that behavior is too important to be left to psy- selection by consequences plays a similar role in chologists: “…whether behavior analysis will psychology. But effective sciences breed applica- be called psychology is a matter for the future tions, and by that measure behavior analysis has to decide” (Skinner, 1990, p. 1210). Selection by demonstrated its effectiveness. I’m inclined to consequences is the foundation of behavior anal- agree with the statement, variously attributed, ysis. Applications and all, it’s our science now. Chapter 32 Learning and Behavior Revisited

It is so easy...to think that we give an explanation when we only restate a fact. —Charles Darwin

We’ve considered phenomena of elicitation, plexity. The contingency between crying and reinforcement, discrimination and conditioning, feeding is likely to affect the infant’s behavior. among others. I think it’s time for a review. Let’s We can expect an increase in crying, but only do one in the context of examples of social behav- some time since his last feeding, when milk has ior involving parents and a child. Beyond mere again become reinforcing through a motivating observation, our simplest procedure is stimulus or establishing operation, deprivation. presentation. No signal precedes the stimulus Let’s add stimulus control. We superimpose and no response needs to occur before it’s pre- a discriminative stimulus on either of the other sented. Suppose a nursing mother starts out by operations. First consider the mother’s behavior. feeding her newborn infant independently of his When she feeds the infant, the suckling produces behavior (we’ll assume the child is male not for the letting down of milk into her breasts. Once sexist reasons, but rather so we can easily distin- she begins feeding the infant when he cries, the guish the mother and the infant as she and he). crying becomes a reliable antecedent of the feed- Her presentation of the nipple is an instance of ing and the mother discovers she begins letting stimulus presentation and may affect the infant’s down her milk as soon as he begins to cry. The behavior. He’s likely to turn toward her breast stimulus, crying, is followed by another stimu- and begin to suckle. From the nursing mother’s lus, the infant’s suckling. This relation should be point of view, the suckling is also a stimulus and familiar as an example of Pavlov’s respondent elicits the letting down of her milk. This glandu- conditioning. When Pavlov presented dogs with lar response moves the milk toward her nipples, light and then food, the dogs began to salivate where it becomes available to the suckling infant. after light as well as after food, just as the mother Assume now the mother switches to demand lets down her milk during the crying as well as feeding, feeding the infant only when he begins during the feeding. to cry. The feeding becomes a consequence of A discriminative stimulus can also be super- a response, crying. This relation is, of course, a imposed on the consequential operations of rein- response-stimulus contingency; we translate it forcement or punishment. Now the infant has as the effect of a response on the probability of grown a bit and sleeps through the night. The a stimulus. In this instance, the infant isn’t fed mother begins feeding him when he begins to unless he cries; without the crying the probabil- cry during the day but not at night. These times ity of being fed is zero. This is, of course, just are correlated with daylight and darkness, and one example of a contingency. Responses can soon the infant begins to discriminate between raise or lower stimulus probability; they can them. Nighttime crying decreases relative to turn things off as well as turn things on; and daytime crying, and later subtler discrimina- they can change other contingencies, though our tions allow the mother to begin to shape other examples here don’t require such levels of com- kinds of behavior to replace the crying. Until

305 306 • Part X. Conclusion then, during daylight the infant is fed when he milk and the infant’s suckling as examples, but cries but when it’s dark he isn’t; in other words, stimuli are of varied sorts. We could replace light and dark signal the different contingencies. them with aversive ones. The shouting of an During light, crying raises the likelihood of a abusive parent might occur independently of feeding; during dark it doesn’t. the child’s behavior; or it might occur only after When we examine behavioral situations, it’s some response, such as crying; or it might occur often useful to describe them in terms of appro- only in the presence of some other stimulus, as priate operations; sometimes a proper descrip- when the child learns that his father shouts only tion requires a combination. For example, when his mother is around; or it might be that suppose the mother learns that when her infant if he cries his mother always quiets his shout- begins to make fussy noises near bedtime, he’s ing father, so her presence is an occasion when likely to fall asleep quickly if she picks him up he can avoid or escape from his father’s shout- and rocks him. His fussy noises set the occasion ing by crying. These are all examples of aversive for rocking and his subsequent falling asleep contingencies. sets the occasion for tucking him into his crib. It’s sometimes convenient to distinguish With regard to the mother’s behavior, both parts different types of contingent stimuli. Organ- of this sequence involve stimulus-control opera- isms work to produce or remain in the pres- tions superimposed on consequential opera- ence of some, called appetitive, rewarding or tions. The rocking produces a consequence, the reinforcing: food, entertaining company, money, sleeping infant, that’s in turn a discriminative among many others. They work to remove or stimulus for the tucking in; a stimulus contin- stay away from others, sometimes called aver- gent on responding in one part of the situation sive, noxious or punishing : noise, dull company, serves as a discriminative stimulus in another. extremes of heat or cold, among many others. These examples involve interactions between Some stimuli have obvious biological signifi - the infant’s behavior and the mother’s. In the cance while others acquire their signifi cance context of such interactions, the infant learns during the organism’s lifetime. It’s diffi cult to to give special attention to social stimuli. These classify any stimulus unambiguously as a rein- later become signifi cant in many kinds of social forcer or a punisher, especially given the rela- behavior, such as attending to what others say, tivity of reinforcement and punishment. And, taking turns in conversation and saying things having admitted appetitive and aversive stimuli that affect the behavior of others. Maybe you’ve as classes of contingent stimuli, we must also noticed that many of the human examples we’ve recognize that relatively neutral or insignifi - used to illustrate basic processes involve chil- cant stimuli can enter into contingencies. If the dren with little or no verbal behavior, such as infant reaches out and touches his mother, for young infants or children with autism or other example, his movement is a response and con- developmental challenges. This was no accident, tact is its consequence. Obviously, no stimulus because the basic contingencies are most effec- is likely to be completely without signifi cance; tive when they’re uncontaminated by talk. Ver- these classes represent points or regions in a bal behavior is a special kind of social behavior continuous range of stimulus types rather than that comes later in the interactions of parents three discrete categories, and the categorization and children. of stimuli can change with establishing or moti- vational operations. A response may lower as well as raise the KINDS OF CONTINGENCIES probability with which events occur. Suppose AND CONTINGENT STIMULI the father fi nds the infant’s cries aversive. If the infant is especially likely to cry if his dia- Stimuli involved in response-stimulus relations per hasn’t been changed for a while, the father are contingent stimuli. We’ve used the mother’s can avoid the infant’s crying by changing the Chapter 32. Learning and Behavior Revisited • 307 diaper. The probability of crying increases stimulus presentation, which may involve the as time passes without a diaper change, and response-independent delivery of appetitive decreases whenever a diaper change occurs. In stimuli or aversive stimuli or relatively neutral other words, changing the diaper is the father’s or insignifi cant stimuli. avoidance response. A response-stimulus con- Kinds of contingencies and contingent stim- tingency is defi ned as the effect of a response on uli are summarized in Table 32–1; each can be stimulus probability, and we distinguish among combined with a discriminative stimulus. For contingencies in terms of whether the effect is any procedure, it may be instructive to locate it an increase or a decrease in probability. In this or its various stages among the classes shown in last case, in which the diaper change avoids the the table. infant’s crying, the stimulus is aversive and its Now the infant is older and his mother allows probability is reduced by the father’s response. him to crawl around and explore some rooms in Now suppose the infant has fallen asleep and his home. After he’s done this a few times over while watching television the father accidentally several days, the mother takes him to a room awakens him by turning up the sound too loud. in the corner of which is a new toy chest. She The infant begins to cry. The contingent stimu- opens the chest and gives him a favorite toy. The lus, the crying, is again aversive, but this time next day she puts him down in another room a response, turning up the sound on the televi- and he immediately sets out for the room with sion, has raised its probability. The crying may the toy chest, getting there quickly and without punish the turning up of the sound, in that the making any wrong turns. His exploration of the father may from now on be less likely to do so room on the previous days involved behavior while the infant is sleeping. Changes in prob- that produced relatively neutral consequences. ability are, of course, not limited to all-or-none But when something more signifi cant was intro- cases. For example, the infant might not be fed duced, the new chest with some of his toys in it, every time he cries, and he might not awaken the child proved he had learned the layout of the and cry every time someone turns up the televi- rooms. The example is analogous to an experi- sion sound. ment on latent learning. In classifying contingent stimuli, we recog- Consider one more example. The mother has nized relatively neutral or insignifi cant stimuli often taken the infant to the pediatrician’s offi ce as well as appetitive or aversive stimuli. Contin- for routine exams. Time in the waiting room has gencies also range from those in which responses reliably been followed by seeing the pediatrician raise stimulus probability to those in which they in the examining room. One evening the infant lower it, and within this range is the special case becomes ill and the mother takes him to a hospi- in which a response has no effect on stimulus tal urgent care unit instead of the pediatrician’s probability. This special case is equivalent to offi ce. There the pediatrician checks the infant’s

Table 32–1 Kinds of Response-Stimulus Contingencies and Contingent Stimuli Type of Response-Stimulus Contingency Type of Contingent Response Raises Response Doesn’t Aff ect Response Lowers Stimulus Probability of Stimulus Probability of Stimulus Probability of Stimulus Appetitive, rewarding, Positive Reinforcement Stimulus presentation Negative punishment or reinforcing Relatively neutral or Sensory consequences Stimulus presentation Sensory consequences insignifi cant Aversive, noxious, or Positive punishment Stimulus presentation Negative reinforcement punishing 308 • Part X. Conclusion symptoms and then gives him an injection. The infant’s cry is sometimes elicited by events, such injection makes the infant cry. A few days later as painful stimuli, while at other times it occurs the mother takes the infant to the pediatrician’s because such events as the mother’s presence are offi ce for a follow-up exam. Even though he its consequence. We come to understand behav- hadn’t gotten the aversive injection in the exam- ioral situations by separating out stimulus and ining room, the infant begins to cry as soon as response relations that enter into them. That’s he and his mother enter the waiting room. When the business of experimental analysis. the pediatrician’s signifi cance was changed by On close examination, some distinctions the injection, the crying in the waiting room implied by our taxonomy seem to diminish in proved the infant had learned the contingent importance. In the analysis of behavior, classifi - relation between the waiting room and seeing cations often have fuzzy boundaries and distinc- the pediatrician. The waiting room and seeing tions can become arbitrary. We noted such a case the pediatrician are analogous to the initially with respect to presenting or removing stimuli: neutral stimuli of a procedure called sensory is water effective as a reinforcer because of its preconditioning. Constructing other examples presentation or because it terminates dry mouth that correspond to the various cells in the table or other events correlated with thirst? In the fi nal is a worthwhile exercise. analysis, we discarded the distinction between One way to judge the relative signifi cance presenting and removing stimuli in favor of a of events is to compare the probabilities of the behavioral account that considered the relation responses they occasion. If we were interested between the responding that produced a rein- in the child’s playing with toys and eating, we forcer and the responding produced by that might see which he did when both toys and food reinforcer. All consequences of responding can were freely available. When the child becomes be characterized as environmental changes. older, an opportunity to play with friends might Every procedure takes place in some envi- function to reinforce eating if he’s reluctant ronment, so we could next get rid of discrimina- to fi nish a meal, but if he’s just encountered tive stimuli, observing as we did so that we can some cousins he’s never met before at a family deal with them by incorporating them into our get-together and is reluctant to play with them, it response defi nitions. Thus, if a child’s requests might help to see whether the opportunity to eat for candy are granted when his grandparents a favorite food could reinforce playing. In other are present but not when they’re absent, the words, the effect of a contingency may depend response of this contingency can be defi ned to on the relation between the responses that pro- include only those requests he makes when his duce and are occasioned by contingent stimuli. grandparents are around. If later analysis shows We are again reminded that motivating opera- us we can’t even defi ne response classes inde- tions are concerned with the conditions deter- pendently of contingencies, we might ultimately mining the effectiveness of contingent stimuli as decide to dispense with that distinction too. But reinforcers or as punishers. distinctions like these have been useful along the These classifi cations don’t guarantee any way. We can, so to speak, throw away the ladder stimulus or response will have just a single after we’ve climbed it. function: a stimulus in a contingent relation Eventually the child is likely to become a with one response may be in a discriminative fl uent speaker. When we moved by way of relation with another; a response elicited by social learning from learning without words to one stimulus may be involved in contingencies learning with words, we discovered our basic with others. For example, the mother’s presence operations were relevant to verbal as well as may be a contingent stimulus when she comes nonverbal behavior. Verbal classes such as tact- at the infant’s cry and a discriminative stimu- ing and manding were, in effect, names for the lus when the infant learns things happen when relevant processes entering into verbal behavior. she’s there that don’t happen when she’s not; the They also provided contexts within which we Chapter 32. Learning and Behavior Revisited • 309 more fully developed the implications of higher- couldn’t agree on such fundamentals as the basic order classes and recognized potentials for mental units. Was green a unitary sensation or a novel behavior in adduction and other sources combination of the sensations yellow and blue? of emergent behavior. Speaking metaphorically, In this context, Watson advocated behavior, we’ve traveled a long way. not consciousness or mind, as the only legiti- We’ve typically preferred the concrete to the mate subject matter of psychology. He excluded abstract. We’ve built our accounts of the complex mental events such as images as proper areas of from simpler parts, much as biologists have built inquiry on methodological grounds. His version their accounts of complex organisms from anal- of behaviorism came to be called methodological yses of cells and their components. We didn’t behaviorism, he rejected the study of private have to invent new processes to fi ll in gaps. By events. That study would call for a new kind proceeding this way we kept our approach par- of behaviorism, radical behaviorism , perhaps simonious. Our frequent appeals to the details because in looking at the behavior of the scien- may sometimes have seemed constraining, but tist it turned its own science upon itself. environments are properties of the world about Many early studies of animal behavior were which we can often do something. We needn’t devoted to invertebrates (Jennings, 1906; Loeb, be weighed down by prejudices about human 1900; Lubbock, 1882) but attention gradually behavior evolved over many centuries. Think- shifted to vertebrates. Instrumental learning ing analytically about behavior and the environ- was introduced by the mazes of Small (1899– ment is liberating, not limiting. It keeps us from 1900) and Yerkes (1907) and the problem boxes traps created by unconstrained theorizing and of Thorndike (1898). Thorndike moved from unwarranted assumptions about the sources of animal studies to analyses of human learning, behavior (Skinner, 1961). but in doing so was exceptional. Others usu- ally made the extension from animal to human behavior through theoretical statements rather TWO PATHS IN THE STUDY than experiments. Pavlov (1927), for example, OF LEARNING: A CAPSULE addressed human language in his theory of the HISTORY second signal system, but his main impact was through his canine research (Todes, 2014). Dis- The study of learning has at times been regarded tinguishing instrumental learning from respon- as a foundational component of experimental dent conditioning in the early decades of the psychology , but in the late 19th century it was 20th century set the stage for elaborations of overshadowed by other issues, such as those discrimination learning, in the progression from of sensation and perception. By the turn of the jumping stands to rat chambers and pigeon century, research on animal behavior had been boxes (Lashley, 1930; Skinner, 1930, 1938). The stimulated by the work of Darwin (1859), and phenomena of reinforcement schedules fol- the fundamentals of research on human mem- lowed later (Ferster & Skinner, 1957). ory and verbal learning had been provided by Preoccupation with the scientifi c legitimacy Ebbinghaus (1885/1964). Both historical lines of psychology disposed researchers to look to found homes within university laboratories, but other sciences for principles of scientifi c method, despite their academic proximity they remained and gradually the behaviorism of Watson (1919) separate. converged with the operationism newly devel- Early in the 20th century, John B. Watson oping in physics (Bridgman, 1927) and with (1913) addressed what was called his behaviorist the logical positivism in the philosophy of sci- manifesto to a psychology marked by substan- ence (Ayer, 1946). Behaviorism became promi- tial disagreements about its methods and subject nent and the 1930s and 1940s became a period matter. Researchers claimed access to the con- of grand behavioral theories . Applications to tent of consciousness through introspection but human behavior and to language became an 310 • Part X. Conclusion issue in the competition among the systems of play between verbal behavior and the nonverbal Guthrie (1935), Skinner (1938), Hull (1943) and behavior on which it depends, and we still have Tolman (1948), among others, but attempts to much to learn about each. integrate animal learning with research on lan- We must identify the properties of behav- guage gradually lost infl uence as the fortunes ior we hold in common with other organisms of early behaviorist theories based on stimulus- along with those unique to us. Given the myr- response associations declined. Even so, in rely- iad properties of human verbal behavior, both ing on operational defi nitions, methodological elementary phenomena of animal learning and behaviorism became the foundation of virtu- conditioning and the complexities of human ally all contemporary experimental psychology, learning and remembering must come together though it has veered from its foundations in in any effective account. Our treatments of other ways. equivalence classes, awareness and instruc- Meanwhile, the fi eld of human learning tional control couldn’t have proceeded with- and memory went its separate way. The pre- out analyses of antecedent stimuli. Important cedence of Ebbinghaus dominated studies of varieties of human behavior such as autoclitics serial-position effects, massed versus spaced and self-control are based on both verbal behav- practice, meaningfulness, and so on. Thorndike ior and discriminations of our own behavior, and Woodworth (1901), inspired perhaps by so any analysis failing to build on the more questions about the educational value of such elementary processes of stimulus control and classical disciplines as Greek and Latin, began contingencies is bound to be defi cient. Uniting to investigate transfer of learning. The practical these two approaches to learning is a task long relevance of studies of human learning seemed overdue; behavior analysis encompasses all obvious, and the effects on verbal learning of varieties of animal and human learning and is such variables as sleep and distraction and moti- well-equipped for it. vation became the basis for advice on study hab- its. The theoretical underpinnings were the same as those of animal learning, but even though TWO PATHS IN THE STUDY OF laws of effect, contiguity, association and gener- VERBAL BEHAVIOR: ANOTHER alization gave a superfi cial appearance of unity, CAPSULE HISTORY the two experimental lines that had originated as separate entities remained so. Beyond com- The history of the two approaches to learning mon appeals to theoretical principles like asso- we’ve just considered was mostly before my ciation and lip service about mutual relevance, time. Not so for two competing approaches to these research traditions didn’t much infl uence verbal behavior and language. Skinner had each other. begun outlining his treatment of verbal behav- It can reasonably be claimed that these two ior long before he had a conversation about aspects of learning still remain separate (Cata- behaviorism with the philosopher Alfred North nia, 1985). We might even ask the point of treat- Whitehead during a dinner for the Harvard ing them together. Perhaps animal learning and Society of Fellows in 1934 (Skinner, 1957, pp. conditioning and human learning and remem- 456–460). Whitehead asked Skinner to account bering should go their separate ways. But here for his saying “No black scorpion is falling upon I’ve argued otherwise. Such a course would be this table.” Skinner accepted the challenge. a mistake for several reasons: learning phenom- Skinner regarded Verbal Behavior as his ena studied with animals also occur in human most important work. I was fortunate to get behavior; the nature of complex human learning the chance to read it early. During the Spring is clarifi ed by analyses in terms of more elemen- semester of 1957, along with a half dozen or so tary processes; perhaps most important, human other Columbia College undergraduates, I took behavior especially is characterized by the inter- a weekly seminar on verbal behavior jointly Chapter 32. Learning and Behavior Revisited • 311 conducted by three faculty members: Ralph F. structure during post-Sputnik efforts to translate Hefferline, Fred S. Keller, and W. N. Schoenfeld Russian technical materials into English. Lan- (Catania, 1997a). During the fi rst several weeks guage translation was a major focus as the United we spent our time on a mimeographed docu- States and the Soviet Union faced off against ment, Skinner’s William James lectures , which each other during the Cold War. Early attempts he delivered at Harvard University in 1947 and simply to substitute words in one language for which was a forerunner of the book. Each class their equivalents in another were unsuccess- began with a thorough summary of the current ful for a variety of reasons. For example, many material by one of the faculty members. We fi n- words have multiple equivalents: should bar be ished with the lectures about midway through translated as a noun, either a lever or a place to the course, when the book based on them was drink, or as a verb, to stop?; and grammatical published and fi nally became available. As we distinctions in one language may be absent in progressed through the book we had the advan- another; Russian doesn’t use articles, so how do tage of having read the lectures, with which we you decide whether an English translation of a compared it. Russian noun should be preceded by a or the or In the Fall of 1959 I was in the audience as no article at all? Backed by the prestige of Alan a graduate student in the Harvard Depart- Turing’s mathematics (Hodges, 1985) and grow- ment of Psychology when an MIT linguist ing computer technology, linguists argued their named Noam Chomsky gave a colloquium on fi eld was the only one that could produce effec- his new transformational grammar. Skinner tive computer translations. Grant monies were was in attendance, but Chomsky didn’t men- at stake, and at the time Skinner’s Verbal Behavior tion his forthcoming review of Verbal Behavior, (1957) probably looked like competition. which must already have been written because Major components of linguistics include it appeared shortly afterwards (Chomsky, 1959). syntax , concerned with grammatical structure; It overshadowed many favorable reviews of semantics, concerned with problems of meaning; Skinner’s book (Knapp, 1992). Chomsky was and phonology, concerned with the structure of mainly concerned about language structure, and speech. Pragmatics, concerned with the functions for many years it was assumed his critique had of language, is closest to the concerns of analyses destroyed Skinner’s position. Linguists too often of verbal behavior, but tends to get less attention concentrate more on whether verbal behavior within linguistics than the other areas. Linguis- is grammatical than on how it functions (Cata- tic accounts often focus on structure whereas nia, 2003b; Jackendoff, 2002; Pinker, 1994). Most behavioral ones often focus on function, so behavior analysts who read Chomsky’s review, debates over the relative importance of the fi elds including Skinner, concluded Chomsky missed are usually about as useful as those, now long the point of the book, so no reply seemed nec- gone in biology, over whether anatomy or phys- essary (R. A. Harris, 1993). Years passed before iology is the more important. responses to Chomsky’s arguments appeared Among the controversial issues was whether (e.g., MacCorquodale, 1970). Unfortunately, in consequences play a role in the child’s acquisi- the meantime some concluded Skinner didn’t tion of language, and especially in the child’s write a reply because he couldn’t produce an acquisition of grammatical structure. It was effective one. claimed behavioral accounts could not deal A critical precursor of contemporary psy- with productivity, the child’s capacity to produce cholinguistics was the effort to write programs new sentences the child could never have heard for computer translation from one language to before. Appeals to the poverty of the stimulus another. That history included the evolution argued the child’s verbal environment isn’t rich of computers in Allied efforts to break military enough to support acquisition of grammar; if so, codes during World War II and later applica- some grammatical features must be “prewired,” tions of mathematics to problems of language in the sense they’ll emerge even without rel- 312 • Part X. Conclusion evant contingencies. But those who examine capacity to utter novel sentences. Yet contempo- exchanges between parents and children typi- rary behavior analysis can identify not one but cally fi nd contingencies play an indispensable many sources of novel behavior. role (Moerk, 1980, 1983; Whitehurst & Valdez- Even if it were proved children don’t have Menchaca, 1988). to learn all the details of grammar because some The case for the poverty of the stimulus argues are built in, that wouldn’t mean there aren’t lots that verbal environments don’t include the of other things about verbal behavior they’ll still ungrammatical instances that should be there to have to learn. In fact, there’s plenty of evidence support claims that a child’s grammatical behav- that rich verbal environments in which parents ior is shaped through natural contingencies, spend lots of time interacting verbally with their either because such instances don’t occur in the children can make vast and lasting differences speech children hear, or because such instances in their verbal competence (B. Hart & Risley, aren’t corrected when children make them, or 1980, 1995; Moerk, 1992). Contingencies matter. both. For example, if a child is questioned about The effects, described as meaningful differences a laughing frog muppet, the child may hear Is the in the everyday experience of young American muppet who is laughing a frog? but won’t ever hear children, appear in vocabulary growth rate, Is the muppet who laughing is a frog? (Crain, 1991). school performance and IQ scores. What else The child later asks questions with the former emerging from the analysis of behavior could grammatical syntax and rarely if ever with the have greater relevance? latter ungrammatical one. What keeps the child I believe we’re at last emerging from our from making the latter error? eclipse. It wasn’t total. Along with other col- The simplest reply is that all possible nega- leagues, I began teaching both undergraduate tive instances aren’t necessary for learning. For and graduate courses using Skinner’s book. It example, the echoic shaping of phonetic struc- was a way to maintain a small verbal commu- ture doesn’t demand an environment including nity within which to practice the language of nonnative as well as native speech sounds, any verbal behavior. Meanwhile, with the growth of more than that a pigeon discriminating between applied behavior analysis, verbal behavior has green and red needs exposure to the entire spec- become a crucial part of therapies targeting the trum (Moerk, 1992). With syntax, a verbal envi- verbal defi cits of autism-spectrum children. It ronment including contrasts between at least makes a difference to know manding is a good two grammatical structures provides differences place to start and to understand that building a on which learning can be based: for example, solid foundation of tacting and other basic ver- both active and passive voices are grammati- bal classes is a prerequisite for more complex cal but provide a contrast that may enable later verbal skills. You won’t fi nd those distinctions discriminations of grammatical versus ungram- in linguistics. Based on the growth of online chat matical (Palmer, 1996, 1998). sites in which groups of parents discuss Skin- During the nineteenth century eclipse of ner’s Verbal Behavior, some believe more people Darwinism (Bowler, 1983), many believed have read the book since its 2007 fi ftieth anni- Darwinism was dead because existing genetic versary than in the entire half-century following theory provided no grounds for the variations its publication. As we learn more about the third needed by selection. That eclipsed ended in the variety of selection it has illuminated, it’s a good early twentieth century when genetics showed bet its light will grow even brighter over time. that mutations could provide those variations. Behavior analysis has been similarly eclipsed (Catania, 1987), and here too the issues have LEARNING AND EVOLUTION been about where the variations come from. Chomsky had argued that behavior analysts had Biology has evolved from a theoretical disci- nothing to say about productivity, the child’s pline to a descriptive one. The double helical Chapter 32. Learning and Behavior Revisited • 313 structure of DNA was once a theory of sorts, but ments in those disciplines (Carroll, 2005; Kiecker when Watson and Crick (1953) constructed their & Lumsden, 2005; West-Eberhart, 2003). Attend- model and showed how it worked detailed con- ing to them will serve us well. fi rming investigations followed soon after. Now In discussing types of selection, we’ve we can look at the molecule. Biologists and their argued that properties of learning parallel those colleagues in related disciplines have created a of evolution, because the selection or survival of taxonomy of fundamental biochemical and other patterns of behavior in an organism’s lifetime processes. That taxonomy provides the parts, has much in common with the selection or sur- and biologists can show what a biological system vival of individuals in the evolution of species does and how it does it by demonstrating how (Catania, 1978; Skinner, 1966, 1975). Our discus- those parts fi t together and interact (Carroll, 2005, sion of social learning made a similar case for the p. 10, 13, 295). The contemporary microbiolo- selection of cultural practices (Catania, 2001b; gist doesn’t trade in theories of cell metabolism Dawkins, 1976; M. Harris, 1977; Skinner, 1981b). but instead shows how cell metabolism works. Our own verbal behavior makes this variety of Seeing how behavior works is perhaps trickier selection of special interest. Each type of selec- sometimes, because behavior doesn’t sit still for tion involves variation that provides the source us while we look at it. It’s now more respectable materials on which it operates, and each involves than once it was to speak of taxonomies for cat- some mechanism for selecting what survives. egories other than plants and animals (Ross & Whatever else happens at every level, behavior Spurrett, 2004, p. 604), so it’s appropriate for us gets selected and environments do the selecting. to expand our taxonomy. The parallels between Darwinian natural A source of controversy is the relation selection and operant selection also extend to between behavior and physiology, and the the problems of acceptance each has faced. Like nervous system in particular. Those studying Galileo’s displacement of the earth from the the neurophysiology of learning need to know center of the universe to an orbit around the what happens in behavior to know what they sun and Freud’s challenge to the central status should look for in nervous systems. A neuro- of human consciousness, these accounts over- physiologist who thinks learning is mainly stim- turned traditional ways of thinking about our ulus-response associations will look for different place in nature. Phylogenic and ontogenic selec- things than one who thinks learning is a product tion faced similar substantive challenges. For of ontogenic selection. Skinner compared the example, artifi cial selection was familiar in Dar- relation between behavior analysis and neuro- win’s time; what was questioned was whether science to that between genetics and biochemis- selection could operate naturally. The operant try: “It is the function of the science of behavior parallel is provided by shaping, another artifi cial at the present time to give neurologists their selection procedure, as when an experimenter assignments, as it was the function of genetics shapes a pigeon’s fi gure-8 turns or as when a prior to the discovery of DNA to give modern behavior therapist shapes the vocalizations of geneticists their assignment with respect to the a nonverbal child on the autism spectrum. The gene” (Skinner, 1988, p. 60; see also p. 461). effectiveness of shaping is self-evident; what’s Parallels with biology often provide hints questioned is whether it operates naturally to for useful directions in the analysis of behavior. produce the varied patterns of behavior we see That should be no surprise: after all is said and in everyday life. done, behavior remains central to biological sys- It’s not good enough to argue that in humans tems. Despite the origins of behavior analysis in the effects of shaping are often masked by ver- philosophy and psychology, we’re part of biol- bally governed behavior. We’d like to have ogy and must cultivate our intellectual ties with documented cases in which the changes in the biological sciences. If we do so, we can antici- contingencies are identifi ed early and tracked, pate continuing insights from ongoing develop- but typically we only have outcomes, after the 314 • Part X. Conclusion natural contingencies have done their work. be alert for the effects of such contingencies than For example, we assume ontogenic selection to assume they don’t exist. was involved in shaping the skill with which In Darwin’s account of evolution, the rela- grizzly bears catch salmon in the rivers of the tions among populations of organisms couldn’t Pacifi c Northwest, but we mainly see the dif- be expressed in terms of similar topographies . ferences between the ineffi cient performances For example, males and females within some of the young novices and the well-coordinated species differ more from each other in form than actions of the experienced adults; we don’t see some pairs of organisms within completely unre- the shaping itself, because it continues over too lated species (the social insects provide striking long a time. instances). For Darwin, the important basis for Shaping can be hard to see if you don’t know distinguishing among populations was descent. what to look for. If you’ve done actual shaping To defi ne relations among populations, look at you’re more apt to notice it when it happens nat- where they came from. Darwin’s achievement, in urally than if you’ve only read about it. Thus, the fact, was more description than explanation . His parents who always wait a while before attend- account of evolution didn’t depend on any the- ing to a crying child may not notice they’ve grad- ory specifying the mechanics of evolution. Even ually shaped louder and more annoying cries. though he couldn’t say exactly how selection did The attention reinforces the crying, and annoy- it, he had described how evolution works. ing cries are, by defi nition, the ones most likely In the analysis of behavior, we deal with to get attention. If you watch what a parent does populations of responses. These populations are when a child throws tantrums, it’s usually easy sometimes not well-defi ned. The problem isn’t to guess where the tantrums came from. very different from Darwin’s. Darwin clearly Time is another factor in the acceptance of recognized the arbitrary nature of the concept of both types of selection. For Darwinian natural species, but unambiguous defi nitions of species selection, the question once was whether the were no more critical to his account than unam- earth had existed long enough for such selec- biguous defi nitions of stimulus or response tion to have taken place. Upward revisions of classes are to a behavioral account. When we the age of the earth resolved the problem. The distinguish among words by the circumstances comparable problem is easier to deal with for in which they’re uttered (e.g., fi re as a mand, operant selection. Even with rapid breeders a tact, an echoic or a textual response), we’re like the fruit fl y , genetic experiments take days. simply distinguishing among classes of ver- Shaping, however, can be demonstrated in min- bal responses based on their origins. Ambigu- utes. If reinforcers can do so much to behavior ous cases will necessarily occur because, just as when contingencies are deliberately arranged organisms have many ancestors, responses have over relatively short periods of time, isn’t it rea- many origins. If that weren’t so we wouldn’t sonable to assume they’ll also affect behavior need analysis. when natural contingencies operate over sub- We might do well to see whether our anal- stantial periods in an organism’s lifetime? Many yses of social contingencies can effectively be contingencies can take hold of behavior over extended to human behavior on a larger scale. the course of a year in the life of a young child. Can analyses of verbal behavior contribute to Compared to how long most artifi cial cases of our understanding of the rapid cultural changes shaping last, a year is an extremely long time. that led to such moral revolutions as the demise Some contingencies are subtle, especially given of dueling and the overthrow of slavery and still the broad range of events serving as reinforcers. ongoing changes in the role of women (Appiah, The behavior they produce may be desirable or 2010)? Can analyses of conditional discrimina- not. Given what artifi cial contingencies can do tions contribute to our understanding of the quickly, natural contingencies should be able to social contingencies that foster tolerance (Meyer do a lot in a long time. It’s more appropriate to & Brysac, 2012)? Can analyses of the side effects Chapter 32. Learning and Behavior Revisited • 315 of reinforcement and of aversive control con- taxonomy. They can be combined in various tribute to our understanding of how social and ways, when it may be appropriate to speak of political precursors determine not only whether behavior synthesis, which is another name for confl icts are peacefully or violently resolved, applied behavior analysis. For example, we but also how one or another mode of resolution can synthesize some kinds of sequential perfor- may favor the kinds of political and cultural sys- mances through chaining procedures. In a more tems that follow (Chenoweth & Stephan, 2011; complex instance, we can combine discrimina- Nevin, 1996a, 1996b, 2003, 2005)? Our research tive stimuli, reinforcement schedules and delays enhances our understanding of concepts like of reinforcement so the resulting contingencies self and causality and volition and judgment are analogous to those when we speak of self- and explanation, but it does not—indeed can- control. In the verbal domain, we can integrate not—displace them (Field & Hineline, 2008; echoic behavior, tacting and listener behav- Hineline, 1990; Killeen & Jacobs, 2016; Neu- ior into a higher-order class called naming. If ringer, 2014; Nevin, 1969; Rachlin, 2014). By our syntheses are successful, we can use them clarifying our place in the evolution of life on to clarify the properties of behavior; if they are our planet, we may strengthen our stewardship unsuccessful, we can use them to identify com- of human social institutions and human envi- ponents of performance missed or taken for ronments. granted in preceding analyses (e.g., as when we study variables affecting the likelihood of com- mitment responses in self-control procedures, BEHAVIOR ANALYSIS AND or as when we discover, in designing an animal BEHAVIOR SYNTHESIS analogue of some human performance, that ver- bal behavior must have had a role we hadn’t So where do we as individuals fi t in all this? allowed for). Because many important human Skinner described himself as “a locus—a place problems involve creating new behavior (e.g., in which a number of variables come together in teaching children with disabilities), the applica- a unique confl uence to yield an equally unique tions of our methods are often matters of behav- achievement” (1957, p. 313), and elsewhere as ior synthesis (Fisher et al., 2011; Lattal & Perone, “a locus, a place in which certain genetic and 1998; Madden, 2012). environmental causes come together to have The term learning has receded into the back- a common effect” (1972/1999, p. 398). We can ground in all this; perhaps it’s outlived its use- fi nd the same point echoed by Igor Stravinsky, fulness. New behavior has many sources. We in program notes for his own recording of his can modify behavior hierarchies, shape new most famous work: “I was guided by no system responses, construct higher-order classes, gen- whatever in Le Sacre du Printemps…. I heard and erate discriminations, form equivalence classes, I wrote what I heard. I am the vessel through solve problems and create novel behavior which Le Sacre passed.” We may recall the “lucky through adduction. Our understanding of these mud that got to sit up and look around” in Von- phenomena depends at least in part on whether negut (1963, ch. 99), but Skinner and Stravinsky we’ve developed a vocabulary consistent with were not passively looking; their quotations are and worthy of them. We can assume our current about their creations. And we as behavior ana- behavioral vocabulary will evolve as research lysts have learned to do more than merely look progresses. Yet in emphasizing behavioral proce- around. We’ve learned to solve problems by dures and outcomes, it adheres closely to what’s changing our environments; we’ve learned what done and what’s observed as we expand our sci- behavior can do. ence and apply it. The success of behavior analysis Behavior analysis begins with complex will be measured by its survival in the behavior of behavior and breaks it down into its compo- those who practice it and by the effectiveness of nents. These are the elements of our behavioral the behavior syntheses following from it. Appendix I. Detailed Chapter Contents

PART I. INTRODUCTION PART II. BEHAVIOR WITHOUT LEARNING Chapter 1. Learning and Behavior Chapter 3. Evolution and Development

The Language of Learning and Behavior Chaos Theory and Darwin’s Butterfl y Function and Structure The Nature of Evolution Behavior Analysis Recipes and Blueprints Antecedents, Behavior, Consequences Variation and Selection Stimuli and Responses The Origins of Complexity Behavior Hierarchies Evolution and Development Phylogeny and Ontogeny Three Kinds of Selection Chapter 2. A Behavior Taxonomy

Observing Chapter 4. Motor and Sensory Systems Presenting Stimuli Arranging Consequences The Evolution of Motor and Sensory Systems Signaling Events and Procedures Motor Systems Signaling Stimulus Presentations Sensory Systems Signaling Consequences Touch: Breast Self-Examination and Signal Detection Establishing the Effectiveness of Consequences Hearing: A Dual System Verbal Governance Vision: Blood on the Walls, Dust Storms and Haloes Summary Proprioception and Biofeedback Sensory Gradients and Inhibitory Interactions

Chapter 5. Elicited and Emitted Behavior

The Refl ex: Elicitation Properties of Elicited Behavior Eliciting Stimuli and Response Probabilities Types of Stimulus Response Relations Successive Elicitations: Habituation Patterning of Behavior in Time From Elicited to Emitted Behavior The Role of Exercise Stimulus Presentations in Imprinting

317 318 • Appendices

PART III. LEARNING WITHOUT WORDS: PART IV. LEARNING WITHOUT WORDS: CONSEQUENCES OPERANT CLASSES Chapter 6. Consequences of Responding: Reinforcement Chapter 11. Operants: The Selection of Behavior

Mazes and Learning Curves Shaping: Differential Reinforcement of Successive Experimental Chambers and Cumulative Records Approximations The Vocabulary of Reinforcement Natural and Artifi cial Selection in Shaping Misconceptions about Reinforcement Shaping: From Animals in Wartime to Pets Reinforcement Isn’t Bribery Differentiation and Induction The Myth of Hidden Costs Response Classes Contingent Acts of Kindness Examples of Differential Reinforcement Self-Reinforcement as Misnomer Shaping as Signal Detection Function Versus Topography Chapter 7. Reinforcers as Opportunities for Behavior Chapter 12. The Structure of Operants The Relativity of Reinforcement The Acquisition of Behavior Differential Reinforcement of Temporal Organization Sensory Motor Learning Response Sequences: Chains Versus Chunks The Analysis of Reinforcement Mediating Behavior Strength: Resistance to Change versus Rate of Variable Behavior Responding The Shaping of Physiological Responses Activation and Coupling Higher-Order Classes and Operant Contingencies

Chapter 8. Reinforcement, Free Reinforcement and Chapter 13. Motivating Variables and Reinforcer Classes Extinction Assessing Reinforcers Extinction Versus Inhibition Reinforcer Classes and Reinforcer-Specifi c Effects Side Effects of Extinction Conditional or Conditioned Reinforcers Extinction Versus Free Reinforcement Concurrent Chain Schedules: Preference The Vocabulary of Free Reinforcement Self Control Extinction and Superstition Procrastination Free Choice Preference Motivating Events in Aversive Control Chapter 9. Consequences of Responding: Punishment

The Vocabulary of Punishment Comparing Reinforcement and Punishment The Relativity of Punishment Side Effects of Punishment Eliciting Effects of Punishers Discriminitive Effects of Punishers Negative Punishment: Timeout The Ethics of Punishment

Chapter 10. Consequences of Responding: Escape and Avoidance

Escape Elicited Responding and Escape The Ambiguous Distinction between Positive and Negative Reinforcement Criteria for Identifying Contingencies Touching the Hot Stove: Natural Aversive Contingencies Avoidance: Hard to Initiate but Easy to Maintain The Nature of the Reinforcer in Avoidance Extinction after Negative Reinforcement Responding versus Not Responding Appendix I • 319

PART V. LEARNING WITHOUT WORDS: CONTINGENCIES Chapter 14. Parameters of Reinforcement: Delays and Chapter 18. Behavior Synthesis Schedules Multiple and Mixed Schedules: Observing Responses Variable Ratio and Variable Interval Schedules Multiple Schedules: Inhibitory Interactions Yoked Schedules Chained, Tandem, and Second Order Schedules Fixed Ratio and Fixed Interval Schedules Extended Chains Limited Hold Brief Stimuli Differential Reinforcement of Response Rates Concurrent Schedules: Matching and Maximizing Delay of Reinforcement Natural Foraging Delays and the Ratio-Interval Difference A Schedule Taxonomy Reinforcement Schedules and Causation The Taxonomy of Reinforcement Schedules PART VI. LEARNING WITHOUT WORDS: CONDITIONING Chapter 15. Discriminated Operants: Stimulus Control Chapter 19. Respondent Behavior: Conditioning Discrimination in the Vernacular Conditional Refl exes The Nature of Discriminated Operants The Role of Elicited Responses Attending to Properties of Stimuli Types of Conditioning Gradients of Stimulus Control Conditioning, Contiguity and Consequences Feature-Positive Discriminations Consequences of Elicited Behavior Fading: Stimulus Control by Successive Approximations Autoshaping The Vocabulary of Differential Reinforcement Stimulus Combinations in Conditioning Natural Concepts Overshadowing, Blocking, and Inhibitory Stimuli

Chapter 16. Conditional Discrimination and Stimulus Chapter 20. Operant Respondent Interactions: Emotion Classes Conditioning and Emotion Relations as Stimulus Dimensions The Language of Emotion Matching to Sample and Oddity Pre-aversive and Pre-appetitive Stimuli Symbolic Behavior: Equivalence Classes Preparedness Higher-Order Classes of Behavior Contingencies Operating on Subclasses within Higher-Order Classes Origins of Structure Animal Navigation

Chapter 17. Sources of Novel Behavior

Toward a Taxonomy of Novel Behavior Reinforcement of Variations: Shaping and Fading Emergence of New Responses: Higher Order Classes Equivalence Classes and Frames Combining Classes: Adduction Serial Coordinations Coordinations in Parallel Joint Control Fluency and Teaching 320 • Appendices

PART VII. LEARNING WITH WORDS: PART VIII. WHEN VERBAL AND NONVERBAL VERBAL BEHAVIOR BEHAVIOR INTERACT Chapter 21. Social Learning Chapter 26. Verbal Governance

Kinds of Social Contingencies Multiple Causation Learning about Others Verbally Governed and Contingency Shaped Behavior Learning from Others Instructional Control Learning about Oneself Insensitivity to Contingencies The Selection of Cultural Contingencies Shaping Verbal Behavior Effects of Shaped Verbal Behavior on Other Behavior The Development of Correspondences Chapter 22. Words as Stimuli and Responses Verbally Governed and Contingency-Shaped Verbal Behavior Verbal Behavior: Not a Synonym for Language Correspondences Between Spoken and Written Classes Echoic Behavior Chapter 27. Prejudice as Verbally Governed Discrimination Transcription Textual Behavior Building Up and Breaking Down Equivalences Dictation Taking From Dichotomies to Continua: Race and Gender Relations Among the Classes Discriminating within and between Classes The Replication of Verbal Behavior Attitudes, Intentions, and Attributions Parallels in Music Mass Shootings as an Example of Multiple Causation The Advantages of Diversity Chapter 23. Antecedents and Consequences of Words Chapter 28. Verbal Function: Coordinations among Intraverbals: Thematic Classes Classes Intraverbal Chains Intraverbal Chunks The Listener’s or Reader’s Behavior Verbal Learning From Action to Acting to Literature: Creating Worlds The Consequences of Verbal Operants: Manding Interlocking Verbal Contingencies The Mand Replication Verbal Governance Verbal Shaping Chapter 24. Contact of Verbal Behavior with the Attention to Verbal Stimuli Environment The Coherence of Interlocking Verbal Classes When Verbal Behavior Becomes a Closed System Tacting Abstraction and the Coordination of Words with Stimulus Properties The Extension of Verbal Classes Metaphor and its Relatives The Language of Private Events Naming as a Higher-Order Class Language Development

Chapter 25. Verbal Behavior Conditional on Verbal Behavior

Descriptive Autoclitics: Discriminating Our Own Verbal Behavior The Conjunction of Verbal Units The Verbal Behavior of Nonhumans Appendix I • 321

PART IX. REMEMBERING AND KNOWING PART X. CONCLUSION Chapter 29. Remembering Chapter 31. Applied Behavior Analysis

Mnemonics The Autism Spectrum and Other Developmental The Metaphor of Storage, Retention and Retrieval Challenges Storage: Encoding Attention Defi cit Hyperactivity Disorder and the Delay Retention: Memory Reorganization Gradient Retrieval: Cue dependency Measurement and Methods in Applied Settings Kinds of Remembering Behavior Analysis and the Bottom Line Iconic, Long-Term and Short-Term Remembering Behavioral Pharmacology Iconic Memory: The Persisting Effects of Stimuli Behavioral Economics Short Term Memory: The Role of Rehearsal Teaching: The Unfulfi lled Promise Long Term Memory: Interference and Forgetting Hearsal and Rehearsal in Remembering Chapter 32. Structure and Function in Learning Discriminated Remembering Kinds of Contingencies and Contingent Stimuli Chapter 30. Knowing Two Paths in the Study of Learning: A Capsule History Two Paths in the Study of Verbal Behavior: Another Visualizing Capsule History The Metaphor of Mental Representations Learning and Evolution Problem Solving Behavior Analysis and Behavior Synthesis Artifi cial Intelligence: Chess Expert Performance Transfer Simulation Science as Seeing Back to the Insightful Ape References

Ader, R., & Cohen, N. (1985). CNS-immune system Azrin, N. H., Hutchinson, R. R., & McLaughlin, R. (1965). interactions: Conditioning phenomena. Behavioral and The opportunity for aggression as an operant reinforcer. Brain Sciences, 8, 379–394. Journal of the Experimental Analysis of Behavior, 8, 171–180. The American Heritage Dictionary of the English Language. (2011). Badia, P., Suter, S., & Lewis, P. (1966). Rat vocalization to (5th ed.). Boston: Houghton Miffl in Harcourt. shock with and without a CS. Psychonomic Science, 4, 117– Anderson, J. R. (1993). Problem solving and learning. American 118. Psychologist, 48, 35–44. Baer, D. M., Peterson, R. F., & Sherman, J. A. (1967). The Andronis, P. T., Layng, T. V. J., & Goldiamond, I. (1997). development of imitation by reinforcing behavioral Contingency adduction of “symbolic aggression” by similarity to a model. Journal of the Experimental Analysis of pigeons. Analysis of Verbal Behavior, 14, 5–17. Behavior, 10, 405–416. Anger, D. (1956). The dependence of interresponse times Baer, D. M., Wolf, M. M., & Risley, T. R. (1968). Some current upon the relative reinforcement of different interresponse dimensions of applied behavior analysis. Journal of Applied times. Journal of Experimental Psychology, 52, 145–161. Behavior Analysis, 1, 91–97. Anger, D. (1963). The role of temporal discriminations in the Baer, R. A., Detrich, R., & Weninger, J. M. (1988). On the reinforcement of Sidman avoidance behavior. Journal of functional role of the verbalization in correspondence Experimental Psychology, 6, 477–506. training procedures. Journal of Applied Behavior Analysis, Appiah, K. A. (2010). The honor code: How moral revolutions 21, 345–356. happen. New York: Norton. Bailey, J. S., & Burch, M. R. (2005). Ethics for behavior analysts. Ayer, A. J. (1946). Language, truth and logic (2nd ed.). New York: Mahwah, NJ: Erlbaum. Dover. Balda, R. P., Kamil, A. C., & Grim, K. (1986). Revisits to Ayllon, T., & Azrin, N. H. (1968). The token economy. New York: empty cache sites by nutcrackers. Animal Behaviour, 34, Appleton-Century-Crofts. 1289–1298. Azrin, N. H. (1956). Some effects of two intermittent Bandura, A. (1976). Self-reinforcement: Theoretical and schedules of immediate and nonimmediate punishment. methodological considerations. Behaviorism, 4, 135–155. Journal of Psychology, 42, 3–21. Bandura, A. (1995). Comments on the crusade against the Azrin, N. H., & Hake, D. F. (1969). Positive conditioned causal effi cacy of human thought. Journal of Behavior suppression: Conditioned suppression using positive Therapy and Experimental Psychiatry, 26, 179–190. reinforcers as the unconditioned stimuli. Journal of the Bandura, A., Adams, N. E., & Beyer, J. (1977). Cognitive Experimental Analysis of Behavior, 12, 167–173. processes mediating behavioral change. Journal of Azrin, N. H., & Holz, W. C. (1966). Punishment. In W. Personality and Social Psychology, 35. K. Honig (Ed.), Operant behavior: Areas of research and Barrett, B. H. (2002). The Technology of Teaching revisited. application (pp. 380–447). New York: Appleton-Century- Concord, MA: Cambridge Center for Behavioral Studies. Crofts. Barrett, J. E., & Stanley, J. A. (1980). Maintenance of Azrin, N. H., Holz, W. C., Ulrich, R., & Goldiamond, I. responding by squirrel monkeys under a concurrent (1961). The control of the content of conversation shock-postponement, fi xed-interval shock-presentation by reinforcement. Journal of the Experimental Analysis of schedule. Journal of the Experimental Analysis of Behavior, Behavior, 4, 4–25. 34, 117–129. Azrin, N. H., Hutchinson, R. R., & Hake, D. F. (1966). Bauer, D. W., & Miller, J. L. (1982). Stimulus-response Extinction-induced aggression. Journal of the Experimental compatibility and the motor system. Quarterly Journal of Analysis of Behavior, 9, 191–204. Experimental Psychology, 34A, 367–380.

351 352 • References

Baum, W. M. (1974). On two types of deviation from the Blough, D. S. (1959). Delayed matching in the pigeon. Journal matching law: Bias and undermatching. Journal of the of the Experimental Analysis of Behavior, 2, 151–160. Experimental Analysis of Behavior, 22, 231–242. Blough, D. S. (1989). Odd-item search in pigeons: Display Baum, W. M. (2002). From molecular to molar: a paradigm size and transfer effects. Journal of Experimental Psychology: shift in behavior analysis. Journal of the Experimental Animal Behavior Processes, 15, 14–22. Analysis of Behavior, 78, 95–116. Blough, D. S. (1992). Effects of stimulus frequency and Baum, W. M. (2011). Behaviorism, private events, and the reinforcement variables on reaction time. Journal of the molar view of behavior. Behavior Analyst, 34, 185–200. Experimental Analysis of Behavior, 57, 47–50. Bechterev, V. M. (1933). General principles of human refl exology (tr. Boakes, R. A. (1973). Response decrements produced by E. Murphy & W. Murphy). London: Jarrolds. extinction and by response-independent reinforcement. Bem, D. J. (1967). Self perception: An alternative interpretation Journal of the Experimental Analysis of Behavior, 19, 293–302. of cognitive dissonance phenomena. Psychological Review, Bolinger, D. (1973). Truth is a linguistic question. Language, 74, 183–200. 49, 539–550. Berg, J. P., & McDowell, J. J. (2011). Quantitative, steady- Bolles, R. C. (1970). Species-specifi c defense reactions and state properties of Catania’s computational model of the avoidance learning. Psychological Review, 77, 32–48. operant reserve. Behavioural Processes, 75, 71–83. Bones, R., Keenan, M., Askin, D., Adams, L., Taylor, D., & Berko, J. (1958). The child’s learning of English morphology. Nicholas, O. (2001). The effects of response topography Word, 14, 150–177. on functional equivalence class formation. Psychological Bersh, P. J., & Lambert, J. V. (1975). The discriminative control Record, 51, 89–110. of free-operant avoidance despite exposure to shock Boren, J. J., & Devine, D. D. (1968). The repeated acquisition during the stimulus correlated with nonreinforcement. of behavioral chains. Journal of the Experimental Analysis of Journal of the Experimental Analysis of Behavior, 23, 111–120. Behavior, 11, 651–660. Bickel, W. K., Green, L., & Vuchinich, R. E. (1995). Behavioral Boren, M. C. P., & Gollub, L. R. (1972). Accuracy of economics. Journal of the Experimental Analysis of Behavior, performance on a matching-to-sample procedure under 64, 257–262. interval schedules. Journal of the Experimental Analysis of Bijou, S. W. (1966). A functional analysis of retarded Behavior, 18, 65–77. development. International review of research in mental Bottoms, B. L., Najdowski, C. J., & Goodman, G. S. (Eds.). retardation, 1, 1–19. (2009). Children as victims, witnesses, and offenders. New York: Bijou, S. W., & Baer, D. M. (1961). Child development, Vol 1: Guilford. A systematic and empirical theory. Norwalk, CT: Appleton Bowler, P. J. (1983). The eclipse of Darwinism. Baltimore: Johns Century Crofts. Hopkins University Press. Binder, C. (1996). Behavioral fl uency: Evolution of a new Brady, J. V. (2007). Behavior analysis in the space age. The paradigm. The Behavior Analyst, 19, 163–197. Behavior Analyst Today, 8, 398–412. Bix, H. P. (2000). Hirohito and the making of modern Japan. New Brainerd, C. J., Reyna, V. F., & Brandse, E. (1995). Are York: Harper Collins. children’s false memories more persistent than their true Blackman, D. E. (1968). Conditioned suppression or memories? Psychological Science, 6, 359–364. facilitation as a function of the behavioral baseline. Brandauer, C. (1958). The effects of uniform probabilities of Journal of the Experimental Analysis of Behavior, 11, 53–61. reinforcement on the response rate of the pigeon. (doctoral Blackman, D. E. (1977). Conditioned suppression and the dissertation), Columbia University. effects of classical conditioning on operant behavior. Breland, K., & Breland, M. (1951). A fi eld of applied animal In W. K. Honig & J. E. R. Staddon (Eds.), Handbook psychology. American Psychologist, 6, 202–204. of operant behavior (pp. 340–363). Englewood Cliffs, NJ: Breland, K., & Breland, M. (1961). The misbehavior of Prentice-Hall. organisms. American Psychologist, 16, 681–684. Bloom, H. S., Criswell, E. L., Pennypacker, H. S., Catania, A. Bridgman, P. W. (1927). The logic of modern physics. New York: C., & Adams, C. K. (1982). Major stimulus dimensions Macmillan. determining detection of simulated breast lesions. Bright, M. (1985). Animal language. Ithaca, NY: Cornell. Perception and Psychophysics, 32, 251–260. Brown, P. L., & Jenkins, H. M. (1968). Auto-shaping of the Blough, D. S. (1958a). A method for obtaining psychophysical pigeon’s key-peck. Journal of the Experimental Analysis of thresholds from the pigeon. Journal of the Experimental Behavior, 11, 1–8. Analysis of Behavior, 1, 31–43. Brown, R. (1973). A fi rst language. Cambridge, MA: Harvard Blough, D. S. (1958b). New test for tranquillizers. Science, 127, University Press. 586–587. References • 353

Brown, R., & McNeill, D. (1966). The “tip of the tongue” Catania, A. C. (1965). Interocular transfer of discriminations phenomenon. Journal of Verbal Learning and Verbal in the pigeon. Journal of the Experimental Analysis of Behavior, 5, 325–337. Behavior, 8, 147–155. Bruner, C. A., Avila, R. S., Acuña, L., & Gallardo, L. M. Catania, A. C. (1969). Concurrent performances: Inhibition (1998). Effects of reinforcement rate and delay on of one response by reinforcement of another. Journal of the acquisition of lever pressing by rats. Journal of the the Experimental Analysis of Behavior, 12, 731–744. Experimental Analysis of Behavior, 69, 59–75. Catania, A. C. (1970). Reinforcement schedules and Bushell, D., Jr. (1978). An engineering approach to the psychophysical judgments: A study of some temporal elementary classroom: The Behavior Analysis Follow- properties of behavior. In W. N. Schoenfeld (Ed.), The Through Project. In A. C. Catania & T. A. Brigham (Eds.), theory of reinforcement schedules (pp. 1–42). New York: Handbook of applied behavior analysis: Social and instructional Appleton-Century-Crofts. processes (pp. 525–563). New York: Irvington. Catania, A. C. (1971). Reinforcement schedules: The role of Butler, R. A. (1957). The effect of deprivation of visual responses preceding the one that produces the reinforcer. incentives on visual exploration motivation in monkeys. Journal of the Experimental Analysis of Behavior, 15, 271–287. Journal of Comparative and Physiological Psychology, 50, 177– Catania, A. C. (1972). Chomsky’s formal analysis of natural 179. languages: A behavioral translation. Behaviorism, 1, 1–15. Bykov, K. M. (1957). The cerebral cortex and the internal organs (tr. Catania, A. C. (1973a). The concept of the operant in the W. H. Gantt). New York: Chemical Publishing. analysis of behavior. Behaviorism, 1, 103–116. Calkins, M. W. (1894). Association. Psychological Review, 1, 476– Catania, A. C. (1973b). The psychologies of structure, 483. function, and development. American Psychologist, 28, Calkins, M. W. (1896). Association. II. Psychological Review, 3, 434–443. 32–49. Catania, A. C. (1975a). Freedom and knowledge: An Cameron, J., Banko, K. M., & Pierce, W. D. (2001). Pervasive experimental analysis of preference in pigeons. Journal of negative effects of rewards on intrinsic motivation: The the Experimental Analysis of Behavior, 24, 89–106. myth continues. The Behavior Analyst, 24, 1–44. Catania, A. C. (1975b). The myth of self-reinforcement. Camões-Costa, V., Erjavec, M., & Horne, P. J. (2011). The Behaviorism, 3, 192–199. impact of body-part-naming training on the accuracy Catania, A. C. (1976). Self-reinforcement revisited. Behaviorism, of imitative performances in 2– to 3–year-old children. 4, 157–162. Journal of the Experimental Analysis of Behavior, 96, 291–315. Catania, A. C. (1978). The psychology of learning: some Camp, D. S., Raymond, G. A., & Church, R. M. (1967). lessons from the Darwinian revolution. Annals of the New Temporal relationship between response and punishment. York Academy of Sciences, 309, 18–28. Journal of Experimental Psychology, 74, 114–123. Catania, A. C. (1980a). Autoclitic processes and the structure Carr, J. E., Severtson, J. M., & Lepper, T. L. (2009). of behavior. Behaviorism, 8, 175–186. Noncontingent reinforcement is an empirically supported Catania, A. C. (1980b). Freedom of choice: A behavioral treatment for problem behavior exhibited by individuals analysis. In G. H. Bower (Ed.), The psychology of learning and with developmental disabilities. Research in Developmental motivation. Volume 14 (pp. 97–145). New York: Academic Disabilities, 30, 44–57. Press. Carrigan, P. F., Jr., & Sidman, M. (1992). Conditional Catania, A. C. (1981). The fl ight from experimental analysis. discrimination and equivalence relations: A theoretical In C. M. Bradshaw, E. Szabadi, & C. F. Lowe (Eds.), analysis of control by negative stimuli. Journal of the Quantifi cation of steady-state operant behaviour (pp. 49–64). Experimental Analysis of Behavior, 58, 183–204. Amsterdam: Elsevier/North-Holland. Carroll, S. B. (2005). Endless forms most beautiful. New York: Catania, A. C. (1985). The two psychologies of learning: Blind Norton. alleys and nonsense syllables. In S. Koch & D. E. Leary Carroll, S. B. (2006). The making of the fi ttest. New York: (Eds.), A century of psychology as science (pp. 322–335). New Norton. York: McGraw-Hill. Catania, A. C. (1963a). Concurrent performances: A baseline Catania, A. C. (1987). Some Darwinian lessons for behavior for the study of reinforcement magnitude. Journal of the analysis. A review of Peter J. Bowler’s The Eclipse of Experimental Analysis of Behavior, 6, 299–300. Darwinism. Journal of the Experimental Analysis of Behavior, Catania, A. C. (1963b). Concurrent performances: 47, 249–257. Reinforcement interaction and response independence. Catania, A. C. (1989). Speaking of behavior. Journal of the Journal of the Experimental Analysis of Behavior, 6, 252–263. Experimental Analysis of Behavior, 52, 193–196. Catania, A. C. (1964). On the visual acuity of the pigeon. Catania, A. C. (1992). Learning (3rd ed.). Englewood Cliffs, NJ: Journal of the Experimental Analysis of Behavior, 7, 361–366. Prentice-Hall. 354 • References

Catania, A. C. (1993). The unconventional philosophy of Catania, A. C. (2011b). On Baum’s public claim that he science of behavior analysis. Journal of the Experimental has no signifi cant private events: A commentary on Analysis of Behavior, 60, 449–452. “Behaviorism, private events, and the molar view of Catania, A. C. (1995a). Higher-order behavior classes: behavior”. Behavior Analyst, 34, 227–236. Contingencies, beliefs, and verbal behavior. Journal of Catania, A. C. (2012). Chart. The Maddow blog, http:// Behavior Therapy and Experimental Psychiatry, 26, 191–200. maddowblog.msnbc.com/news (Dec 17), 15973646. Catania, A. C. (1995b). Selection in biology and behavior. In Catania, A. C. (2013a). Learning (5th ed.). Croton-on-Hudson, J. T. Todd & E. K. Morris (Eds.), Modern perspectives on NY: Sloan. B. F. Skinner and contemporary behaviorism (pp. 185–194). Catania, A. C. (2013b). A natural science of behavior. Review Westport, CT: Greenwood. of General Psychology, 17, 133–139. Catania, A. C. (1997a). An orderly arrangement of well- Catania, A. C. (2017). Antecedents: How behavior analysis known facts: Retrospective review of B. F. Skinner’s chose me. In R. D. Holdsambeck & H. S. Pennypacker Verbal Behavior. Contemporary Psychology, 42, 967–970. (Eds.), Behavioral science: Tales of inspiration, discovery, and Catania, A. C. (1997b). Remembering Nat Schoenfeld. Behavior service, Omnibus Edition (pp. 541–574). Beverly, MA: Analyst, 20, 31–36. Cambridge Center for Behavioral Studies. Catania, A. C. (2001a). Positive psychology and positive Catania, A. C., & Cerutti, D. (1986). Some nonverbal reinforcement. American Psychologist, 56, 86–87. properties of verbal behavior. In T. Thompson & M. D. Catania, A. C. (2001b). Three varieties of selection and their Zeiler (Eds.), Analysis and integration of behavioral units (pp. implications for the origins of language. In G. Györi 185–211). Hillsdale, NJ: Erlbaum. (Ed.), Language evolution: Biological, linguistic and philosophical Catania, A. C., DeLeon, I. G., & Cataldo, M. F. (2001). perspectives (pp. 55–71). New York/Bern: Peter Lang. A master’s program in applied behavior analysis: Catania, A. C. (2002). The watershed years of 1958–1962 in Contingencies for initiation and maintenance. Behavior the Harvard Pigeon Laboratory. Journal of the Experimental Analysis Today, 1(4), 58–63. Analysis of Behavior, 77, 327–345. Catania, A. C., & Dobson, R. (1972). Concurrent Catania, A. C. (2003a). Verbal governance, verbal shaping, and performances: Rate and accuracy of free-operant oddity attention to verbal stimuli. In K. A. Lattal & P. N. Chase responding. Journal of the Experimental Analysis of Behavior, (Eds.), Behavior theory and philosophy (pp. 301–321). New 17, 25–35. York: Kluwer/Academic Press. Catania, A. C., & Gill, C. A. (1964). Inhibition and behavioral Catania, A. C. (2003b). Why behavior should matter to contrast. Psychonomic Science, 1, 257–258. linguists. Behavioral and Brain Sciences, 26, 670–672. Catania, A. C., Horne, P. J., & Lowe, C. F. (1989). Transfer of Catania, A. C. (2004). Antecedents and consequences of function across members of an equivalence class. Analysis words. European Journal of Behavior Analysis, 5, 55–64. of Verbal Behavior, 7, 99–110. Catania, A. C. (2005a). Attention-defi cit/hyperactivity Catania, A. C., & Keller, K. J. (1981). Contingency, contiguity, disorder (ADHD): Delay-of-reinforcement gradients and correlation, and the concept of causation. In P. Harzem other behavioral mechanisms. Behavioral and Brain Sciences, & M. D. Zeiler (Eds.), Predictability, correlation, and contiguity. 28, 419–424. Advances in analysis of behaviour, Volume 2 (pp. 125–167). Catania, A. C. (2005b). Attention-defi cit/hyperactivity Chichester: Wiley. disorder (ADHD): One process or many? Behavioral and Catania, A. C., Lowe, C. F., & Horne, P. J. (1990). Nonverbal Brain Sciences, 28, 446–450. behavior correlated with the shaped verbal behavior of Catania, A. C. (2005c). The nonmaintenance of behavior by children. Analysis of Verbal Behavior, 8, 43–55. noncontingent reinforcement. European Journal of Behavior Catania, A. C., Matthews, B. A., & Shimoff, E. (1982). Analysis, 6, 89–94. Instructed versus shaped human verbal behavior: Catania, A. C. (2005d). The operant reserve: A computer Interactions with nonverbal responding. Journal of the simulation in (accelerated) real time. Behavioural Processes, Experimental Analysis of Behavior, 38, 233–248. 69, 257–278. Catania, A. C., Matthews, B. A., & Shimoff, E. (1990). Catania, A. C. (2007). Strange visions: Haloes, dust storms Properties of rule-governed behaviour and their and blood on the walls. Skeptical Inquirer, 31(1), 47–50. implications. In D. E. Blackman & H. Lejeune (Eds.), Catania, A. C. (2011a). Model versus model or model Behaviour analysis in theory and practice: Contributions and versus data? A commentary on Berg and McDowell’s controversies (pp. 215–230). Hove & London, UK: Erlbaum. “Quantitative, steady-state properties of Catania’s Catania, A. C., Matthews, T. J., Silverman, P. J., & Yohalem, computational model of the operant reserve.”. Behavioural R. (1977). Yoked variable-interval and variable-ratio Processes, 87, 84–87. responding in pigeons. Journal of the Experimental Analysis of Behavior, 28, 155–161. References • 355

Catania, A. C., Ono, K., & de Souza, D. (2000). Sources of simulations. In A. C. Kamil & T. D. Sargent (Eds.), novel behavior: Stimulus control arranged for different Foraging behavior (pp. 39–76). New York: Garland. response dimensions. European Journal of Behavior Analysis, Conrad, R. (1964). Acoustic confusions in immediate 1, 23–32. memory. British Journal of Psychology Monograph Supplements, Catania, A. C., & Reynolds, G. S. (1968). A quantitative 55, 75–84. analysis of the behavior maintained by interval schedules Cook, R. G., Wright, A. A., & Sands, S. F. (1991). Interstimulus of reinforcement. Journal of the Experimental Analysis of interval and viewing time effects in monkey list memory. Behavior, 11, 327–383. Animal Learning and Behavior, 19, 153–163. Catania, A. C., & Sagvolden, T. (1980). Preference for free Crain, S. (1991). Language acquisition in the absence of choice over forced choice in pigeons. Journal of the experience. Behavioral and Brain Sciences, 14, 597–650. Experimental Analysis of Behavior, 34, 77–86. Crider, A., Schwartz, G. E., & Shnidman, S. (1969). On the Catania, A. C., Sagvolden, T., & Keller, K. J. (1988). criteria for instrumental autonomic conditioning: A reply Reinforcement schedules: retroactive and proactive effects to Katkin and Murray. Psychological Bulletin, 71, 455–461. of reinforcers inserted into fi xed-interval performances. Critchfi eld, T. S. (1993). Signal-detection properties of verbal Journal of the Experimental Analysis of Behavior, 49, 49–73. self-reports. Journal of the Experimental Analysis of Behavior, Catania, A. C., & Shimoff, E. (1998). The experimental 60, 495–514. analysis of verbal behavior. Analysis of Verbal Behavior, 15, Cuvo, A. (2000). Development and function of consequence 97–100. classes in operant behavior. The Behavior Analyst, 23, 57– Catania, A. C., Shimoff, E., & Matthews, B. A. (1989). An 68. experimental analysis of rule-governed behavior. In S. C. Daniels, A. C. (2009). Oops! 13 management practices that waste Hayes (Ed.), Rule-governed behavior: Cognition, contingencies, time and money. Atlanta, GA: Performance Management and instructional control (pp. 119–150). New York: Plenum. Publications. Catania, A. C., Yohalem, R., & Silverman, P. J. (1980). Darwin, C. (1859). On the origin of species by means of natural Contingency and stimulus change in chained schedules selection. London: John Murray. of reinforcement. Journal of the Experimental Analysis of Darwin, C. (1871). The descent of man. London: John Murray. Behavior, 33, 213–219. Darwin, C. (1872). The expression of the emotions in man and Catania, K. C., & Kaas, J. H. (1996). The unusual nose and animals. London: John Murray. brain of the star-nosed mole. BioScience, 46, 578–586. Davis, H., & Pérusse, R. (1988). Numerical competence in Catania, K. C., & Kaas, J. H. (1997). The mole nose instructs animals: Defi nitional issues, current evidence, and a new the brain. Somatosensory and Motor Research, 14, 56–58. research agenda. Behavioral and Brain Sciences, 11, 561–615. Chadwick, P. D. J., Lowe, C. F., Horne, P. J., & Higson, P. Davison, M., & McCarthy, D. (1988). The matching law: A J. (1994). Modifying delusions: The role of empirical research review. Hillsdale, NJ: Erlbaum. testing. Behavior Therapy, 25, 35–49. Dawkins, R. (1976). The selfi sh gene. New York: Oxford Chang, J., & Halliday, J. (2005). Mao: The unknown story. New University Press. York: Knopf. Dawkins, R. (1982). The extended phenotype. San Francisco: Charney, R. (1980). Speech roles and the development of Freeman. personal pronouns. Journal of Child Language, 7, 509–528. Dawkins, R. (1986). The blind watchmaker. New York: Norton. Chase, S. (1938). The tyranny of words. New York: Harcourt, Day, W. F. (1969). On certain similarities between the Brace & World. philosophical investigations of Ludwig Wittgenstein Chenoweth, E., & Stephan, M. J. (2011). Why civil resistance and the operationism of B. F. Skinner. Journal of the works. New York: Columbia University Press. Experimental Analysis of Behavior, 12, 489–506. Chomsky, N. (1959). Review of B. F. Skinner’s Verbal Behavior. de Rose, J. C., McIlvane, W. J., Dube, W. V., Galpin, V. C., & Language, 35, 26–58. Stoddard, L. T. (1988). Emergent simple discrimination Chomsky, N., & Miller, G. A. (1963). Introduction to the established by indirect relation to differential formal analysis of natural languages. In R. D. Luce, R. consequences. Journal of the Experimental Analysis of R. Bush, & E. Galanter (Eds.), Handbook of mathematical Behavior, 50, 1–20. psychology. Volume II (pp. 269–321). New York: Wiley. De Villiers, P. A., & De Villiers, J. G. (1974). On this, that, Chung, S.-H. (1965). Effects of effort on response rate. and the other: Nonegocentrism in very young children. Journal of the Experimental Analysis of Behavior, 8, 1–7. Journal of Experimental Child Psychology, 18, 438–447. Coates, T.-N. (2015). Between the world and me. New York: DeCasper, A. J., & Fifer, W. P. (1980). Of human bonding: Spiegel and Grass. Newborns prefer their mothers’ voices. Science, 208, Collier, G. H., & Rovee-Collier, C. K. (1981). A comparative 1174–1176. analysis of optimal foraging behavior: Laboratory 356 • References degli Espinosa, F. (2011). Verbal behaviour development for children Dreger, A. (2015). Galileo’s middle fi nger. New York: Penguin. with autism. University of Southampton, Southampton. Dreyfus, H. L. (1992). What computers still can’t do. Cambridge, Delgado, L. E., & Lutzger, J. R. (1988). Training young parents MA: MIT Press. to identify and report their children’s illnesses. Journal of Driscoll, C. A., Macdonald, D. W., & O’Brien, S. J. (2009). Applied Behavior Analysis, 21, 311–319. From wild animals to domestic pets, an evolutionary view Delius, J. D., & Nowak, B. (1982). Visual symmetry recognition of domestication. Proceedings of the National Academy of by pigeons. Psychological Research, 44, 199–212. Sciences, 106, 9971–9978. Deluty, M. Z. (1978). Self-control and impulsiveness involving Dube, W. V., MacDonald, R. P. F., Mansfi eld, R. C., Holcomb, aversive events. Journal of Experimental Psychology: Animal W. L., & Ahearn, W. H. (2004). Toward a behavioral Behavior Processes, 4, 250–266. analysis of joint attention. The Behavior Analyst, 27, 197– Dennett, D. C. (1987). The intentional stance. Cambridge, MA: 207. MIT Press. Dube, W. V., & McIlvane, W. J. (1995). Stimulus-reinforcer Derrickson, J. G., Neef, N. A., & Cataldo, M. F. (1993). Effects relations and emergent matching to sample. Psychological of signalling invasive procedures on a hospitalized infant’s Record, 45, 591–612. affective behavior. Journal of Applied Behavior Analysis, 26, Dube, W. V., McIlvane, W. J., Callahan, T. D., & Stoddard, L. T. 133–134. (1993). The search for stimulus equivalence in nonverbal Deutsch, R. (1974). Conditioned hypoglycemia: A mechanism organisms. Psychological Record, 43, 761–778. for saccharin-induced sensitivity to insulin in the rat. Dube, W. V., McIlvane, W. J., Maguire, R. W., Mackay, H. A., Journal of Comparative and Physiological Psychology, 86, 350– & Stoddard, L. T. (1989). Stimulus class formation and 358. stimulus-reinforcer relations. Journal of the Experimental Dews, P. B. (1958). Effects of chlorpromazine and promazine Analysis of Behavior, 51, 51–65. on performance on a mixed schedule of reinforcement. Duncker, K. (1945). On problem solving. Psychological Journal of the Experimental Analysis of Behavior, 1, 73–82. Monographs, 58(5, Whole No. 270). Dews, P. B. (1962). The effect of multiple SΔ periods on Dunham, P. J. (1977). The nature of reinforcing stimuli. In W. responding on a fi xed-interval schedule. Journal of the K. Honig & J. E. R. Staddon (Eds.), Handbook of operant Experimental Analysis of Behavior, 5, 369–374. behavior (pp. 98–124). Englewood Cliffs, NJ: Prentice- Dews, P. B. (1970). Drugs in psychology. A commentary on T. Hall. Thompson & C. R. Schuster’s Behavioral Pharmacology. Dworkin, B. R., & Miller, N. E. (1986). Failure to replicate Journal of the Experimental Analysis of Behavior, 13, 395–406. visceral learning in the acute curarized rat preparation. Diagnostic and Statistical Manual of Mental Disorders (DSM-IV- Behavioral Neuroscience, 100, 299–314. TR). (2000). (4th, Text Revision ed.). Washington, DC: Ebbinghaus, H. (1885/1964). Über das Gedächtnis (Memory, American Psychiatric Association. tr. H. A. Ruger & C. E. Bussenius). Leipzig/New York: Dinsmoor, J. A. (1983). Observing and conditioned Duncker & Humblot/Dover. reinforcement. Behavioral and Brain Sciences, 6, 693–728. Eckerman, D. A., Hienz, R. D., Stern, S., & Kowlowitz, V. Dinsmoor, J. A. (1985). The role of observing and attention (1980). Shaping the location of a pigeon’s peck: Effect of in establishing stimulus control. Journal of the Experimental rate and size of shaping steps. Journal of the Experimental Analysis of Behavior, 43(3), 365–381. Analysis of Behavior, 33, 299–310. Dinsmoor, J. A. (1989). Selective observing: Pigeons turn Edelman, G. M. (1987). Neural Darwinism. New York: Basic discriminative stimuli on and off by pecking separate Books. keys. In N. W. Bond & D. A. T. Siddle (Eds.), Psychobiology: Eimas, P. D., & Miller, J. L. (1992). Organization in the Issues and Applications (pp. 443–456). North-Holland: perception of speech by young infants. Psychological Science, Elsevier. 3, 340–345. Dinsmoor, J. A. (1995). Stimulus control. The Behavior Analyst, Eimas, P. D., Siqueland, E. R., Jusczyk, P., & Vigorito, J. (1971). 18, 51–68 and 253–269. Speech perception in early infancy. Science, 171, 303–306. Donahoe, J. W., & Palmer, D. C. (1994). Learning and complex Eisenberger, R., & Cameron, J. (1996). Detrimental effects of behavior. Boston: Allyn and Bacon. reward: Reality or myth? American Psychologist, 51, 1153– Donahoe, J. W., & Vegas, R. (2004). Pavlovian Conditioning: 1166. The CS-UR Relation. Journal of Experimental Psychology: Epstein, R. (1981). On pigeons and people: A preliminary Animal Behavior Processes, 30, 17–33. look at the Columban Simulation Project. Behavior Analyst, Dorsey, M. F., Iwata, B. A., Ong, P., & McSween, T. E. (1980). 4, 43–55. Treatment of self-injurious behavior using a water mist: Epstein, R., Lanza, R. P., & Skinner, B. F. (1980). Symbolic Initial response suppression and generalization. Journal of communication between two pigeons (Columba livia Applied Behavior Analysis, 13, 343–353. domestica). Science, 207, 543–545. References • 357

Epstein, R., Lanza, R. P., & Skinner, B. F. (1981). “Self- Ferster, C. B. (1960). Intermittent reinforcement of matching awareness” in the pigeon. Science, 212, 695–696. to sample in the pigeon. Journal of the Experimental Analysis Epstein, R., & Skinner, B. F. (1980). Resurgence of of Behavior, 3, 259–272. responding after the cessation of response-independent Ferster, C. B. (1978). Is operant conditioning getting bored reinforcement. Proceedings of the National Academy of with behavior? Journal of the Experimental Analysis of Sciences, 77, 6251–6253. Behavior, 29, 347–349. Escobar, R., & Bruner, C. A. (2007). Response induction Ferster, C. B., & DeMyer, M. K. (1961). The development during the acquisition and maintenance of lever pressing of performances in autistic children in an automatically with delayed reinforcement. Journal of the Experimental controlled environment. Journal of Chronic Diseases, 13, Analysis of Behavior, 88, 29–49. 312–345. Esper, E. A. (1973). Analogy and association in linguistics and Ferster, C. B., & Skinner, B. F. (1957). Schedules of reinforcement. psychology. Athens, GA: University of Georgia Press. New York: Appleton-Century-Crofts. Estes, W. K., & Skinner, B. F. (1941). Some quantitative Fetterman, J. G. (1996). Dimensions of stimulus complexity. properties of anxiety. Journal of Experimental Psychology, 29, Journal of Experimental Psychology: Animal Behavior Processes, 390–400. 22, 3–18. Etzioni, A. (2003). The monochrome society. Princeton: Princeton Field, D. P., & Hineline, P. N. (2008). Dispositioning and the University Press. obscured roles of time in psychological representations. Evans, R. J. (2004). The coming of the Third Reich. New York: Behavior and Philosophy, 36, 5–69. Penguin. Fields, L., Adams, B. J., Verhave, T., & Newman, S. (1990). Evans, R. J. (2005). The Third Reich in power. New York: Penguin. The effects of nodality on the formation of equivalence Evans, R. J. (2009). The Third Reich at war. New York: Penguin. classes. Journal of the Experimental Analysis of Behavior, 53, Eyre, H. L. (2007). Keller’s Personalized System of Instruction: 345–358. Was it a fl eeting fancy or is there a revival on the horizon? Fields, L., Arntzen, E., Nartey, R. K., & Eilifsen, C. (2012). The Behavior Analyst Today, 8, 317–324. Effects of a meaningful, a discriminative, and a Falk, J. L. (1977). The origin and functions of adjunctive meaningless stimulus on equivalence class formation. behavior. Animal Learning and Behavior, 5, 325–335. Journal of the Experimental Analysis of Behavior, 97, 163–181. Fantino, E., & Abarca, N. (1985). Choice, optimal foraging, Fields, L., Landon-Jimenez, D. V., Buffi ngton, D., M., & and the delay-reduction hypothesis. Behavioral and Brain Adams, B. J. (1995). Maintained nodal-distance effects in Sciences, 8, 315–330. equivalence classes. Journal of the Experimental Analysis of Favell, J. E., McGimsey, J. F., & Jones, M. L. (1978). The use Behavior, 64, 129–145. of physical restraint in the treatment of self-injury and as Fields, L., Tittelbach, D., Shamoun, K., Watanabe, M., Fitzer, positive reinforcement. Journal of Applied Behavior Analysis, A., & Matneja, P. (2007). The effects of differential 11, 225–241. training procedures on linked perceptual class formation. Fearing, F. (1930). Refl ex action. Baltimore: Williams and Journal of the Experimental Analysis of Behavior, 87, 97–119. Wilkins. Fields, L., & Verhave, T. (1987). The structure of equivalence Fedorchak, P. M., & Bolles, R. C. (1986). Differential outcome classes. Journal of the Experimental Analysis of Behavior, 48, effect using a biologically neutral outcome difference. 317–332. Journal of Experimental Psychology: Animal Behavior Processes, Findley, J. D. (1962). An experimental outline for building and 12, 125–130. exploring multioperant behavior repertoires. Journal of the Felton, M., & Lyon, D. O. (1966). The post-reinforcement Experimental Analysis of Behavior, 5, 113–166. pause. Journal of the Experimental Analysis of Behavior, 9, Findley, J. D., & Brady, J. V. (1965). Facilitation of large ratio 131–164. performance by use of conditioned reinforcement. Fennelly, M., Galletly, D. C., & Purdie, G. I. (1991). Is caffeine Journal of the Experimental Analysis of Behavior, 8, 125–129. withdrawal the mechanism of postoperative headache? Fisher, W. W., & Bouxsein, K. (2011). Developing function- Anesthesia and Analgesia, 72, 449–453. based reinforcement procedures for problem behavior. Ferrari, M., & Harris, S. L. (1981). The limits and motivating In W. W. Fisher, C. C. Piazza, & H. S. Roane (Eds.), potential of sensory stimuli as reinforcers for autistic Handbook of applied behavior analysis (pp. 335–347). New children. Journal of Applied Behavior Analysis, 14, 339–343. York: Guilford. Ferster, C. B. (1958). Control of behavior in chimpanzees Fisher, W. W., Piazza, C. C., Bowman, L. G., Hagopian, L. P., and pigeons by time out from positive reinforcement. Owens, J. C., & Slevin, I. (1992). A comparison of two Psychological Monographs, 72(8, Whole No. 461). approaches for identifying reinforcers for persons with severe and profound disabilities. Journal of Applied Behavior Analysis, 25, 491–498. 358 • References

Fisher, W. W., Piazza, C. C., & Roane, H. S. (Eds.). (2011). Garcia, J., & Koelling, R. A. (1966). Relation of cue to Handbook of applied behavior analysis. New York: Guilford. consequence in avoidance learning. Psychonomic Science, 4, Flanagan, B., Goldiamond, I., & Azrin, N. H. (1958). 123–124. Operant stuttering: The control of stuttering behavior Gardner, R. A., & Gardner, B. T. (1969). Teaching sign through reponse-contingent consequences. Journal of the language to a chimpanzee. Science, 165, 664–672. Experimental Analysis of Behavior, 1, 173–177. Garwood, C. (2007). Flat earth. New York: St. Martin’s Press. Flora, S. R., & Flora, D. B. (1999). Effects of extrinsic Gellately, R. (2007). Lenin, Stalin, and Hitler: The age of social reinforcement for reading during childhood on reported catastrophe. New York: Knopf. reading habits of college students. Psychological Record, 49, Geller, I. (1960). The acquisition and extinction of conditioned 3–14. suppression as a function of the base-line reinforcer. Follman, M., Aronsen, G., & Pan, D. (2017). US mass Journal of the Experimental Analysis of Behavior, 3, 235–240. shootings, 1982–2017: Data From Mother Jones’ Gibson, E. J. (1965). Learning to read. Science, 148, 1066–1072. investigation. Mother Jones, http://www.motherjones. Gilbert, T. F. (1978). Human competence. New York: McGraw- com/politics/2012/2012/mass-shootings-mother-jones- Hill. full-data. Gillan, D. J. (1981). Reasoning in the chimpanzee: II. Foss, D. J. (1988). Experimental psycholinguistics. Annual Transitive inference. Journal of Experimental Psychology: Review of Psychology, 39, 301–348. Animal Behavior Processes, 7, 150–164. Fowler, H., & Trapold, M. A. (1962). Escape performance as a Gleick, J. (1987). Chaos. New York: Viking. function of delay of reinforcement. Journal of Experimental Gleitman, L. R., & Rozin, P. (1973). Teaching reading by use Psychology, 63, 464–467. of a syllabary. Reading Research Quarterly, 8, 447–483. Fraenkel, G. S., & Gunn, D. L. (1961). The orientation of animals. Goh, H.-L., Iwata, B. A., & DeLeon, I. G. (2000). Competition New York: Dover. between noncontingent and contingent reinforcement Francisco, M. T., Borrero, J. C., & Sy, J. R. (2008). Evaluation of schedules during response acquisition. Journal of Applied absolute and relative reinforcer value using progressive- Behavior Analysis, 33, 195–205. ratio schedules. Journal of Applied Behavior Analysis, 41, Goldiamond, I. (1966). Perception, language, and 189–202. conceptualization rules. In B. Kleinmuntz (Ed.), Problem Frey, P. W. (Ed.) (1977). Chess skill in man and machine. New solving (pp. 183–224). New York: Wiley. York: Springer-Verlag. Goldiamond, I. (1974). Toward a constructional approach to Friedlander, B. Z. (1968). The effect of speaker identity, social problems: ethical and constitutional issues raised voice infl ection, vocabulary, and message redundancy by applied behavior analysis. Behaviorism, 2, 1–84. on infants’ selection of vocal reinforcement. Journal of Goldiamond, I., & Thompson, D. M. (2002). A functional Experimental Child Psychology, 6, 443–459. analysis of behavior and its extensions. In P. T. Andronis Friman, P. C., Hayes, S. C., & Wilson, K. G. (1998). Why (Ed.), The Blue Books: Goldiamond’s functional analysis of behavior analysts should study emotion: The example of behavior (CD). Concord, MA: Cambridge Center for anxiety. Journal of Applied Behavior Analysis, 31, 137–156. Behavioral Studies. Friman, P. C., & Piazza, C. C. (2011). Behavioral pediatrics: Gollub, L. R. (1977). Conditioned reinforcement: Schedule Integrating applied behaivor analysis with pediatric effects. In W. K. Honig & J. E. R. Staddon (Eds.), medicine. In W. W. Fisher, C. C. Piazza, & H. S. Roane Handbook of operant behavior (pp. 288–312). Englewood (Eds.), Handbook of applied behavior analysis (pp. 433–450). Cliffs, NJ: Prentice Hall. New York: Guilford. Goodman, G. S. (2006). Children’s eyewitness memory: A Fromkin, V. A. (1971). The non-anomalous nature of modern history and contemporary commentary. Journal anomalous utterances. Language, 47, 27–52. of Social Issues, 62, 811–832. Galef, B. G., Jr., & Stein, M. (1985). Demonstrator infl uence Gormezano, I. (1972). Investigations of defense and reward on observer diet preference: Analysis of critical social conditioning in the rabbit. In A. H. Black & W. F. Prokasy interactions and olfactory signals. Animal Learning and (Eds.), Classical conditioning II (pp. 151–181). New York: Behavior, 13, 31–38. Appleton-Century-Crofts. Gallistel, C. R. (1980). The organization of action. Hillsdale, NJ: Gould, S. J. (1996). Full house. New York: Harmony Books. Erlbaum. Gouzoules, S., Gouzoules, H., & Marler, P. (1984). Rhesus Gallistel, C. R. (1990). The organization of learning. Cambridge, monkey (Macaca mulatta) screams: Representational MA: MIT Press. signalling in the recruitment of agonistic aid. Animal Galton, F. (1879). Psychometric experiments. Brain, 2, 149– Behaviour, 32, 182–193. 162. Graefe, T. M., & Watkins, M. J. (1980). Picture rehearsal: An effect of selectively attending to pictures no longer in References • 359

view. Journal of Experimental Psychology: Human Learning and aspects of behavioral development. Annual Review of Memory, 6, 156–162. Psychology, 38, 91–128. Gray, J. (1953). How animals move. London: Cambridge Hampl, K. F., Schneider, M. C., Rüttiman, U., Ummenhofer, University Press. W., & Drewe, J. (1995). Perioperative administration Green, C. W., Reid, D. H., White, L. K., Halford, R. C., Brittain, of caffeine tablets for prevention of postoperative D. P., & Gardner, S. M. (1988). Identifying reinforcers for headaches. Canadian Journal of Anesthesia, 42, 789–792. persons with profound handicaps: Staff opinion versus Harlow, H. F. (1949). The formation of learning sets. systematic assessment of preferences. Journal of Applied Psychological Review, 56, 51–65. Behavior Analysis, 21, 31–43. Harnad, S. (2003). Categorical perception. In L. Nadel (Ed.), Green, D. M., & Swets, J. A. (1966). Signal detection theory and Encyclopedia of cognitive science. New York: Macmillan. psychophysics. New York: Wiley. Harris, A. H., & Turkkan, J. S. (1981). Generalization of Green, L., & Myerson, J. (2004). A discounting framework for conditioned blood pressure elevations: Schedule and choice with delayed and probabilistic rewards. Psychological stimulus control effects. Physiology and Behavior, 26, 935– Bulletin, 130, 769–792. 940. Greenspoon, J. (1955). The reinforcing effect of two spoken Harris, M. (1977). Cannibals and kings. New York: Random sounds on the frequency of two responses. American House. Journal of Psychology, 68, 409–416. Harris, R. A. (1993). The linguistic wars. New York: Oxford. Greer, R. D., & McDonough, S. H. (1999). Is the learn unit a Hart, B. M., Reynolds, N. J., Baer, D. M., Brawley, E. R., & fundamental measure of pedagogy? Behavior Analyst, 22, Harris, F. R. (1968). Effect of contingent and non- 5–16. contingent social reinforcement on the cooperative play Greer, R. D., & Ross, D. E. (2004). Verbal behavior analysis: of a preschool child. Journal of Applied Behavior Analysis, A program of research in the induction and expansion 1, 73–76. of complex verbal behavior. Journal of Early and Intensive Hart, B., & Risley, T. R. (1980). In vivo language intervention: Behavior Intervention, 1, 141–165. Unanticipated general effects. Journal of Applied Behavior Grosch, J., & Neuringer, A. (1981). Self-control in pigeons Analysis, 13, 407–432. under the Mischel paradigm. Journal of the Experimental Hart, B., & Risley, T. R. (1995). Meaningful differences in the Analysis of Behavior, 35, 3–21. everyday experience of young American children. Baltimore: Paul Gurteen, P. M., Horne, P. J., & Erjavec, M. (2011). Rapid H. Brookes. word learning in 13– and 17–month-olds in a naturalistic Hart, B. L. (1973). Refl exive behavior. In G. Berman (Ed.), two-word procedure: Looking versus reaching measures. Perspectives on animal behavior (pp. 171–193). Glenview, IL: Journal of Experimental Child Psychology, 109, 201–217. Scott, Foresman. Guthrie, E. R. (1935). The psychology of learning. New York: Hartline, H. K., & Ratliff, F. (1957). Inhibitory interaction of Harper. receptor units in the eye of Limulus. Journal of General Guthrie, E. R., & Horton, G. P. (1946). Cats in a puzzle box. Physiology, 40, 357–673. New York: Rinehart. Hartline, H. K., Wagner, H. G., & Ratliff, F. (1956). Inhibition Hackenberg, T. D. (1992). Choice in situations of time- in the eye of Limulus. Journal of General Physiology, 39, based diminishing returns: Immediate versus delayed 651–673. consequences of action. Journal of the Experimental Analysis Hartman, A. M. (1975). Analysis of conditions leading to the of Behavior, 57, 67–80. regulation of water fl ow by a beaver. Psychological Record, Hagopian, L. P., Crockett, J. L., van Stone, M., DeLeon, I. 25, 427–431. G., & Bowman, L. G. (2000). Effects of noncontingent Hayes, K., & Hayes, C. (1951). The intellectual development reinforcement on problem behavior and stimulus of a home-raised chimpanzee. Proceedings of the American engagement: The role of satiation, extinction, and Philosophical Society, 95, 105–109. alternative reinforcement. Journal of Applied Behavior Hayes, S. C. (1994). Relational frame theory: A functional Analysis, 33, 433–449. approach to verbal events. In S. C. Hayes, L. J. Hayes, Hailman, J. P. (1969). How an instinct is learned. Scientifi c M. Sato, & K. Ono (Eds.), Behavior analysis of language and American, 221(6), 98–106. cognition (pp. 9–30). Reno, NV: Context Press. Hall, R. V., Lund, D., & Jackson, D. (1968). Effects of teacher Hayes, S. C., Barnes-Holmes, D., & Roche, B. (2001). Relational attention on study behavior. Journal of Applied Behavior frame theory: A post-Skinnerian account of human language and Analysis, 1, 1–12. cognition. New York: Kluwer Academic/Plenum. Hall, W. G., & Oppenheim, R. W. (1987). Developmental Hayes, S. C., & Wilson, K. G. (1994). Acceptance and psychobiology: Prenatal, perinatal, and early postnatal commitment therapy: Altering the verbal support for experiential avoidance. The Behavior Analyst, 17, 289–303. 360 • References

Hayes, S. C., Zettle, R. D., & Rosenfarb, I. (1989). Rule- maintained operant behavior. Journal of Comparative and following. In S. C. Hayes (Ed.), Rule-governed behavior: Physiological Psychology, 51, 461–467. Cognition, contingencies, and instructional control (pp. 191–220). Herrnstein, R. J., & Sidman, M. (1958). Avoidance conditioning New York: Plenum. as a factor in the effects of unavoidable shocks on food- Hearst, E. (1958). The behavioral effects of some temporally reinforced behavior. Journal of Comparative and Physiological defi ned schedules of reinforcement. Journal of the Psychology, 51, 380–385. Experimental Analysis of Behavior, 1, 45–55. Hilgard, E. R. (1986). Divided consciousness: Multiple controls in Hearst, E., & Jenkins, H. M. (1974). Sign-tracking: The human thought and action (Expanded ed.). New York: Wiley. stimulus-reinforcer relation and directed action. Austin, TX: Hineline, P. N. (1970). Negative reinforcement without shock Psychonomic Society. reduction. Journal of the Experimental Analysis of Behavior, Hebb, D. O. (1949). The organization of behavior. New York: 14, 259–268. Wiley. Hineline, P. N. (1977). Negative reinforcement and avoidance. Hebb, D. O. (1956). The distinction between “classical” and In W. K. Honig & J. E. R. Staddon (Eds.), Handbook of “instrumental.”. Canadian Journal of Psychology, 10, 165– operant behavior (pp. 364–414). Englewood Cliffs, NJ: 166. Prentice-Hall. Hefferline, R. F. (1958). The role of proprioception in the Hineline, P. N. (1981). The several roles of stimuli in negative control of behavior. Transactions of the New York Academy reinforcement. In P. Harzem & M. D. Zeiler (Eds.), of Sciences, 20, 739–764. Predictability, correlation, and contiguity (pp. 203–246). New Hefferline, R. F., & Perera, T. B. (1963). Proprioceptive York: Wiley. discrimination of a covert operant without Its observation Hineline, P. N. (1990). The origins of environment-based by the subject. Science, 139, 834–835. psychological theory. Journal of the Experimental Analysis of Held, R., & Hein, A. (1963). Movement-produced stimulation Behavior, 53, 305–320. in the development of visually guided behavior. Journal of Hineline, P. N. (2001). Beyond the molar-molecular Comparative and Physiological Psychology, 56, 872–876. distinction: We need multiscaled analyses. Journal of the Hemmes, N. S. (1973). Behavioral contrast in pigeons depends Experimental Analysis of Behavior, 75, 342–347. upon the operant. Journal of Comparative and Physiological Hineline, P. N. (2016). Narrative – A Major Gap in Our Account Psychology, 85, 171–178. of Verbal Behavior. In the Symposium: Future Research Topics Herman, L. M., & Forestell, P. H. (1985). Reporting presence in Verbal Behavior: Meaning, Narrative, Unique Infl uences and or absence of named objects by a language-trained Voluntary Control. Paper presented at the Association for dolphin. Neuroscience and Biobehavioral Reviews, 9, 667–681. Behavior Analysis International, Chicago. Herrick, R. M., Myers, J. L., & Korotkin, A. L. (1959). Hinson, J. M., & Staddon, J. E. R. (1981). Maximizing on Changes in SD and in SΔ rates during the development interval schedules. In C. M. Bradshaw, E. Szabadi, & C. of an operant discrimination. Journal of Comparative and F. Lowe (Eds.), Quantifi cation of steady-state operant behavior Physiological Psychology, 52, 359–364. (pp. 35–47). Amsterdam: Elsevier/North-Holland. Herrnstein, R. J. (1961). Relative and absolute strength of Hirsh, M. (2010). The liberators: America’s witnesses to the Holocaust. response as a function of frequency of reinforcement. New York: Bantam Books. Journal of the Experimental Analysis of Behavior, 4, 267–272. Hodges, A. (1985). Alan Turing: The enigma of intelligence. Herrnstein, R. J. (1964a). Aperiodicity as a factor in choice. London: Unwin. Journal of the Experimental Analysis of Behavior, 7, 179–182. Hoffman, H. S. (1996). Amorous turkeys and addicted ducklings: Herrnstein, R. J. (1964b). Secondary reinforcement and rate A search for the causes of social attachment. Boston: Authors of primary reinforcement. Journal of the Experimental Cooperative. Analysis of Behavior, 7, 27–36. Hoffman, H. S., & Fleshler, M. (1959). Aversive control with Herrnstein, R. J. (1970). On the law of effect. Journal of the the pigeon. Journal of the Experimental Analysis of Behavior, Experimental Analysis of Behavior, 13, 243–266. 2, 213–218. Herrnstein, R. J., & Hineline, P. N. (1966). Negative Hoffman, H. S., & Fleshler, M. (1962). The course of reinforcement as shock-frequency reduction. Journal of emotionality in the development of avoidance. Journal of the Experimental Analysis of Behavior, 9, 421–430. Experimental Psychology, 64, 288–294. Herrnstein, R. J., Loveland, D. H., & Cable, C. (1976). Natural Holland, J. G. (1958). Human vigilance. Science, 128, 61–67. concepts in pigeons. Journal of Experimental Psychology: Hollard, V. D., & Delius, J. D. (1982). Rotational invariance in Animal Behavior Processes, 2, 285–311. visual pattern recognition by pigeons and humans. Science, Herrnstein, R. J., & Morse, W. H. (1957). Some effects 218, 804–806. of response-independent positive reinforcement on References • 361

Holz, W. C., & Azrin, N. H. (1961). Discriminative properties procedural form and function. Journal of Applied Behavior of punishment. Journal of the Experimental Analysis of Analysis, 27, 131–144. Behavior, 4, 225–232. Iwata, B. A., Pace, G. M., Kalsher, M. J., Cowdery, G. E., Honig, W. K., Boneau, C. A., Burstein, K. R., & Pennypacker, & Cataldo, M. F. (1990). Experimental analysis and H. S. (1963). Positive and negative generalization gradients extinction of self-injurious escape behavior. Journal of obtained after equivalent training conditions. Journal of Applied Behavior Analysis, 23, 11–27. Comparative and Physiological Psychology, 56, 111–116. Jackendoff, R. (2002). Foundations of Language. Oxford: Oxford Hooper, J. (2002). Of moths and men: An evolutionary tale. New University Press. York: Norton. James, W. (1890). The principles of psychology. Volume 1. New Horne, P. J., & Erjavec, M. (2007). Do infants show generalized York: Holt. imitation of gestures? Journal of the Experimental Analysis Jaynes, J. (1976). The origin of consciousness in the breakdown of the of Behavior, 87, 63–87. bicameral mind. Boston, MA: Houghton Miffl in. Horne, P. J., Hughes, J. C., & Lowe, C. F. (2006). Naming Jenkins, H. M., & Harrison, R. H. (1960). Effect of and categorization in young children: IV. Listener discrimination training on auditory generalization. Journal behavior training and transfer of function. Journal of the of Experimental Psychology, 59, 246–253. Experimental Analysis of Behavior, 85, 247–373. Jenkins, H. M., & Moore, B. R. (1973). The form of the auto- Horne, P. J., & Lowe, C. F. (1996). On the origins of naming shaped response with food or water reinforcers. Journal of and other symbolic behavior. Journal of the Experimental the Experimental Analysis of Behavior, 20, 163–181. Analysis of Behavior, 65, 185–241. Jenkins, H. M., & Sainsbury, R. S. (1970). Discrimination Horne, P. J., Lowe, C. F., & Harris, F. D. A. (2007). Naming learning with the distinctive feature on positive or negative and categorization in young children: V. Manual sign trials. In D. I. Mostofsky (Ed.), Attention: Contemporary training. Journal of the Experimental Analysis of Behavior, 87, theory and analysis (pp. 239–273). New York: Appleton- 367–381. Century-Crofts. Horne, P. J., Lowe, C. F., & Randle, V. R. L. (2004). Naming Jennings, H. S. (1906). Behavior of the lower organisms. New York: and categorization in young children: II. Listener behavior Macmillan. training. Journal of the Experimental Analysis of Behavior, 81, Johnson, K. R., & Layng, T. V. J. (1992). Breaking the 267–288. structuralist barrier: Literacy and numeracy with fl uency. Hubel, D. H., & Wiesel, T. N. (2005). Brain and visual perception. American Psychologist, 47, 1475–1490. New York: Oxford. Johnson, P. (1982). The functional equivalence of imagery Hübner, M. M. C., Austin, J. L., & Miguel, C. F. (2008). Effects and movement. Quarterly Journal of Experimental Psychology, of praising qualifying autoclitics on the frequency of 34A, 349–365. reading. Analysis of Verbal Behavior, 24, 55–62. Johnston, J. M. (2000). Behavior analysis and the R&D Hull, C. L. (1920). Quantitative aspects of the evolution of paradigm. The Behavior Analyst, 23, 141–148. concepts: An experimental study. Psychological Monographs, Johnston, J. M., & Pennypacker, H. S. (1993). Strategies and tactics 28, No. 123. of behavioral research (2nd ed.). Hillsdale, NJ: Erlbaum. Hull, C. L. (1943). Principles of behavior. New York: Appleton- Jonçich, G. (1968). The sane positivist. A biography of Edward L. Century-Crofts. Thorndike. Middletown, CT: Wesleyan University Press. Hursh, S. (1991). Behavioral economics of drug self- Jones, M. (2004, Sept 19). Who was abused. New York Times administration. Journal of the Experimental Analysis of Magazine. Behavior, 56, 377–393. Kahneman, D. (2011). Thinking, fast and slow. New York: Huxley, R. (1970). The development of the correct use of Farrar, Straus and Giroux. subject personal pronouns in two children. In G. B. F. Kamil, A. C., Yoerg, S. I., & Clements, K. C. (1988). Rules d’Arcais & W. J. M. Levelt (Eds.), Advances in psycholinguistics to leave by: Patch departure in foraging blue jays. Animal (pp. 141–165). Amsterdam: Elsevier/North-Holland. Behaviour, 36, 843–853. Irvine, K. D., & Rauskolb, C. (2001). Boundaries in Kamin, L. J. (1956). The effects of termination of the CS development: Formation and function. Annual Review of and avoidance of the US on avoidance learning. Journal of Cell Developmental Biology, 17, 189–214. Comparative and Physiological Psychology, 49, 420–424. Ison, J. R., & Hoffman, H. S. (1983). Refl ex modifi cation in Kastak, C. R., & Schusterman, R. J. (2002). Sea lions and the domain of startle: II. The anomalous history of a equivalence: Expanding classes by exclusion. Journal of the robust and ubiquitous phenomenon. Psychological Bulletin, Experimental Analysis of Behavior, 78, 449–465. 94, 3–17. Kastak, C. R., Schusterman, R. J., & Kastak, D. (2001). Iwata, B. A., Pace, G. M., Cowdery, G. E., & Miltenberger, R. Equivalence classifi cation by California sea lions using G. (1994). What makes extinction work: An analysis of 362 • References

class-specifi c reinforcers. Journal of the Experimental Killeen, P. R. (2004). Minding behavior. Behavior and Philosophy, Analysis of Behavior, 76, 131–158. 32, 125–147. Katz, J. S., & Wright, A. A. (2006). Same/different abstract- Killeen, P. R. (2011). Markov model of smoking cessation. concept learning by pigeons. Journal of Experimental Proceedings of the National Academy of Sciences, 108, 15549– Psychology: Animal Behavior Processes, 32, 80–86. 15556. Kawamura, S. (1959). The process of sub-culture propagation Killeen, P. R., & Amsel, A. (1987). The kinematics of among Japanese macaques. Primates, 2, 43–60. locomotion toward a goal. Journal of Experimental Kay, P., & Kempton, W. (1984). What is the Sapir-Whorf Psychology: Animal Behavior Processes, 13, 92–101. hypothesis? American Anthropologist, 86, 65–79. Killeen, P. R., & Bizo, L. A. (1998). The mechanics of Kelleher, R. T., Goldberg, S. R., & Krasnegor, N. A. (1975). reinforcement. Psychonomic Bulletin and Review, 5, 221–238. Symposium on control of drug-taking behavior by Killeen, P. R., & Hall, S. S. (2001). The principle components schedules of reinforcement. Pharmacological Reviews, 27, of response strength. Journal of the Experimental Analysis of 289–548. Behavior, 75, 111–134. Kelleher, R. T., & Gollub, L. R. (1962). A review of positive Killeen, P. R., & Jacobs, K. W. (2016). Coal is not black, conditioned reinforcement. Journal of the Experimental snow is not white, food is not a reinforcer: The roles of Analysis of Behavior, 5, 543–597. affordances and dispositions in the analysis of behavior. Kelleher, R. T., Riddle, W. C., & Cook, L. (1962). Observing Behavior Analyst. doi:10.1007/s40614–016–0080–7 responses in pigeons. Journal of the Experimental Analysis Killeen, P. R., & Sitomer, M. T. (2003). MPR. Behavioural of Behavior, 5, 3–13. Processes, 62, 49–64. Keller, F. S. (1941). Light aversion in the white rat. Psychological Killeen, P. R., Tannock, R., & Sagvolden, T. (2011). The four Record, 4, 235–250. causes of ADHD: A framework. In C. Stanford & R. Keller, F. S. (1968). “Goodbye, teacher...”. Journal of Applied Tannock (Eds.), Behavioral neuroscience of attention defi cit Behavior Analysis, 1, 79–89. hyperactivity disorder and its treatment (pp. 392–425). Berlin: Keller, F. S. (2009). At my own pace: The autobiography of Fred S. Springer-Verlag. Keller. Cornwall-on-Hudson, NY: Sloan. Kim, H., & Bao, S. (2008). Distributed representation of Keller, F. S., & Schoenfeld, W. N. (1950). Principles of psychology. perceptual categories in the auditory cortex. Journal of New York: Appleton-Century-Crofts. Computational Neuroscience, 24, 277–290. Keller, K. J. (1974). The role of elicited responding in Kimble, G. A. (1947). Conditioning as a function of the time behavioral contrast. Journal of the Experimental Analysis of between conditioned and unconditioned stimuli. Journal Behavior, 21, 249–257. of Experimental Psychology, 37, 1–15. Kendall, S. B. (1965). Spontaneous recovery after extinction Kimmel, H. D. (1976). Notes from “Pavlov’s Wednesdays”: with periodic time-outs. Psychonomic Science, 2, 117–118. Pavlov’s law of effect. American Journal of Psychology, 89, Kershaw, I. (1998). Hitler 1889–1936: Hubris. New York: 553–556. Norton. King, A. P., & West, M. J. (1985). Social metrics of song Kershaw, I. (2000). Hitler 1936–1945: Nemesis. New York: learning. Learning and Motivation, 15, 441–458. Norton. Kirschner, M. W., & Gerhart, J. C. (2005). The plausibility of Kiecker, C., & Lumsden, A. (2005). Compartments and their life: Resolving Darwin’s dilemma. New Haven: Yale University boundaries in vertebrate brain development. Nature Press. Reviews Neuroscience, 6, 553–564. Kish, G. B. (1966). Studies of sensory reinforcement. In W. Kiecker, C., & Lumsden, A. (2009). Recent advances in neural K. Honig (Ed.), Operant Behavior: Areas of research and development. Biology Reports, 1, online journal. application (pp. 109–159). New York: Appleton-Century- Kiesler, C. A., Nisbett, R. E., & Zanna, M. P. (1969). On Crofts. inferring one’s beliefs from one’s behavior. Journal of Kluender, K. R., Diehl, R. L., & Killeen, P. R. (1987). Japanese Personality and Social Psychology, 11, 321–327. quail can learn phonetic categories. Science, 237, 1195– Killeen, P. R. (1972). A yoked-chamber comparison of 1197. concurrent and multiple schedules. Journal of the Knapp, T. J. (1992). Verbal Behavior: The other reviews. Experimental Analysis of Behavior, 18, 13–22. Analysis of Verbal Behavior, 10, 87–95. Killeen, P. R. (1988). The refl ex reserve. Journal of the Köhler, W. (1927). The mentality of apes (tr. E. Winter; 2nd revised Experimental Analysis of Behavior, 50, 319–331. edition). London: Routledge & Kegan Paul. Killeen, P. R. (1992). Mechanics of the animate. Journal of the Kolers, P. A. (1966). Reading and talking bilingually. American Experimental Analysis of Behavior, 57, 429–463. Journal of Psychology, 79, 357–376. Killeen, P. R. (1994). Mathematical principles of reinforcement. Korzybski, A. (1941). Science and sanity (2nd ed.). Lancaster, Behavioral and Brain Sciences, 17, 105–172. PA: International Non-Aristotelian Library. References • 363

Kuczaj, S. A., III. (1977). The acquisition of regular and Lindsley, O. R. (1956). Operant conditioning methods applied irregular past tense forms. Journal of Verbal Learning and to research in chronic schizophrenia. Psychiatric Research Verbal Behavior, 16, 589–600. Reports, 6, 118–139. Kundel, H. L., Nodine, C. F., & Krupinski, E. A. (1990). Lindsley, O. R. (1992). Precision teaching: Discoveries and Computer-displayed eye position as a visual aid to effects. Journal of Applied Behavior Analysis, 25, 51–57. pulmonary nodule interpretation. Investigative radiology, 25, Loeb, J. (1900). Comparative physiology of the brain and comparative 890–896. psychology. New York: Putnam. Lakoff, G. (1987). Women, fi re, and dangerous things. Chicago: Loeb, J. (1918/1973). Forced movements, tropisms, and animal University of Chicago Press. conduct. New York: Dover. Lakoff, G., & Johnson, M. (1980). Metaphors we live by. Chicago: Loftus, E. F. (1993). The reality of repressed memories. University of Chicago Press. American Psychologist, 48, 518–537. Lakoff, G., & Johnson, M. (1999). Philosophy in the fl esh. New Loftus, E. F., & Palmer, J. C. (1974). Reconstruction of York: Basic Books. automobile destruction: An example of the interaction Lalli, J. S., Casey, S. D., & Kates, K. (1997). Noncontingent between language and memory. Journal of Verbal Learning reinforcement as treatment for severe problem behavior: and Verbal Behavior, 13, 585–589. Some procedural variations. Journal of Applied Behavior Loftus, E. F., & Zanni, G. (1975). Eyewitness testimony: The Analysis, 30, 127–136. infl uence of the wording of a question. Bulletin of the Lalli, J. S., Mace, F. C., Wohn, T., & Livezey, K. (1995). Psychonomic Society, 5, 86–88. Identifi cation and modifi cation of a response-class Loisette, A. (1899). Assimilative memory, or, How to attend and hierarchy. Journal of Applied Behavior Analysis, 28, 551–559. never forget. New York: Funk & Wagnalls. Lane, H. (1961). Operant control of vocalizing in the chicken. Long, G. M. (1980). Iconic memory: A review and critique Journal of the Experimental Analysis of Behavior, 4, 171–177. of the study of short-term visual storage. Psychological Lane, H. (1965). The motor theory of speech perception: A Bulletin, 88, 785–820. critical review. Psychological Review, 72, 275–309. Lorenz, K. (1937). The companion in the bird’s world. Auk, Lashley, K. S. (1930). The mechanism of vision. I. A method 54, 245–273. for rapid analysis of pattern vision in the rat. Journal of Lovaas, O. I. (1964). Cue properties of words: The control Genetic Psychology, 37, 453–460. of operant responding by rate and content of verbal Lashley, K. S. (1951). The problem of serial order in behavior. operants. Child Development, 35, 245–256. In L. A. Jeffress (Ed.), Cerebral mechanisms in behavior (pp. Lovaas, O. I., & Simmons, J. Q. (1969). Manipulation of self- 112–146). New York: Wiley. destruction in three retarded children. Journal of Applied Laties, V. G., Weiss, B., Clark, R. L., & Reynolds, M. D. (1965). Behavior Analysis, 2, 143–157. Overt “mediating” behavior during temporally spaced Lowe, C. F. (1980). Determinants of human operant behavior. responding. Journal of the Experimental Analysis of Behavior, In P. Harzem & M. D. Zeiler (Eds.), Advances in the analysis 8, 107–116. of behavior. Volume 1. Reinforcement and the organization of Lattal, K. A., & Perone, M. (Eds.). (1998). Handbook of research behavior (pp. 159–192). Chichester, England: Wiley. methods in human operant behavior. NY: Plenum. Lowe, C. F., & Chadwick, P. D. J. (1990). Verbal control of Lea, S. E. G. (1979). Foraging and reinforcement schedules in delusions. Behavior Therapy, 21, 461–479. the pigeon: Optimal and non-optimal aspects of choice. Lowe, C. F., Horne, P. J., Harris, F. D. A., & Randle, V. R. Animal Behavior, 27, 875–886. L. (2002). Naming and categorization in young children: Leitenberg, H. (1966). Conditioned acceleration and Vocal tract training. Journal of the Experimental Analysis of conditioned suppression in pigeons. Journal of the Behavior, 78, 527–549. Experimental Analysis of Behavior, 9, 205–212. Lowe, C. F., Horne, P. J., & Hughes, J. C. (2005). Naming and Lerman, D. C., Iwata, B. A., & Wallace, M. D. (1999). Side categorization in young children: III. Vocal tract training effects of extinction: Prevalence of bursting and and transfer of function. Journal of the Experimental aggression during the treatment of self-injurious Analysis of Behavior, 83, 47–65. behavior. Journal of Applied Behavior Analysis, 32, 1–8. Lowenkron, B. (1997). The role of naming in the development Lerman, D. C., & Toole, L. M. (2011). Developing function- of joint control. Journal of the Experimental Analysis of based punishment procedures for problem behavior. Behavior, 68, 244–247. In W. W. Fisher, C. C. Piazza, & H. S. Roane (Eds.), Lowenkron, B. (1998). Some logical functions of joint Handbook of applied behavior analysis (pp. 348–369). New control. Journal of the Experimental Analysis of Behavior, 69, York: Guilford. 327–354. Liberman, A. M. (1982). On fi nding that speech is special. Lowenkron, B. (2004). Meaning: A verbal behavior account. American Psychologist, 37, 148–167. Analysis of Verbal Behavior, 20, 77–97. 364 • References

Lubbock, J. (1882). Ants, bees, and wasps. New York: D. McDowell, J. J. (2004). A computational model of selection Appleton. by consequences. Journal of the Experimental Analysis of Lubinski, D., & Thompson, T. (1987). An animal model of the Behavior, 81, 297–317. interpersonal communication of interoceptive (private) McDowell, J. J. (2013). A quantitative evolutionary theory states. Journal of the Experimental Analysis of Behavior, 48, of adaptive behavior dynamics. Psychological Review, 120, 1–15. 731–750. MacCorquodale, K. (1970). On Chomsky’s review of Skinner’s McGlinchey, A., & Keenan, M. (1997). Stimulus equivalence Verbal Behavior. Journal of the Experimental Analysis of and social categorization in Northern Ireland. Behavior Behavior, 13, 83–99. and Social Issues, 7, 113–128. MacPhail, E. M. (1968). Avoidance responding in pigeons. Meehl, P. E. (1950). On the circularity of the Law of Effect. Journal of the Experimental Analysis of Behavior, 11, 629–632. Psychological Bulletin, 47, 52–75. Madden, G. J. (Ed.) (2012). APA Handbook of Behavior Analysis. Mehler, J., Jusczyk, P., Lambert, G., Halsted, N., Bertoncini, Washington, DC: American Psychological Association. J., & Amiel-Tison, C. (1988). A precursor of language Mahoney, M. J., & Bandura, A. (1972). Self-reinforcement in acquisition in young infants. Cognition, 29, 143–178. pigeons. Learning and Motivation, 3, 293–303. Meyer, K. E., & Brysac, S. B. (2012). Pax ethnica. Philadelphia: Maki, R. H. (1979). Right-left and up-down are equally PublicAffairs. discriminable in the absence of directional words. Bulletin Michael, J. (1975). Positive and negative reinforcement, a of the Psychonomic Society, 14, 181–184. distinction that is no longer necessary; or a better way to Malone, J. C. (1990). Theories of learning. Belmont, CA: talk about bad things. Behaviorism, 3, 33–44. Wadsworth. Michael, J. (1989). Establishing operations. Concepts and Malott, R. W., & Cumming, W. W. (1964). Schedules of principles of behavior analysis (pp. 57–72). Kalamazoo, MI: interresponse time reinforcement. Psychological Record, 14, Association for Behavior Analysis. 221–252. Milgram, S. (1963). Behavioral study of obedience. Journal of Manabe, K., Kawashima, T., & Staddon, J. E. R. (1995). Abnormal and Social Psychology, 67, 371–378. Differential vocalization in budgerigars: Towards an Miller, G. A. (1956). The magical number seven plus or experimental analysis of naming. Journal of the Experimental minus two: Some limits on our capacity for processing Analysis of Behavior, 63, 111–126. information. Psychological Review, 63, 81–97. Markoff, J. (2011). Computer wins on ‘Jeopardy!’: Trivial, it’s Miller, N. E., & Carmona, A. (1967). Modifi cation of a visceral not. New York Times. response, salivation in thirsty dogs, by instrumental Masataka, N. (2007). Music, evolution and language. training with water reward. Journal of Comparative and Developmental Science, 10, 35–39. Physiological Psychology, 63, 1–6. Matthews, B. A., Catania, A. C., & Shimoff, E. (1985). Effects Mineka, S., Davidson, M., Cook, M., & Keir, R. (1984). of uninstructed verbal behavior on nonverbal responding: Observational learning of snake fear in Rhesus monkeys. Contingency descriptions versus performance Journal of Abnormal Psychology, 93, 355–372. descriptions. Journal of the Experimental Analysis of Behavior, Mineka, S., & Zinbarg, R. (2006). A contemporary learning 43, 155–164. theory perspective on the etiology of anxiety disorders: Matthews, B. A., Shimoff, E., & Catania, A. C. (1987). Saying It’s not what you thought it was. American Psychologist, 61, and doing: A contingency-space analysis. Journal of 10–26. Applied Behavior Analysis, 20, 69–74. Moerk, E. L. (1980). Relationships between parental input Matthews, B. A., Shimoff, E., Catania, A. C., & Sagvolden, frequencies and children’s language acquisition: A T. (1977). Uninstructed human responding: Sensitivity to reanalysis of Brown’s data. Journal of Child Language, 7, ratio and interval contingencies. Journal of the Experimental 1–14. Analysis of Behavior, 27, 453–467. Moerk, E. L. (1983). A behavioral analysis of controversial Mazur, J. E. (1991). Choice. In I. H. Iversen & K. A. Lattal topics in fi rst language acquisition: Reinforcements, (Eds.), Experimental analysis of behavior. Part 1 (pp. 219– corrections, modeling, input frequencies, and the three- 250). Amsterdam: Elsevier/North-Holland. term contingency pattern. Journal of Psycholinguistic Mazur, J. E. (1996). Procrastination by pigeons: Preference Research, 12, 129–155. for larger, more delayed work requirements. Journal of the Moerk, E. L. (1992). First language: Taught and learned. Baltimore: Experimental Analysis of Behavior, 65, 159–171. Paul H. Brookes. Mazur, J. E. (1998). Procrastination by pigeons with fi xed- Montee, B. B., Miltenberger, R. G., & Wittrock, D. (1995). interval response requirements. Journal of the Experimental An experimental analysis of facilitated communication. Analysis of Behavior, 69, 185–197. Journal of Applied Behavior Analysis, 28, 189–200. References • 365

Moore, C., & Frye, D. (1986). Context, conservation and Nevin, J. A. (1982). On resisting extinction: A review of the meanings of more. British Journal of Developmental Jonathan Schell’s The Fate of the Earth. Journal of the Psychology, 4, 169–178. Experimental Analysis of Behavior, 38, 349–353. Morgan, M. J., Fitch, M. D., Holman, J. G., & Lea, S. E. G. Nevin, J. A. (1992). An integrative model for the study of (1976). Pigeons learn the concept of an “A.”. Perception behavioral momentum. Journal of the Experimental Analysis and Psychophysics, 5, 57–66. of Behavior, 57, 301–316. Morse, W. H., & Kelleher, R. T. (1977). Determinants of Nevin, J. A. (1996a). The momentum of compliance. Journal reinforcement and punishment. In W. K. Honig & J. E. of Applied Behavior Analysis, 64, 535–547. R. Staddon (Eds.), Handbook of operant behavior (pp. 174– Nevin, J. A. (1996b). War initiation and selection by 200). Englewood Cliffs, NJ: Prentice-Hall. consequences. Journal of Peace Research, 33, 99–108. Motokawa, T. (1989). Sushi science and hamburger science. Nevin, J. A. (2003). Retaliating against terrorists. Behavior and Perspectives on Biology and Medicine, 32, 489–504. Social Issues, 12, 109–128. Müller, G. E., & Pilzecker, A. (1900). Experimentelle Beiträge Nevin, J. A. (2005). The inertia of affl uence. Behavior and Social zür Lehre vom Gedächtnis. Zeitschrift für Psychologie, Issues, 14, 7–20. Ergänzungsband 1. Nevin, J. A. (2015). Behavioral momentum: A scientifi c metaphor. Muller, R. A. (2016). Now: The physics of time. New York: Vineyard Haven, MA: Nevin. Norton. Nevin, J. A., & Grace, R. C. (2000). Behavioral momentum Neisser, U. H., N. (1992). Phantom fl ashbulbs: False and the Law of Effect. Behavioral and Brain Sciences, 23, recollections of hearing the news about Challenger. In 73–130. E. Winograd & U. Neisser (Eds.), Affect and accuracy in Nevin, J. A., & Grace, R. C. (2005). Resistance to extinction recall: Studies of “fl ashbulb memories” (pp. 9–31). New York: in the steady state and in transition. Journal of Experimental Cambridge University Press. Psychology: Animal Behavior Processes, 31, 199–212. Neiworth, J. J., & Rilling, M. E. (1987). A method for studying Nevin, J. A., McLean, A. P., & Grace, R. C. (2001). Resistance imagery in animals. Journal of Experimental Psychology: to extinction: Contingency termination and generalization Animal Behavior Processes, 13, 203–214. decrement. Animal Learning and Behavior, 29, 176–191. Neuman, P., Ahearn, W. H., & Hineline, P. N. (2000). Pigeons’ Newell, A., Shaw, J. C., & Simon, H. A. (1958). Elements of choices between fi xed-ratio and linear or geometric a theory of human problem solving. Psychological Review, escalating schedules. Journal of the Experimental Analysis of 65, 151–166. Behavior, 73, 93–102. Nisbett, R. E., Caputo, C., Legant, P., & Marecek, J. (1973). Neuringer, A. (1981). Self-experimentation: A call for change. Behavior as seen by the actor and as seen by the observer. Behaviorism, 9, 79–94. Journal of Personality and Social Psychology, 27, 154–164. Neuringer, A. (1986). Can people behave randomly?: The role Noë, A. (2009). Out of our heads. New York: Hill and Wang. of feedback. Journal of Experimental Psychology: General, O’Brien, F., Bible, J., Liu, D., & Simons-Morton, B. G. 115, 62–75. (2017). Do young drivers become safer after being Neuringer, A. (2002). Operant variability: Evidence, functions, involved in a collision? Psychological Science, 1–7. and theory. Psychonomic Bulletin and Review, 9, 672–705. doi:10.1177/0956797616688118 Neuringer, A. (2004). Reinforced variability in animals and Olson, R. (2015). Houston, we have a narrative. Chicago: people. American Psychologist, 59, 891–906. University of Chicago Press. Neuringer, A. (2014). Operant variability and the evolution Olton, D. S. (1979). Mazes, maps, and memory. American of volition. International Journal of Comparative Psychology, Psychologist, 34, 583–596. 27, 204–223. Olton, D. S., & Samuelson, R. J. (1976). Remembrance Neuringer, A., & Neuringer, M. (1974). Learning by following of places passed: Spatial memory in rats. Journal of a food source. Science, 184, 1005–1008. Experimental Psychology: Animal Behavior Processes, 2, 97–116. Neuringer, A., & Schneider, B. A. (1968). Separating the Ortony, A., & Turner, J. (1990). What’s basic about basic effects of interreinforcement time and number of emotions? Psychological Review, 97, 315–331. interreinforcement responses. Journal of the Experimental Page, S., & Neuringer, A. (1985). Variability is an operant. Analysis of Behavior, 11, 661–667. Journal of Experimental Psychology: Animal Behavior Processes, Nevin, J. A. (1969). Signal detection theory and operant 11, 429–452. behavior: A review of David M. Green & John A. Palmer, D. C. (1996). Achieving parity: The role of automatic Swets’ Signal Detection Theory and Psychophysics. Journal of the reinforcement. Journal of the Experimental Analysis of Experimental Analysis of Behavior, 12, 475–480. Behavior, 65, 289–290. 366 • References

Palmer, D. C. (1998). The speaker as listener: The interpretation Poling, A., Weetjens, B., Cox, C., Beyene, N. W., Bach, H., & of structural regularities in verbal behavior. Analysis of Sully, A. (2011). Using trained pouched rats to detect land Verbal Behavior, 15, 3–16. mines: Another victory for operant conditioning. Journal Paniagua, F. A., & Baer, D. M. (1982). The analysis of of Applied Behavior Analysis, 44, 351–355. correspondence training as a chain reinforceable at any Polson, D. A. D., Grabavac, D. M., & Parsons, J. A. (1997). point. Child Development, 53, 786–798. Intraverbal stimulus-response reversibility: Fluency, Parker, A. (2003). In the blink of an eye: How vision kick-started the familiarity effects, and implications for stimulus big bang of evolution. London: Free Press. equivalence. Analysis of Verbal Behavior, 14, 19–40. Parsons, H. M. (1974). What happened at Hawthorne? Science, Potts, L., Eshleman, J. W., & Cooper, J. O. (1993). Ogden R. 183, 922–932. Lindsley and the historical development of precision Pashler, H., McDaniel, M., Rohrer, D., & Bjork, R. A. (2008). teaching. The Behavior Analyst, 16, 177–189. Learning styles: Concepts and evidence. Psychological Poulson, C. L. (1984). Operant theory and methodology in Science in the Public Interest, 9, 105–119. infant vocal conditioning. Journal of Experimental Child Pavlov, I. P. (1927). Conditioned refl exes (G. V. Anrep, Trans.). Psychology, 38, 103–113. London: Oxford University Press. Premack, D. (1959). Toward empirical behavior laws: I. Peele, D. B., & Ferster, C. B. (1982). Autoshaped key pecking Positive reinforcement. Psychological Review, 66, 219–233. maintained by access to a social space. Journal of the Premack, D. (1962). Reversibility of the reinforcement Experimental Analysis of Behavior, 38, 181–189. relation. Science, 136, 255–257. Pennypacker, H., & Iwata, B. A. (1990). Mammacare: A case Premack, D. (1970). A functional analysis of language. Journal history in behavioral medicine. In D. E. Blackman & H. of the Experimental Analysis of Behavior, 14, 107–125. Lejeune (Eds.), Behaviour Analysis in Theory and Practice (pp. Premack, D. (1971). Catching up with common sense or two 259–288). Hove: Erlbaum. sides of a generalization: Reinforcement and punishment. Pepperberg, I. M. (1988). The importance of social interaction In R. Glaser (Ed.), The nature of reinforcement (pp. 121–150). and observation in the acquisition of communicative New York: Academic Press. competence: Possible parallels between avian and human Provine, R. R. (1976). Development of function in nerve nets. learning. In T. R. Zentall & J. B. G. Galef (Eds.), Social In J. Fentress (Ed.), Simpler networks and behavior (pp. 203– learning (pp. 279–299). Hillsdale, NJ: Erlbaum. 220). Sunderland, MA: Sinauer. Perone, M. (2003). Negative effects of positive reinforcement. Provine, R. R. (1984). Wing-fl apping during development and Behavior Analyst, 26, 1–14. evolution. American Scientist, 72, 448–455. Peterson, G. B. (2004). A day of great illumination: B. F. Provine, R. R. (1988). A hierarchy of developmental Skinner’s discovery of shaping. Journal of the Experimental contingencies: A review of Purves and Lichtman’s Analysis of Behavior, 82, 317–328. Principles of Neural Development. Journal of the Experimental Peterson, L. R., & Peterson, M. J. (1959). Short-term retention Analysis of Behavior, 50, 565–569. of individual verbal items. Journal of Experimental Provine, R. R. (2004). Laughing, tickling, and the evolution Psychology, 58, 193–198. of speech and self. Current Directions in Psychological Science, Peterson, N. (1960). Control of behavior by presentation of 13, 215–218. an imprinted stimulus. Science, 132, 1395–1396. Provine, R. R. (2012). Curious behavior: Yawning, laughing, Pfungst, O. (1911). Clever Hans (the horse of Mr. Von Osten): A hiccupping, and beyond. Cambridge, MA: Belknap/Harvard contribution to experimental animal and human psychology (tr. C. University Press. L. Rahn). New York: Holt. Pryor, K. W. (1999). Don’t shoot the dog! (rev. ed.). New York: Pilgrim, C., Jackson, J., & Galizio, M. (2000). Acquisition of Bantam. arbitrary conditional discriminations by young normally Pryor, K. W. (2009). Reaching the animal mind: Clicker training and developing children. Journal of the Experimental Analysis of what it teaches us about animals. New York: Scribner. Behavior, 73, 177–193. Pryor, K. W., Haag, R., & O’Reilly, J. (1969). The creative Pinker, S. (1994). The language instinct. New York: Morrow. porpoise: Training for novel behavior. Journal of the Pitts, R. C., & Malagodi, E. F. (1991). Preference for less Experimental Analysis of Behavior, 12, 653–661. frequent shock under fi xed-interval schedules of electric- Pullum, G. K. (1991). The great Eskimo vocabulary hoax. In shock presentation. Journal of the Experimental Analysis of The great Eskimo vocabulary hoax and other irreverent essays on Behavior, 56, 21–32. the study of language (pp. 159–171). Chicago: University of Platt, J. R. (1973). Percentile reinforcement: Paradigms for Chicago Press. experimental analysis of response shaping. In G. H. Rachlin, H. (1971). On the tautology of the matching law. Bower (Ed.), The psychology of learning and motivation. Volume Journal of the Experimental Analysis of Behavior, 15, 249–251. 7 (pp. 271–296). New York: Academic Press. Rachlin, H. (1974). Self-control. Behaviorism, 2, 94–107. References • 367

Rachlin, H. (2014). The escape of the mind. New York: Oxford Rescorla, R. A. (1967). Pavlovian conditioning and its proper University Press. control procedures. Psychological Review, 74, 71–80. Rachlin, H., & Baum, W. M. (1972). Effect of alternative Rescorla, R. A. (1988). Pavlovian conditioning: It’s not what reinforcement: Does the source matter? Journal of the you think it is. American Psychologist, 43, 151–160. Experimental Analysis of Behavior, 18, 231–241. Rescorla, R. A., & Solomon, R. L. (1967). Two process learning Rachlin, H., & Green, L. (1972). Commitment, choice and theory: Relationships between Pavlovian conditioning self-control. Journal of the Experimental Analysis of Behavior, and instrumental learning. Psychological Review, 74, 151– 17, 15–22. 182. Rachlin, H., Green, L., Kagel, J. H., & Battalio, R. C. (1976). Rescorla, R. A., & Wagner, A. R. (1972). A theory of Economic demand theory and psychological studies of Pavlovian conditioning: Variations in the effectiveness of choice. In G. H. Bower (Ed.), The psychology of learning and reinforcement and nonreinforcement. In A. H. Black & motivation (Vol. 10, pp. 129–154). New York: Academic W. F. Prokasy (Eds.), Classical conditioning II (pp. 64–99). Press. New York: Appleton-Century-Crofts. Rajala, A. Z., Rwininger, K. R., Lancaster, K. M., & Populin, L. Restle, F. (1957). Discrimination of cues in mazes: A C. (2010). Rhesus monkeys (Macaca mulatta) do recognize resolution of the “place versus response” question. themselves in the mirror: Implications for the evolution Psychological Review, 64, 217–228. of self-recognition. PLoS ONE, 5(9), e12865. Revusky, S. H., & Garcia, J. (1970). Learned associations over Ramus, F. (2002). Language discrimination by newborns: long delays. In G. H. Bower (Ed.), The psychology of learning Teasing apart phonotactic, rhythmic, and intonational and motivation. Volume 4 (pp. 1–44). New York: Academic cues. Annual Review of Language Acquisition, 2, 85–115. Press. Randell, T., & Remington, B. (1999). Equivalence relations Reynolds, G. S. (1961a). Attention in the pigeon. Journal of the between visual stimuli: The functional role of naming. Experimental Analysis of Behavior, 4, 203–208. Journal of the Experimental Analysis of Behavior, 71, 395–415. Reynolds, G. S. (1961b). Behavioral contrast. Journal of the Ratliff, F. (1965). Mach bands: Quantitative studies on neural Experimental Analysis of Behavior, 4, 57–71. networks in the retina. San Francisco: Holden-Day. Reynolds, G. S., Catania, A. C., & Skinner, B. F. (1963). Ratliff, F., & Hartline, H. K. (1959). The responses of Conditioned and unconditioned aggression. Journal of the Limulus optic nerve fi bers to patterns of illumination Experimental Analysis of Behavior, 6, 73–74. on the receptor mosaic. Journal of General Physiology, 42, Reynolds, G. S., & Skinner, B. F. (1962). Technique for 1241–1255. reinforcing either of two organisms with a single food Reber, A. S. (1976). Implicit learning of synthetic languages: magazine. Journal of the Experimental Analysis of Behavior, the role of instructional set. Journal of Experimental 5, 58. Psychology: Human Learning and Memory, 2, 88–94. Riess, B. F. (1946). Genetic changes in semantic conditioning. Reddy, M. J. (1979). The conduit metaphor - a case of frame Journal of Experimental Psychology, 36, 143–152. confl ict in our language about language. In A. Ortony Risley, T. R. (1977). The development and maintenance of (Ed.), Metaphor and thought (pp. 284–324). New York: language: An operant model. In B. C. Etzel, J. M. LeBlanc, Cambridge University Press. & D. M. Baer (Eds.), New developments in behavioral research Reed, D. H. (2007). Natural selection and genetic diversity. (pp. 81–101). Hillsdale, NJ: Erlbaum. Heredity, 99, 1–2. Risley, T. R., & Hart, B. (1968). Developing correspondence Reid, R. L. (1958). The role of the reinforcer as a stimulus. between the non-verbal and verbal behavior of preschool British Journal of Psychology Monograph Supplements, 49, 202– children. Journal of Applied Behavior Analysis, 1, 267–281. 209. Roane, H. S., Rihgdahl, J. E., Kelley, M. E., & Glover, A. C. Reinhard, G., & Lachnit, H. (2002). Differential conditioning (2011). Single-case experimental designs. In W. W. Fisher, of anticipatory pupillary dilation responses in humans. C. C. Piazza, & H. S. Roane (Eds.), Handbook of applied Biological Psychology, 60, 51–68. behavior analysis (pp. 132–147). New York: Guilford. Rendall, D., Rodman, P. S., & Emond, R. E. (1996). Vocal Roediger, H. L., III. (1980). Memory metaphors in cognitive recognition of individuals and kin in free-ranging rhesus psychology. Memory and Cognition, 8, 231–246. monkeys. Animal Behaviour, 51, 1007–1015. Rogers-Warren, A. R., & Baer, D. M. (1976). Correspondence Rentfro, A. D. (2010, Nov/Dec). Turning fearful cats friendly. between saying and doing: Teaching children to share and Animal Sheltering, 51–56. praise. Journal of Applied Behavior Analysis, 9, 335–354. Repp, A. C., & Deitz, S. M. (1974). Reducing aggressive Rosales-Ruiz, J. (2011). From fi erce or fearful to friendly: A shaping and self-injurious behavior of institutionalized retarded program for dogs, cats, and beyond. Paper presented at the children through reinforcement of other behaviors. Berkshire Association for Behavior Analysis and Therapy, Journal of Applied Behavior Analysis, 7, 313–325. Amherst, MA. 368 • References

Rosch, E. H. (1973). Natural categories. Cognitive Psychology, 4, Savage-Rumbaugh, E. S., Rumbaugh, D. M., & Boysen, 328–350. S. (1978). Symbolic communication between two Rosenfarb, I. S., Newland, M. C., Brannon, S. E., & Howey, chimpanzees (Pan troglodytes). Science, 201, 641–644. D. S. (1992). Effects of self-generated rules on the Savage-Rumbaugh, E. S., Rumbaugh, D. M., Smith, S. T., & development of schedule-controlled behavior. Journal of Lawson, J. (1980). Reference: The linguistic essential. the Experimental Analysis of Behavior, 58, 107–121. Science, 210, 922–925. Rosenfeld, H. M., & Baer, D. M. (1970). Unbiased and Sawisch, L. P., & Denny, M. R. (1973). Reversing the unnoticed verbal conditioning: The double agent robot reinforcement contingencies of eating and keypecking procedure. Journal of the Experimental Analysis of Behavior, behaviors. Animal Learning and Behavior, 1, 189–192. 14, 99–107. Schelling, T. C. (1971). On the ecology of micromotives. Rosenhan, D. L. (1973). On being sane in insane places. Public Interest, 25, 61–98. Science, 179, 250–258. Schlinger, H., & Blakely, E. (1987). Function-altering effects Ross, D., & Spurrett, D. (2004). What to say to a skeptical of contingency-specifying stimuli. The Behavior Analyst, metaphysician: A defense manual for cognitive and 10, 41–45. behavioral scientists. Behavioral and Brain Sciences, 27, 603– Schlosberg, H. (1937). The relationship between success and 647. the laws of conditioning. Psychological Review, 44, 379–394. Rozin, P., Millman, L., & Nemeroff, C. (1986). Operation Schoenfeld, W. N. (1950). An experimental approach to of the laws of sympathetic magic in disgust and other anxiety, escape and avoidance behavior. In P. H. Hoch domains. Journal of Personality and Social Psychology, 50, (Ed.), Anxiety (pp. 70–99). New York: Grune & Stratton. 703–712. Schoenfeld, W. N. (1966). Some old work for modern Rumbaugh, D. M., & Gill, T. V. (1976). The mastery of conditioning theory. Conditional Refl ex, 1, 219–223. language-type skills by the chimpanzee (Pan). Annals of Schoenfeld, W. N. (1969). “Avoidance” in behavior theory. the New York Academy of Sciences, 280, 562–578. Journal of the Experimental Analysis of Behavior, 12, 669–674. Sacks, O. (1985). The president’s speech. The man who mistook Schoenfeld, W. N. (Ed.) (1970). The theory of reinforcement his wife for a hat and other clinical tales (pp. 80–85). NY: schedules. New York: Appleton-Century-Crofts. Simon & Schuster. Schoenfeld, W. N., & Cole, B. K. (1972). Stimulus schedules: The Sagvolden, T., Johansen, E. B., Aase, H., & Russell, V. A. t- systems. New York: Harper & Row. (2005). A dynamic developmental theory of Attention- Schooler, J. W., & Engstler-Schooler, T. Y. (1990). Verbal Defi cit/Hyperactivity Disorder (ADHD) predominantly overshadowing of visual memories: Some things are hyperactive/impulsive and combined subtypes. Behavioral better left unsaid. Cognitive Psychology, 22, 36–71. and Brain Sciences, 28, 397–468. Schroeder, S. R., & Holland, J. G. (1968). Operant control Salapatek, P., & Kessen, W. (1966). Visual scanning of of eye movements. Journal of Applied Behavior Analysis, 1, triangles by the human newborn. Journal of Experimental 161–166. Child Psychology, 3, 155–167. Schuster, C. R., & Balster, R. L. (1977). The discriminative Sargisson, R. J., & White, K. G. (2001). Generalization of stimulus properties of drugs. In T. Thompson & P. Dews delayed matching to sample following training at different (Eds.), Advances in behavioral pharmacology. Volume 1 (pp. delays. Journal of the Experimental Analysis of Behavior, 75, 85–138). New York: Academic Press. 1–14. Schuster, C. R., & Thompson, T. (1969). Self administration Saunders, R. R., & Green, G. (1992). The nonequivalence of of and behavioral dependence on drugs. Annual Review of behavioral and mathematical equivalence. Journal of the Pharmacology, 9, 483–502. Experimental Analysis of Behavior, 57, 227–241. Schusterman, R. J., & Kastak, D. (1993). A California sea Saunders, R. R., & Green, G. (1999). A discrimination analysis lion (Zalophus californianus) is capable of forming of training-structure effects on stimulus equivalence equivalence relations. Psychological Record, 43, 823–839. outcomes. Journal of the Experimental Analysis of Behavior, Schwartz, B. (1982). Failure to produce response variability 72, 117–137. with reinforcement. Journal of the Experimental Analysis of Savage-Rumbaugh, E. S. (1986). Ape language. New York: Behavior, 37, 171–181. Columbia University Press. Schwartz, B. (1986). The battle for human nature. New York: Savage-Rumbaugh, E. S., Murphy, J., Sevcik, R. A., Brakke, K. Norton. E., Williams, S. L., & Rumbaugh, D. M. (1993). Language Schwartz, B., & Williams, D. R. (1972). Two different kinds comprehension in ape and child. Monographs of the Society of key peck in the pigeon: Some properties of responses for Research in Child Development, 58 (3–4, Serial No. 233). maintained by negative and positive response-reinforcer contingencies. Journal of the Experimental Analysis of Behavior, 18, 201–216. References • 369

Sebeok, T. A., & Rosenthal, R. (Eds.). (1981). The Clever Hans Shimp, C. (1982). On metaknowledge in the pigeon. Animal phenomenon: Communication with horses, whales, apes, and people. Learning and Behavior, 10, 358–364. New York: New York Academy of Sciences. Sidman, M. (1952). A note on functional relations obtained Sechenov, I. M. (1863/1965). Refl exes of the brain (tr. S. Belsky). from group data. Psychological Bulletin, 49, 263–269. Cambridge, MA: MIT Press. Sidman, M. (1953). Two temporal parameters in the Seiffert, R. (2004). This side of fear. New York Times Magazine, maintenance of avoidance behavior by the white rat. Oct 31 Journal of Comparative and Physiological Psychology, 46, 253– Seligman, M. E. P. (1970). On the generality of the laws of 261. learning. Psychological Review, 77, 406–418. Sidman, M. (1960). Tactics of scientifi c research. New York: Basic Seligman, M. E. P., & Csikszentmihalyi, M. (2000). Positive Books. psychology: An introduction. American Psychologist, 55, Sidman, M. (1971). The behavioral analysis of aphasia. Journal 5–14. of Psychiatric Research, 8, 413–422. Seligman, M. E. P., & Csikszentmihalyi, M. (2001). Reply to Sidman, M. (1987). Two choices are not enough. Behavior comments. American Psychologist, 56, 89–90. Analysis, 22, 11–18. Seyfarth, R. M., Cheney, D. L., & Marler, P. (1980a). Monkey Sidman, M. (1994). Equivalence relations and behavior: A research responses to three different alarm calls: Evidence for story. Boston: Authors Cooperative. predator classifi cation and semantic communication. Sidman, M. (2000). Equivalence relations and the Science, 210, 801–803. reinforcement contingency. Journal of the Experimental Seyfarth, R. M., Cheney, D. L., & Marler, P. (1980b). Vervet Analysis of Behavior, 74, 127–146. monkey alarm calls: Semantic communication in a free- Sidman, M., Cresson, O., Jr., & Willson-Morris, M. (1974). ranging primate. Animal Behaviour, 28, 1070–1094. Acquisition of matching to sample via mediated transfer. Shapiro, M., Xu, Z., Catania, A. C., Neef, N. A., Kranak, M., Journal of the Experimental Analysis of Behavior, 22, 261–273. & Smith, E. (2016). Shaping verbal behavior increases the use Sidman, M., Herrnstein, R. J., & Conrad, D. G. (1957). of specifi c praise by general educators. Paper presented at the Maintenance of avoidance behavior by unavoidable Association for Behavior Analysis International, Chicago. shocks. Journal of Comparative and Physiological Psychology, 50, Sheffi eld, F. D. (1965). Relation between classical conditioning 553–557. and instrumental learning. In W. F. Prokasy (Ed.), Classical Sidman, M., & Stoddard, L. T. (1967). The effectiveness conditioning (pp. 302–322). New York: Appleton-Century- of fading in programming a simultaneous form Crofts. discrimination for retarded children. Journal of the Shepard, R. N., & Metzler, J. (1971). Mental rotation of three- Experimental Analysis of Behavior, 10, 3–15. dimensional objects. Science, 171, 701–703. Sidman, M., Wynne, C. K., Maguire, R. W., & Barnes, T. (1989). Sherman, T. M., & Cormier, W. H. (1974). An investigation Functional classes and equivalence relations. Journal of the of the infl uence of student behavior on teacher behavior. Experimental Analysis of Behavior, 52, 261–274. Journal of Applied Behavior Analysis, 7, 11–21. Siegel, S. (1977). Morphine tolerance acquisition as an Sherrington, C. (1906). The integrative action of the nervous system. associative process. Journal of Experimental Psychology: New York: Scribners. Animal Behavior Processes, 3, 1–13. Shimoff, E., & Catania, A. C. (1995). Using computers to Siegel, S., Hinson, J. M., Krank, M. D., & McCully, J. (1982). teach behavior analysis. Behavior Analyst, 18, 307–316. Heroin “overdose” death: The contribution of drug- Shimoff, E., & Catania, A. C. (1998). The verbal governance associated environmental cues. Science, 216, 436–437. of behavior. In K. A. Lattal & M. Perone (Eds.), Handbook Silverman, P. J. (1971). Chained and tandem fi xed-interval of research methods in human operant behavior (pp. 371–404). schedules of punishment. Journal of the Experimental New York: Plenum. Analysis of Behavior, 16, 1–13. Shimoff, E., & Catania, A. C. (2001). Effects of recording Simmons, R. E., & Scheepers, L. (1996). Winning by a neck: attendance on grades in introductory psychology. Teaching Sexual selection in the evolution of the giraffe. American of Psychology, 28, 192–195. Naturalist, 148(5), 771–786. Shimoff, E., Catania, A. C., & Matthews, B. A. (1981). Skiba, E. A., Pettigrew, L. E., & Alden, S. E. (1971). A Uninstructed human responding: Sensitivity of low-rate behavioral approach to the control of thumbsucking performance to schedule contingencies. Journal of the in the classroom. Journal of Applied Behavior Analysis, 4, Experimental Analysis of Behavior, 36, 207–220. 121–125. Shimp, C. (1966). Probabilistically reinforced choice behavior Skinner, B. F. (1930). On the conditions of elicitation of in pigeons. Journal of the Experimental Analysis of Behavior, certain eating refl exes. Proceedings of the National Academy 9, 443–455. of Sciences, 16, 433–438. 370 • References

Skinner, B. F. (1931). The concept of the refl ex in the Skinner, B. F. (1968). The technology of teaching. New York: description of behavior. Journal of General Psychology, 5, Appleton-Century-Crofts. 427–458. Skinner, B. F. (1969). An operant analysis of problem solving Skinner, B. F. (1932). Drive and refl ex strength. Journal of B. F. Skinner, Contingencies of reinforcement (pp. 133–157). General Psychology, 6, 22–37. New York: Appleton-Century-Crofts. Skinner, B. F. (1933). The rate of establishment of a Skinner, B. F. (1972/1999). A lecture on “having” a poem. discrimination. Journal of General Psychology, 9, 302–350. Cumulative record (Defi nitive ed., pp. 391–401). Acton, Skinner, B. F. (1934a). The extinction of chained refl exes. MA: B. F. Skinner Foundation. Proceedings of the National Academy of Sciences, 20, 234–237. Skinner, B. F. (1975). The shaping of phylogenic behavior. Skinner, B. F. (1934b). Has Gertrude Stein a secret? Atlantic Journal of the Experimental Analysis of Behavior, 24, 117–120. Monthly, reprinted in Cumulative Record (4th edition, pp. Skinner, B. F. (1977). The force of coincidence. The Humanist, 405–415). 37 (3, May/June), 10–11. Skinner, B. F. (1935a). The generic nature of the concepts Skinner, B. F. (1981a). How to discover what you have to say. of stimulus and response. Journal of General Psychology, 12, Behavior Analyst, 4, 1–7. 40–65. Skinner, B. F. (1981b). Selection by consequences. Science, 213, Skinner, B. F. (1935b). Two types of conditioned refl ex and a 501–504. pseudotype. Journal of General Psychology, 12, 66–77. Skinner, B. F. (1983). Intellectual self-management in old age. Skinner, B. F. (1936). The verbal summator and a method for American Psychologist, 38, 239–244. the study of latent speech. Journal of Psychology, 2, 71–107. Skinner, B. F. (1986). The evolution of verbal behavior. Journal Skinner, B. F. (1938). The behavior of organisms: An experimental of the Experimental Analysis of Behavior, 45, 115–122. analysis. New York: Appleton-Century-Crofts. Skinner, B. F. (1987). What religion means to me. Free Inquiry, Skinner, B. F. (1945a). Baby in a box. Ladies’ Home Journal, 62 7 (Spring), 12–13. (Oct), 30–31, 135–136, 138. Skinner, B. F. (1988). Responses to commentaries. In A. C. Skinner, B. F. (1945b). The operational analysis of Catania & S. Harnad (Eds.), The Selection of Behavior: The psychological terms. Psychological Review, 52, 270–277. Operant Behaviorism of B. F. Skinner. New York: Cambridge Skinner, B. F. (1948). “Superstition” in the pigeon. Journal of University Press. Experimental Psychology: Animal Behavior Processes, 38, 168– Skinner, B. F. (1989a). The listener. Recent issues in the analysis of 172. behavior (pp. 35–47). Columbus, OH: Merrill. Skinner, B. F. (1950). Are theories of learning necessary? Skinner, B. F. (1989b). The origins of cognitive thought. Psychological Review, 57, 193–216. American Psychologist, 44, 13–18. Skinner, B. F. (1951). How to teach animals. Scientifi c American, Skinner, B. F. (1990). Can psychology be a science of mind? 185(Dec), 26–29. American Psychologist, 45, 1206–1210. Skinner, B. F. (1953). Science and human behavior. New York: Skinner, B. F. (1999). Cumulative record (Defi nitive Edition, V. Macmillan. G. Laties & A. C. Catania, eds.). Cambridge, MA: B. F. Skinner, B. F. (1956). A case history in scientifi c method. Skinner Foundation. American Psychologist, 11, 221–233. Skinner, B. F., & Vaughan, M. E. (1983). Enjoy old age. New Skinner, B. F. (1957). Verbal behavior. New York: Appleton- York: Norton. Century-Crofts. Small, W. S. (1899–1900). Experimental studies of the mental Skinner, B. F. (1959). An experimental analysis of certain processes of the rat. American Journal of Psychology, 11, emotions. Journal of the Experimental Analysis of Behavior, 1–89. 2, 264. Smith, B. H. (1968). Poetic closure. Chicago: University of Skinner, B. F. (1960). Pigeons in a pelican. American Psychologist, Chicago Press. 15, 28–37. Smith, J. D., & Washburn, D. A. (2005). Uncertainty Skinner, B. F. (1961). The fl ight from the laboratory. In W. monitoring and metacognition by animals. Current Dennis & e. al. (Eds.), Current trends in psychology theory. Directions in Psychological Science, 14, 19–24. Pittsburgh: University of Pittsburgh Press. Smith, K. (1954). Conditioning as an artifact. Psychological Skinner, B. F. (1963). Behaviorism at fi fty. Science, 140, 951– Review, 61, 217–225. 958. Solnick, J. V., Rincover, A., & Peterson, C. R. (1977). Some Skinner, B. F. (1964/1980). 20: Education in 1984. In The determinants of the reinforcing and punishing effects of Oxford dictionary of quotations (3rd ed., pp. 508). New York: timeout. Journal of Applied Behavior Analysis, 10, 415–424. Oxford University Press. Spalding, D. (1873/1954). Instinct with original observations Skinner, B. F. (1966). The phylogeny and ontogeny of on young animals (Reprinted British Journal of Animal behavior. Science, 153, 1204–1213. Behaviour, 2, 2–11). Macmillan’s Magazine, 27, 282–293. References • 371

Sperling, G. (1960). The information available in brief visual Swisher, M. J., & Urcuioli, P. J. (2013). Symmetry in the pigeon presentations. Psychological Monographs, 74(11, Whole No. with sample and comparison stimuli in different locations. 498). Journal of the Experimental Analysis of Behavior, 100, 49–60. Sperling, G., & Reeves, A. (1980). Measuring the reaction time Tauber, K. (1965). Residential segregation. Scientifi c American, of a shift of visual attention. In R. S. Nickerson (Ed.), 213 (August), 12–19. Attention and performance VIII (pp. 347–360). Hillsdale, NJ: Terrace, H. S. (1963a). Discrimination learning with and Erlbaum. without “errors.”. Journal of the Experimental Analysis of Spetch, M. L., Wilkie, D. M., & Pinel, J. P. J. (1981). Backward Behavior, 6, 1–27. conditioning: A reevaluation of the empirical evidence. Terrace, H. S. (1963b). Errorless transfer of a discrimination Psychological Bulletin, 89, 163–175. across two continua. Journal of the Experimental Analysis of Squier, L. H. (1993). The science and art of training: A review Behavior, 6, 223–232. of Pryor’s Lads before the Wind. Journal of the Experimental Terrace, H. S., Petitto, L. A., Sanders, R. J., & Bever, T. G. Analysis of Behavior, 59, 423–431. (1979). Can an ape create a sentence? Science, 206, 891– Sran, S. K., & Borrero, J. C. (2010). Assessing the value of 902. choice in a token system. Journal of Applied Behavior Thelen, E., & Fisher, D. M. (1983). From spontaneous to Analysis, 43, 553–557. instrumental behavior: Kinematic analysis of movement St. Peter, C. C. (2012). Improving accuracy of sleep self- changes during very early learning. Child Development, 54, reports through correspondence training. Psychological 429–440. Record, 62, 623–630. Thomas, J. R. (1979). Matching-to-sample accuracy on fi xed- Staddon, J. E. R., & Simmelhag, V. L. (1971). The “superstition” ratio schedules. Journal of the Experimental Analysis of experiment: A reexamination of its implications for the Behavior, 32, 183–189. principles of adaptive behavior. Psychological Review, 78, Thompson, C. R., & Church, R. M. (1980). An explanation 3–43. of the language of a chimpanzee. Science, 208, 313–314. Steinman, W. M. (1968). Response rate and varied Thompson, R. H., Iwata, B. A., Hanley, G. P., Dozier, C. reinforcement: Reinforcers of similar strengths. L., & Samaha, A. L. (2003). The effects of extinction, Psychonomic Science, 10, 35–36. noncontingent reinforcement, and differential Stewart, G. R. (1945/1975). Names on the land. New York: reinforcement of other behavior as control procedures. Oxford University Press. Journal of Applied Behavior Analysis, 36, 221–238. Stratton, G. M. (1897). Vision without inversion of the retinal Thompson, T., & Pickens, R. (Eds.). (1971). Stimulus properties image. Psychological Review, 4, 341–360; 463–481. of drugs. New York: Appleton-Century-Crofts. Stratton, G. M. (1917). Theophrastus and the Greek physiological Thompson, T., & Schuster, C. R. (1964). Morphine self- psychology before Aristotle. New York: Macmillan. administration, food-reinforced, and avoidance behaviors Straub, R. O., Seidenberg, M. S., Bever, T. G., & Terrace, in Rhesus monkeys. Psychopharmacologia, 5, 87–94. H. S. (1979). Serial learning in the pigeon. Journal of the Thorndike, E. L. (1898). Animal intelligence: An experimental Experimental Analysis of Behavior, 32, 137–148. study of the associative processes in animals. Psychological Stroop, J. R. (1935). Studies of interference in serial verbal Review Monograph Supplements, 2(4). reactions. Journal of Experimental Psychology, 18, 643–662. Thorndike, E. L., & Woodworth, R. S. (1901). The infl uence of Sturgis, E. T., Tollison, C. D., & Adams, H. E. (1978). improvement in one mental function upon the effi ciency Modifi cation of combined migraine-muscle contraction of other functions. Psychological Review, 8, 247–261. headaches using BVP and EMG feedback. Journal of Tinbergen, N. (1960). The Herring Gull’s world (revised edition). Applied Behavior Analysis, 11, 215–223. New York: Basic Books. Sutherland, A. (2006). Kicked, bitten, and scratched. New York: Tinbergen, N. (1972). The animal in its world. Volume 1. Field Viking. studies. Cambridge, MA: Harvard University Press. Sutherland, A. (2008). What Shamu taught me about life, love, and Tinbergen, N., & Perdeck, A. C. (1950). On the stimulus marriage. New York: Random House. situation releasing the begging response in the newly Sutton, R. S., & Barto, A. G. (1998). Reinforcement learning: An hatched Herring Gull chick (Larus a. argentatus Pontopp). introduction. Cambridge, MA: MIT Press. Behavior Analyst, 3, 1–38. Sweeney, M. M., & Urcuioli, P. J. (2010). Refl exivity in pigeons. Todd, J. T., & Morris, E. K. (1992). Case histories in the great Journal of the Experimental Analysis of Behavior, 94, 267–282. power of steady misinterpretation. American Psychologist, Swets, J. A., Dawes, R. M., & Monahan, J. (2000). Psychological 47, 1441–1453. science can improve diagnostic decisions. Psychological Todes, D. P. (2014). Ivan Pavlov: A Russian life in science. New Science in the Public Interest, 1, 1–26. York: Oxford. 372 • References

Tolman, E. C. (1948). Cognitive maps in rats and men. and Awareness, 1962). Analysis of Verbal Behavior, 10, Psychological Review, 55, 189–208. 45–68. Tomasello, M., & Farrar, M. J. (1986). Joint attention and early Verplanck, W. S. (2000). Glossary/thesaurus of behavioral terms language. Child Development, 57, 1454–1463. (CD-ROM). Columbia, MD: CMS Software. Touchette, P. E. (1969). Tilted lines as complex stimuli. Journal Vesonder, G. T., & Voss, J. F. (1985). On the ability to predict of the Experimental Analysis of Behavior, 12, 211–214. one’s own responses while learning. Journal of Memory and Truax, C. B. (1966). Reinforcement and nonreinforcement in Language, 24, 363–376. Rogerian therapy. Journal of Abnormal Psychology, 71, 1–9. Vihman, M. M. (1996). Phonological development: The origins of Trut, L. (1999). Early canid domestication: The farm-fox language in the child. Oxford: Blackwell. experiment. American Scientist, 87, 160–169. Vollmer, T. R., & Athens, E. (2011). Developing function- Tulving, E. (1972). Episodic and semantic memory. In E. based extinction procedures for problem behavior. In W. Tulving & W. Donaldson (Eds.), Organization of memory W. Fisher, C. C. Piazza, & H. S. Roane (Eds.), Handbook of (pp. 381–403). New York: Academic Press. applied behavior analysis (pp. 317–334). New York: Guilford. Tulving, E., & Madigan, S. A. (1970). Memory and verbal Vollmer, T. R., Borrero, J. C., & Wright, C. S. (2001). learning. Annual Review of Psychology, 21, 437–484. Identifying possible contingencies during descriptive Twitmyer, E. B. (1974). A study of the knee jerk (1902). Journal analyses of severe behavior disorders. Journal of Applied of Experimental Psychology, 103, 1047–1966. Behavior Analysis, 34, 269–287. Twyman, J. S. (1998). The Fred S. Keller School. Journal of Vollmer, T. R., & Iwata, B. A. (1991). Establishing operations Applied Behavior Analysis, 31, 695–701. and reinforcement effects. Journal of Applied Behavior Underwood, B. J. (1957). Interference and forgetting. Analysis, 24, 279–291. Psychological Review, 64, 49–60. von Békésy, G. (1960). Experiments in hearing. New York: Underwood, B. J. (1961). Ten years of massed practice on McGraw-Hill. distributed practice. Psychological Review, 68, 229–247. von Békésy, G. (1967). Sensory inhibition. Princeton, NJ: Urcuioli, P. J. (1985). On the role of differential sample Princeton University Press. behaviors in matching-to-sample. Journal of Experimental Vonnegut, K. (1963). Cat’s cradle. New York: Dell. Psychology: Animal Behavior Processes, 11, 502–519. Wagner, A. R., Thomas, E., & Norton, T. (1967). Conditioning Urcuioli, P. J. (1991). Retardation and facilitation of matching with electrical stimulation of motor cortex: Evidence of acquisition by differential outcomes. Animal Learning and a possible source of motivation. Journal of Comparative and Behavior, 19, 29–36. Physiological Psychology, 64, 191–199. Urcuioli, P. J. (2005). Behavioral and associative effects of Wagner, K. R. (1985). How much do children say in a day? differential outcomes in discrimination learning. Learning Journal of Child Language, 12, 475–487. and Behavior, 33, 1–21. Wahler, R. G. (1975). Some structural aspects of deviant child Urcuioli, P. J. (2008). Associative symmetry, anti-symmetry, behavior. Journal of Applied Behavior Analysis, 8, 27–42. and a theory of pigeons’ equivalence-class formation. Walcott, C., Gould, J. L., & Kirschvink, J. L. (1979). Pigeons Journal of the Experimental Analysis of Behavior, 90, 257–282. have magnets. Science, 205, 1027–1029. Urcuioli, P. J., & DeMarse, T. B. (1994). On the relationship Wales, R. (1986). Deixis. In P. Fletcher & M. Garman (Eds.), between differential outcomes and differential sample Language acquisition (2nd ed., pp. 401–428). New York: responding in matching-to-sample. Journal of Experimental Cambridge University Press. Psychology: Animal Behavior Processes, 20, 249–263. Wanchison, B. A., Tatham, T. A., & Hineline, P. N. (1988). Urcuioli, P. J., & Swisher, M. J. (2015). Transitive and anti- Pigeons’ choices in situations of diminishing returns: transitive emergent relations in pigeons: Support for a Fixed- versus progressive-ratio schedules. Journal of the theory of stimulus-class formation. Behavioural Processes, Experimental Analysis of Behavior, 50, 375–394. 112, 49–60. Wasik, B. H. (1970). The application of Premack’s Vaughan, W., Jr. (1988). Formation of equivalence sets in generalization on reinforcement to the management of pigeons. Journal of Experimental Psychology: Animal Behavior classroom behavior. Journal of Experimental Child Psychology, Processes, 14, 36–42. 10, 33–43. Vellend, D. H. (2006). The consequences of genetic diversity Wasserman, E., Franklin, S., & Hearst, E. (1974). Pavlovian in competitive communities. Ecology, 87, 304–311. appetitive contingencies and approach vs. withdrawal to Verhave, T. (1966). The pigeon as a quality-control inspector. conditioned stimuli in pigeons. Journal of Comparative and American Psychologist, 21(2), 109.–115 Physiological Psychology, 86, 616–627. Verplanck, W. S. (1962/1992). Verbal concept “mediators” Watkins, M. J. (1990). Mediationism and the obfuscation of as simple operants (reprinted from: Unaware of where’s memory. American Psychologist, 45, 328–335. awareness, pp. 130–158 in C. W. Eriksen, ed., Behavior References • 373

Watkins, M. J. (1996). Encore: Mediationism and the Wilson, D. S., & Sober, E. (1994). Reintroducing group obfuscation of memory. The Behavior Analyst, 19, 89–110. selection in the human behavioral sciemces. Behavioral and Watson, J. B. (1913). Psychology as the behaviorist views it. Brain Sciences, 17, 585–608. Psychological Review, 20, 158–177. Wilson, F. R. (1998). The hand. New York: Pantheon. Watson, J. B. (1919). Psychology from the standpoint of a behaviorist. Wilson, K. G., & Hayes, S. C. (2000). Why it is crucial Philadelphia: Lippincott. to understand thinking and feeling: An analysis and Watson, J. B., & Rayner, R. (1920). Conditioned emotional application to drug abuse. The Behavior Analyst, 23, 25–43. reactions. Journal of Experimental Psychology, 3, 1–14. Winett, R. A., & Winkler, R. C. (1972). Current behavior Watson, J. D., & Crick, F. H. C. (1953). Molecular structure of modifi cation in the classroom: Be still, be quiet, be docile. nucleic acids. Nature, 171, 737–738. Journal of Applied Behavior Analysis, 5, 499–504. Weiner, H. (1983). Some thoughts on discrepant human– Winner, E. (1979). New names for old things: The emergence animal performances under schedules of reinforcement. of metaphoric language. Journal of Child Language, 6, 469– Psychological Record, 33, 521–532. 491. Weiner, J. (1994). The beak of the fi nch. New York: Knopf. Wittgenstein, L. (1953). Philosophical investigations. New York: Weiss, B. (1983). Behavioral toxicology and environmental Macmillan. health science. American Psychologist, 38, 1174–1187. Wolf, M. M., Risley, T. R., & Mees, H. (1964). Application Weiss, B., & Laties, V. G. (1961). Behavioral thermoregulation. of operant conditioning procedures to the behavior Science, 133, 1338–1344. problems of an autistic child. Behavior Research and Therapy, Weiss, B., & Laties, V. G. (1969). Behavioral pharmacology 1, 306–312. and toxicology. Annual Review of Pharmacology, 9, 297–326. Woodworth, R. S. (1929). Psychology (revised ed.). New York: Wellman, H. M. (1990). The child’s theory of mind. Cambridge, Holt. MA: MIT Press. Wright, A. A. (2012). Memory processing. In T. R. Zentall & Wells, H. G. (1920). The outline of history. Garden City, NY: E. A. Wasserman (Eds.), The Oxford handbook of comparative Garden City Publishing. cognition (pp. 239–260). Oxford: Oxford University Press. Werker, J. F. (1989). Becoming a native listener. American Wright, A. A., Cook, R. G., Rivera, J. J., Shyan, M. R., Scientist, 77, 54–59. Neiworth, J. J., & Jitsumori, M. (1990). Naming, rehearsal, Wertheimer, M. (1959). Productive thinking. New York: Harper and interstimulus interval effects in memory processing. and Row. Journal of Experimental Psychology: Learning, Memory, and West-Eberhart, M. J. (2003). Developmental plasticity and evolution. Cognition, 16, 1043–1059. New York: Oxford University Press. Wright, A. A., & Roediger, H. L., III. (2003). Interference Wetherington, C. L. (1982). Is adjunctive behavior a third processes in monkey auditory list memory. Psychonomic class of behavior? Neuroscience and Biobehavioral Reviews, 6, Bulletin and Review, 10, 696–702. 329–350. Wurster, R. M., Griffi ths, R. R., Findley, J. D., & Brady, White, K. G., & Wixted, J. T. (1999). Psychophysics of J. V. (1977). Reduction of heroin self-administration remembering. Journal of the Experimental Analysis of in baboons by manipulation of behavioral and Behavior, 71, 91–113. pharmacological conditions. Pharmacology Biochemistry and Whitehurst, G. J., & Valdez-Menchaca, M. C. (1988). What is Behavior, 7, 519–528. the role of reinforcement in early language acquisition? Yates, F. A. (1966). The art of memory. Chicago: University of Child Development, 59, 430–440. Chicago Press. Whorf, B. J. (1940). Science and linguistics. Technology Review, Yerkes, R. M. (1907). The dancing mouse. New York: Macmillan. 42, 229–231, 247–248. Yong, E. (2016). I contain multitudes. New York: Ecco. Wilkes, G. (1994). A behavior sample. North Bend, WA: Young, F. A. (1958). Studies of pupillary conditioning. Journal Sunshine Books. of Experimental Psychology, 55, 97–110. Williams, D. R., & Williams, H. (1969). Auto-maintenance Zeigarnik, B. (1927). Das Behalten erledigter und unerledigter in the pigeon: Sustained pecking despite contingent Handlungen. Psychologische Forschung, 9, 1–85. non-reinforcement. Journal of the Experimental Analysis of Zener, K., & McCurdy, H. G. (1939). Analysis of motivation Behavior, 12, 511–520. factors in conditioned behavior. I. Differential effect of Williams, M., & Johnston, J. M. (2002). Training and change in hunger upon conditioned, unconditioned and maintaining the performance of dogs (Canis familiaris) spontaneous salivary secretion. Journal of Psychology, 8, on an increasing number of odor discriminations in a 321–350. controlled setting. Applied Animal Behaviour Science, 78(1), Zentall, T. R., & Levine, J. M. (1972). Observational learning 55–65. and social facilitation in the rat. Science, 178, 1220–1221. 374 • References

Zentall, T. R., & Urcuioli, P. J. (1993). Emergent relations in Zimmerman, J., Hanford, P. H., & Brown, W. (1967). Effects of the formation of stimulus classes by pigeons. Psychological conditioned reinforcement frequency in an intermittent Record, 43, 795–810. free-feeding situation. Journal of the Experimental Analysis Zettle, R. D., & Hayes, S. C. (1982). Rule-governed behavior: A of Behavior, 10, 331–340. potential theoretical framework for cognitive-behavioral therapy. In P. C. Kendall (Ed.), Advances in cognitive- behavioral research and therapy, Volume I (pp. 73–118). New York: Academic Press.

ACKNOWLEDGMENTS

I thank the following sources and the authors for granting permission to use mate- rial from copyrighted works. Full citations appear in the References section. Copy- right by and with permission of the American Association for the Advancement of Science: Fig. 28–1 from Shepard & Metzler (1971). Copyright by and adapted with permission of the Society for the Experimental Analysis of Behavior: Fig. 7–4 from Holz & Azrin (1961); Fig. 11–3,from Reynolds (1961a); Fig. 15–3 from Catania & Reynolds (1968); Fig. 15–7 from Dews (1962); Fig. 18–1 from Geller (1960); Fig. 23-2 from Catania, Shimoff, & Matthews (1989).

A NOTE ON THE COVER ART

The central image on the front cover is Peaceable Kingdom by Edward Hicks, one of several variations on this theme. Reading clockwise from the upper left, the fram- ing works are: Monet’s Cliff at Etretat; Bruegel the Elder’s Tower of Babel (“great” version); Rembrandt’s Conus marmoreus (shell); Henri Rousseau’s Apes in a Grove; Rodin’s Thinker; Earth from space; sample sketches of Darwin’s fi nches; Daumi- er’s Don Quixote; Seurat’s La Grande Jatte; George Richmond’s portrait of Charles Darwin; Rockwell’s The Problem We All Live With; a Canaletto view of San Marco Square in Venice; Volterra’s bust of Michelangelo; da Vinci’s Vetruvius Man; and Constable’s Hay Wain. Allusions to each, some more subtle than others, appear at various points in the text. Along with relevant terms and phrases, the back cover includes fi gures from the text, segments from Muybridge’s photography of animal motion and amateur art by the author (some is distantly related to the front cover art). The central image, just above center, is B. F. Skinner. Other photographs, reading clockwise from noon: Bea Barrett and George Reynolds; the author with Eliot Shimoff dur- ing a classroom pigeon demonstration; Harvard’s Memorial Hall; the author in San Marco Square in Venice; Ivan Pavlov; Deisy de Souza and João Todorov; Terje Sagvolden as a speck in the distance taking a picture in the mountains near Tromsø, Norway; an infant in an Aircrib; Nat Schoenfeld and Fred Keller; Pauline Horne and Fergus Lowe; and Naoko Sugiyama with Koichi Ono and other Japa- nese scholars.