Time, Rate and Conditioning
Total Page:16
File Type:pdf, Size:1020Kb
In press, Psychological Review Time, Rate and Conditioning C. R. Gallistel and John Gibbon University of California, Los Angeles and New York State Psychiatric Institute Abstract We draw together and develop previous timing models for a broad range of conditioning phenomena to reveal their common conceptual foundations: First, conditioning depends on the learning of the temporal intervals between events and the reciprocals of these intervals, the rates of event occurrence. Second, remembered intervals and rates translate into observed behavior through decision processes whose structure is adapted to noise in the decision variables. The noise and the uncertainties consequent upon it have both subjective and objective origins. A third feature of these models is their time-scale invariance, which we argue is a deeply important property evident in the available experimental data. This conceptual framework is similar to the psychophysical conceptual framework in which contemporary models of sensory processing are rooted. We contrast it with the associative conceptual framework. Pavlov recognized that the timing of the pigeons pecked keys to obtain reinforcement conditioned response or CR (e.g., salivation) on fixed interval (FI) schedules, the longer the in a well conditioned subject depended on the fixed interval imposed between the obtaining reinforcement delay, or latency. The longer of one reinforcement and the availability of the interval between the onset of the the next, the longer the pigeon waited after conditioned stimulus or CS (e.g. , the ringing each reinforcement before beginning to peck of a bell) and the delivery of the the key to obtain the next reinforcement. An unconditioned stimulus or US (meat powder), obvious explanation is that his pigeons the longer the latency between CS onset and learned the duration of the interval between a the onset of salivation. An obvious delivered reinforcement and the next arming explanation is that the dogs in Pavlov’s of the key and did not begin to peck until experiment learned the reinforcement latency they judged that the opportunity to obtain and did not begin to salivate until they judged another reinforcement was more or less that the delivery of food was more or less imminent. Again, this is not the sort of imminent. This is not the kind of explanation explanation that Skinner favored, though for that Pavlov favored, because it lacks a clear reasons other than Pavlov’s. basis in reflex physiology. Similarly, Skinner observed that the timing of operant responses In this paper we take the interval-learning was governed by the intervals in the schedule assumption as the point of departure in the of reinforcement. When his analysis of conditioned behavior. We assume that the subjects in conditioning experiments Gallistel and Gibbon 2 do in fact store in memory the durations of the association-froming process in interevent intervals and subsequently recall instrumental conditioning, the reinforcer was those remembered durations for use in the not itself part of the associative structure, it decisions that determine their conditioned merely stamped in the S-R association. behavior. An extensive experimental literature Recently, however, it has been shown that on timed behavior has developed in the last Pavlovian response-outcome (R-O) or few decades (for reviews, see Fantino, outcome-response (O-R) associations are Preston & Dunn, 1993; Gallistel, 1989; important in instrumental conditioning Gibbon & Allan, 1984; Gibbon, Malapani, (Adams & Dickinson, 1981; Colwill & Dale & Gallistel, 1997; Killeen & Fetterman, Rescorla, 1986; Colwill & Delamater, 1995; 1988; Miller & Barnet, 1993; Staddon & Mackintosh & Dickinson, 1979; Rescorla, Higa, 1993). In consequence, it is now widely 1991). Our reading of the most recent trends accepted that the subjects in conditioning in associative theorizing is that, for these and experiments do in some sense learn the other reasons (e.g., Williams, 1982), the two intervals in the experimental protocols. But kinds of conditioning paradigms are no longer those aspects of conditioned behavior that thought to tap fundamentally different seem to depend on knowledge of the association-forming processes. Rather, they temporal intervals are often seen as are thought to give rise to different adjunctive to the process of association associative structures via a single association- formation (e.g., Miller & Barnet, 1993), forming process. which is commonly taken to be the core process mediating conditioned behavior. We In any event, the underlying learning will argue that it is the learning of temporal processes in the two kinds of paradigms are intervals and their reciprocals (event rates) not fundamentally different from the that is the core process in both Pavlovian and perspective of timing theory. The paradigms instrumental conditioning. differ only in the kinds of events that mark the start of relevant temporal intervals or It is our sense that most contemporary alter the expected intervals between associative theorists no longer assume that reinforcements. In Pavlovian paradigms, the the association-forming process itself is animal’s behavior has no effect on the fundamentally different in Pavlovian and delivery of reinforcement. The conditioned instrumental conditioning. Until the behavior is determined by the rate and timing discovery of autoshaping (Brown & Jenkins, of reinforcement when a CS is present 1968), a now widely used Pavlovian relative to the rate and timing of reinforce- procedure for teaching what used to be ment when that CS is not present. In regarded as instrumental responses (pecking a instrumental paradigms, the animal’s behavior key or pressing a lever for food), it was alters the rate and timing of reinforcement. assumed that there were two fundamentally Reinforcements occur at a higher rate when different association-forming processes. One, the animal pecks the key (or presses the which operated in Pavlovian conditioning, lever, etc.) than when it does not. And the required only the temporal contiguity of a time of delivery of the next reinforcement conditioned and unconditioned stimulus. The may depend on the interval since a response- other, which operated in instrumental elicited event such as the previous conditioning, required that a reinforcer stamp reinforcement. In both cases, the essential in the latent association between a stimulus underlying process from a timing perspective and a response. In this older conception of is the learning of the contingency between the Gallistel and Gibbon 3 rate of reinforcement (or expected interval time scale. Superimposability means that the between reinforcements) and some state of normalized curves (or, in the limit, individual affairs (bell ringing vs. bell not ringing or key points) fall on top of each other. We give being pecked vs. key not being pecked, or several examples in what follows, beginning rapid key pecking versus slow key pecking). with Figures 1A and 2. An extremely Thus, we will not treat these conditioning important empirical consequence of the new paradigms separately. We will move back and conceptual framework is that it stimulates forth between them. research to test the limits of fundamentally important principles like this. We develop our argument around models that we have ourselves elaborated, because CR Timing we are more intimately familiar with them. We emphasize, however, that there are The learning of temporal intervals in the several other timing models (e.g., Church & course of conditioning is most directly Broadbent, 1990; Fantino et al., 1993; evident in the timing of the conditioned Grossberg & Schmajuk, 1991; Killeen & response in protocols where reinforcement Fetterman, 1988; Miller & Barnet, 1993; occurs at some fixed delay after a marking Staddon & Higa, 1993). We do not imagine event. In what follows, this delay is called the our own models to be the last word. In fact, reinforcement latency, T. we will call attention at several points to difficulties and lacunae in these models. Our Some well established facts of CR timing goal is to make clear essential features of a are: conceptual framework that differs quite fundamentally from the framework in which •The CR is maximally likely at the conditioning is most commonly analyzed. We reinforcement latency. When there is a expect that as this framework becomes more fixed latency between a marking event widely used, the models rooted in it will such as placement in the experimental become more sophisticated, more complete, chamber, the delivery of a previous and of ever broader scope. reinforcement, the sounding of a tone, the extension of a lever, or the illumination of We also use the framework to call a response key, the probability that a attention to quantitative features of well trained subject will make a conditioning data that we believe have far conditioned response increases as the reaching theoretical implications, most time of reinforcement approaches, notably the many manifestations of time- reaching a maximum at the reinforcement scale invariance. A conditioning result is time latency (Figures 1 and 2). scale invariant if the graph of the result looks the same when the experiment is repeated at a •The distribution of CR onsets and offsets is different time scale, by changing all the scalar: There is a constant coefficient of temporal intervals in the protocol by a variation in the distribution of response common scaling factor, and the scaling factors probability around the latency