Extinction of Cocaine Self-Administration Reveals Functionally and Temporally Distinct Dopaminergic Signals in the Nucleus Accumbens

Home , Extinction (psychology)

Neuron, Vol. 46, 661–669, May 19, 2005, Copyright ©2005 by Elsevier Inc. DOI 10.1016/j.neuron.2005.04.036 Extinction of Cocaine Self-Administration Reveals Functionally and Temporally Distinct Dopaminergic Signals in the Nucleus Accumbens

Garret D. Stuber,1 R. Mark Wightman,1,3 2004), which can then become associated with neutral and Regina M. Carelli1,2,* environmental stimuli. Because of this, dopaminergic 1Curriculum in Neurobiology transmission within the NAc may promote the formation 2 Department of Psychology of an unnatural relationship between drugs of abuse 3 Department of Chemistry and Neuroscience Center and environmental cues (O’Brien et al., 1993; Everitt et The University of North Carolina al., 1999; Hyman and Malenka, 2001; Wise, 2004). How- Chapel Hill, North Carolina 27599 ever, because cocaine has a potent pharmacological effect to inhibit monoamine uptake (Jones et al., 1995; Wu et al., 2001), it has been difficult to separate in- Summary creases in dopamine due to either the primary pharmacological or secondary conditioned effects of the drug. While Pavlovian and operant conditioning influence Microdialysis studies in self-administrating animals drug-seeking behavior, the role of rapid dopamine have shown that NAc dopamine decreases over min- signaling in modulating these processes is unknown. utes leading up to a lever press for cocaine and then During self-administration of cocaine, two dopami- increases slowly after the drug infusion (Wise et al., nergic signals, measured with 100 ms resolution, oc- 1995). When dopamine changes are examined with curred immediately before and after the lever press greater temporal resolution, electrochemical studies (termed pre- and postresponse dopamine transients). have revealed phasic changes in dopamine (lasting Extinction of self-administration revealed that these only a few seconds), which are time locked to the oper- two signals were functionally distinct. Preresponse ant response for the drug (Phillips et al., 2003b; Stuber transients, which could reflect the motivation to ob- et al., 2005). Consistent with this, subsequent exposure tain the drug, did not decline during extinction. Re- to drug-associated cues led to an increase of dopa- markably, postresponse dopamine transients attenu- mine within the NAc (Ito et al., 2000; Phillips et al., ated as extinction progressed, suggesting that they 2003b) and can lead to the relapse of drug taking, thus encode the learned association between environmen- playing an important role in the manifestation and tal cues and cocaine. A third type of dopamine tran- maintenance of addiction (Weiss et al., 2001; Crombag sient, not time locked to overt stimuli, decreased in et al., 2002). frequency during extinction and correlated with cal- While reward-related learning utilizes dopaminergic culated cocaine concentrations. These results show transmission to associate environmental stimuli with a that dopamine release transients involved in different primary reinforcer (Schultz, 1998; Ito et al., 2000), the aspects of cocaine self-administration are highly precise role that dopaminergic signaling plays during plastic—differentially governed by motivation, learned drug self-administration remains unknown. To address associations linked with environmental stimuli, and this, cyclic voltammetry with 100 ms temporal resolu- the pharmacological actions of cocaine. tion was used to monitor rapid dopamine release during regular cocaine self-administration (maintenance), Introduction extinction, and reinstatement. The superb temporal resolution of this technique revealed that dopaminergic Dopaminergic neurons originating in the ventral teg- signals play three temporally and functionally distinct mental area and projecting to the amygdala, prefrontal roles that are differentially influenced by extinction. cortex, and nucleus accumbens (NAc) are hypothe- sized to relate information about a primary reinforcer, Results such as cocaine, with goal-directed behavior and/or environmental stimuli (McClure et al., 2003; Schultz, Cocaine Self-Administration during 2004). This idea is supported by electrophysiological Extinction/Reinstatement studies in behaving animals, which have demonstrated Intravenously catheterized rats were initially trained to that dopaminergic neurons are initially activated by pri- self-administer cocaine (0.33 mg/infusion; delivered mary reinforcers but adapt over repeated pairings to over 6 s) in 2 hr daily sessions and were surgically pre- respond to conditioned stimuli that are associated with pared for monitoring rapid dopamine signaling in the the reinforcer (Schultz, 1998). Furthermore, the degree NAc core after 2–3 weeks of training. Following recov- of phasic activity of dopamine neurons is a function of ery from surgery and 3–5 days of retraining, rats un- the probability in which a conditioned stimulus predicts derwent a within-session extinction/reinstatement ex- a subsequent reward (Fiorillo et al., 2003), suggesting periment conducted in three phases. In phase one that dopaminergic transmission can adapt to changing (termed maintenance), rats completed two to four lever degrees of uncertainty of which a cue can predict a press responses in quick succession followed by five reward. lever presses with mean interinfusion intervals of 439 ± Virtually all drugs of abuse cause the release of dopa- 33 s. During maintenance, each lever press resulted in mine in the NAc (Di Chiara and Imperato, 1988; Wise, an intravenous cocaine infusion (0.33 mg, 6 s) paired with drug-associated cues (20 s). Lever pressing beha- *Correspondence: [email protected] vior remained stable throughout maintenance, with no Neuron 662

Figure 1. Lever Press Behavior during Main- tenance, Extinction, and Reinstatement of Cocaine Self-Administration (A) Representative behavioral response rec- ord from one animal throughout the three experimental phases. Each vertical tick mark represents a lever press response. The animal was reinstated to self-administer cocaine after extinction by issuing a noncontingent “priming” infusion of cocaine paired with a 20 s presentation of the drug-associated cues (indicated by the arrow). (B–D) Interresponse intervals for lever presses in maintenance, extinction, and reinstatement.

significant difference in the interresponse interval over dopamine signaling relates to the association of lever the trials [F(4, 39) = 0.48; p = 0.74; Figures 1A and 1B]. press responding and the cues that predict cocaine de- In phase two (extinction), saline was substituted for co- livery (Phillips et al., 2003b). Therefore, we hypothe- caine. Each lever press resulted in saline infusion (6 s) sized that the preresponse phasic dopamine signal paired with the drug-associated cues (20 s). This led to would remain during extinction, while the postresponse a dramatic increase in the rate of responding measured dopamine signal would attenuate as the learned asso- over the first ten trials of extinction [F(9, 79) = 6.2; p % ciation between the reinforcer and the drug-associated 0.0001; Figures 1A and 1C] and the eventual cessation cues decreased across extinction trials. For analysis of responding after 20.3 ± 3.0 trials. Following 30 min purposes, extinction was broken up into early (first ten) of no responding, phase three (reinstatement) was initi- and late (last five) lever presses. Figure 3A shows a sin- ated. Specifically, self-administration behavior was re- gle trial in early extinction. Even though each lever established by a noncontingent intravenous infusion of press resulted in a saline infusion paired with the drug- cocaine (0.33 mg, 6 s) paired with the same drug-asso- associated cues, both the pre- and postresponses ciated stimuli. During the reinstatement phase, rats were associated with the lever press. A repeated mea- were allowed to respond five times, in which each response resulted in cocaine paired with the drug- sures ANOVA revealed that preresponse DA transients associated cues, as in maintenance. Response rates remained remarkably stable in amplitude throughout decreased and eventually stabilized over the five early extinction and were not significantly different rela- reinstatement trials [F(4, 39) = 3.1; p < 0.05; Figure 1D]. tive to the maintenance phase [F(10, 87) = 0.83; p = 0.60; Figure 3B]. In contrast, the amplitude of postre- Phasic Dopamine Signals Are Associated with the sponse dopamine transients was maintained during Operant Response during Maintenance early extinction (Figures 3A and 3C) but significantly de- Phasic dopamine time locked to a single operant re- clined as extinction proceeded [repeated measures sponse during maintenance is shown in Figure 2A. As ANOVA; F(10, 87) = 2.44; p % 0.05; Figure 3C]. Post hoc reported previously (Phillips et al., 2003b; Stuber et al., comparisons revealed that the postresponse dopamine 2005), dopaminergic signals at the lever press in main- signal on trial 10 was significantly less than the average tenance have two distinct components: an initial in- dopamine signal during the maintenance phase (p % crease in dopamine in the time interval 10 s before the 0.05). lever press response (termed preresponse) followed by As the extinction phase proceeded, a clear dissoci- a larger increase within 10 s after the response (termed ation between pre- and postresponse dopamine tran- postresponse). The mean preresponse dopaminergic sients became apparent. Figure 3D shows a single lever signal was 52.6 ± 6.2 nM and remained stable in ampli- press response (from the same animal shown in Figure tude from trial to trial during the maintenance phase 3A) during late extinction. A dopamine preresponse is [F(4, 39) = 0.57; p = 0.73; Figure 2B]. Similarly, the mean evident; however, there is no significant dopamine postresponse dopaminergic signal was 74.4 ± 5.8 nM postresponse. A repeated measures ANOVA revealed and also did not significantly change in amplitude throughout the maintenance phase [F(4, 39) = 0.31; p = that the amplitude of the preresponse dopamine tran- 0.75; Figure 2C]. sients during the last five trials of extinction did not significantly decrease relative to maintenance [F(5, Extinction of Cocaine Self-Administration Reveals 47) = 1.55; p = 0.20; Figure 3E]. In contrast, the ampli- a Functional Dissociation between Preresponse tude of the postresponse signal was significantly atten- versus Postresponse Dopamine Signaling uated during the last five trials of extinction relative to We have previously shown that preresponse dopamine postresponses during maintenance [F(5, 47) = 14.89; transients promote cocaine seeking, while postresponse p % 0.0001; Figure 3F]. Modulation of Dopamine by Extinction/Reinstatement 663

drug-associated cues (reinstatement). Figure 4A shows that dopamine was again associated with the operant response during reinstatement. A repeated measures ANOVA revealed no significant change in the amplitude of the preresponse dopamine transients during reinstatement relative to maintenance or extinction [F(6, 55) = 0.68; p = 0.66; Figure 4B], further demonstrating that the preresponse is variable, but persistent. In contrast, the postresponse dopamine transients, while attenuated during extinction, reemerged by the second trial of the reinstatement phase. A repeated measures ANOVA revealed a significant main effect of trial num- ber on the postresponse dopamine amplitude during reinstatement [F(7, 63) = 4.61; p % 0.0005; Figure 4C]. Post hoc tests showed that the postresponse dopamine transients were significantly lower during the “priming” infusion and during the first lever press of reinstatement relative to the average dopamine concentration during the maintenance phase.

“Spontaneous” Dopamine Transients Correlate with Calculated Brain Cocaine Concentrations While dopamine transients are consistently associated with each operant response during maintenance and reinstatement (Figures 2 and 4), we have also identified another aspect of dopamine signaling: spontaneous dopamine transients, which are not associated to any overt behavior (e.g., lever press) or environmental stimuli. These transients occur randomly and increase in Figure 2. Phasic Dopamine in the NAc Time Locked to the Operant frequency upon application of cocaine (Stuber et al., Response during the Maintenance Phase 2005). Examples of spontaneous dopamine transients (A) Pre- and postresponse dopamine associated with the operant response for a single trial. The lever press response occurs at t = occurring during each of the three experimental phases 0, the black bar indicates the duration of the 0.33 mg cocaine infu- are shown in Figures 5A–5C. Across all rats (n = 8), the sion, and the shaded bar indicates the presentation of the drug- frequency of spontaneous dopamine transients was associated cues (lasts for 20 s after the lever press response). 5.6 ± 1.7 min−1, 1.8 ± 0.4 min−1, 3.2 ± 0.8 min−1 in main- Asterisks indicate signals that are dopamine. Inset shows the cyclic tenance, extinction, and reinstatement, respectively voltammogram from the maximum dopamine change (solid line) [F(2, 23) = 5.6; p % 0.05]. To determine whether falling compared to the voltammogram for electrically evoked dopamine release from the same animal. Linear regression analysis revealed cocaine levels during extinction correlated with the at- a high correlation (r2 = 0.94) between the voltammograms, indica- tenuation of the postresponse (see above) or spontane- tive that the measured signal was dopamine. ous dopamine transients, brain cocaine levels were (B and C) Mean (±SEM) preresponse (B) or postresponse (C) dopa- computed using the model of Pan et al. (1991). This mine concentrations during maintenance. model revealed that calculated cocaine levels fluctu- ated little during maintenance and reinstatement and fell exponentially during extinction (results from one an- Linear regression analysis across all animals (n = 8) imal are shown in Figure 5D; dopamine transient fre- and all trials of the session revealed no significant cor- quency across the three phases is shown for the same relation with the amplitude of postresponse time- animal in Figure 5E). locked transients and calculated cocaine concentra- During extinction, calculated cocaine levels fall (Fig- tions [r2 = 0.01; F(1, 136) = 1.4; p = 0.24; slope not sig- ure 6A). At the same time, the frequency of spontane- nificantly different from zero]. The brain cocaine con- ous dopamine transients also decline (Figure 6B). In- centration was calculated using the method of Pan et deed, linear regression analysis (Figure 6C) revealed a al. (1991). Similarly, changes in the amplitude of prere- highly significant correlation between the rate of spon- sponse dopamine transients were also not correlated taneous dopamine transients and calculated brain co- with calculated brain cocaine concentrations [r2 = 0.02; caine levels during extinction across all animals (r2 = F(1, 132) = 3.1; p = 0.08; slope not significantly different 0.93; p % 0.0001). from zero].

The Decrease in Spontaneous Dopamine Transient Postresponse Dopamine Transients Become Frequency Precedes the Increase in Lever Pressing Reassociated with the Operant Response and Activity during Extinction during Reinstatement Analysis of locomotor activity during extinction, which After 30 min without lever press responding, cocaine in this study corresponded with the rate of lever press- self-administration was reestablished by noncontingent ing, was conducted to determine where in time the de- intravenous infusion of 0.33 mg cocaine paired with the crease in spontaneous dopamine transient frequency Neuron 664

Figure 3. Phasic Dopamine in the NAc within ±10 s of the Operant Response during Early and Late Extinction of Cocaine Self-Admin- istration Early extinction: (A) Pre- and postresponse dopamine time locked to the operant response for a single trial in early extinction. The lever press response occurs at t = 0, the clear bar indicates the duration of the saline infusion, and the shaded bar indicates the presentation of the drug-associated cues (lasts for 20 s after the lever press response). Asterisks indicate signals that are dopamine. Inset shows the cyclic voltammogram at the maximal dopamine response (solid line) compared to the voltammogram for electrically evoked dopamine release from the same animal. Linear regression analysis revealed a high correlation (r2 = 0.94) between the voltammograms, indicative that the measured signal was dopamine. (B and C) Mean (±SEM) preresponse (B) or postresponse (C) dopamine concentrations during the first ten trials of extinction compared to maintenance (“M”). Late extinction: (D) Lack of postresponse dopamine time locked to the operant response for a single trial in late extinction from the same animals shown in (A). Aster- isks denote that the cyclic voltammograms for the preresponse were R 0.75. Inset voltammograms, taken less than 1 s after the lever press response, show little similarity to that of electrically evoked dopamine release, indicative that the measured signal is not dopamine. (E and F) Mean (±SEM) preresponse (E) or postresponse (F) dopamine concentrations during the last five trials of extinction compared to maintenance (“M”). *p % 0.05, **p % 0.01 compared to maintenance. occurs relative to the increase in behavioral respond- Phasic Dopamine Time Locked to the Operant ing. While individual spontaneous dopamine transients Response during Maintenance, Extinction, do not appear to be time locked to specific behavioral and Reinstatement or environmental stimuli, the decrease in overall tran- Pre- and postresponse dopamine transients serve dis- sient frequency occurs before locomotor activity in- tinct functions within the NAc (Phillips et al., 2003b). creases. Figure 6B shows that spontaneous dopamine Electrically evoked dopamine transients during a co- transient frequency is maximal at 4 min into extinction, caine self-administration session induce the animal to while behavioral activity is maximal at 8 min (Figure 6D). press the lever within a similar time period in which the preresponse dopamine transients occur, suggesting Discussion that transients before the lever press promote the goal- directed behavior or seeking of the drug (Phillips et al., The present data demonstrate that phasic dopamine 2003b). Consistent with this idea, preresponse tran- signals in the NAc play three distinct roles as revealed sients remain intact and linked with lever pressing by their variation during the maintenance, extinction, throughout the extinction phase. and reinstatement of cocaine self-administration. With While the preresponse transients may promote co- respect to the lever press response, two distinct signals caine seeking, the postresponse dopamine transients are observed: preresponse and postresponse dopa- may encode for learned associations between the mine transients. The preresponse transients are vari- drug-associated stimuli and cocaine. Previous studies able in amplitude but persist over the course of all three (Ito et al., 2000; Phillips et al., 2003b) have demon- phases of the experiment. In contrast, postresponse strated that NAc dopamine increases upon noncontin- dopamine transients were stable during the mainte- gent presentation of drug-associated cues and that in- nance phase, decreased in amplitude during extinction, activation of either the ventral tegmental area or the and were reassociated with the lever press during NAc core (Di Ciano and Everitt, 2004) can reduce re- reinstatement. A third mode of signaling, an increased sponding for cocaine-associated stimuli. Consistent frequency of spontaneous dopamine transients, was with this, postresponses occur during the initial phase observed when calculated brain cocaine concentra- of extinction (Figure 3C) even though cocaine is with- tions were high. Their frequency declined during extinc- held. As extinction progressed, the postresponse am- tion and was restored during reinstatement. These plitudes decreased but at a different rate than the in- spontaneous transients correlated well with calculated crease in lever pressing behavior. This shows that the brain cocaine levels, while the pre- and postresponse postresponse transients do not linearly encode for the dopamine transients did not. Thus, these dopamine behavior but instead reflect a conditioned response transients, revealed by fast-scan cyclic voltammetry, (i.e., related to operant and/or classical conditioning reflect both conditioned and pharmacological actions. aspects of the self-administration task). Modulation of Dopamine by Extinction/Reinstatement 665

tively, these findings support the view that the postresponse dopamine transients reflect conditioned as opposed to pharmacological actions of cocaine. The data here are also consistent with the hypothe- sized role of dopaminergic signaling as a prediction er- ror signal (Schultz et al., 1997; McClure et al., 2003; Redish, 2004). Phasic activation of dopaminergic neurons is determined by how well a stimulus predicts the reward (Fiorillo et al., 2003). Here, as extinction progressed, both the probability that the drug-associated cues predict cocaine delivery and the postresponse dopamine transient amplitude decreased. Importantly, the dynamic decrease in postresponse dopamine signaling during extinction reported here is exactly the response predicted by temporal difference reinforcement learning models for learning-associated signals (Schultz et al., 1997; Redish, 2004). Likewise, the dissociation between the drug-associated cues and cocaine delivery may explain why no significant increase in dopamine is observed relative to the priming infusion of cocaine paired with drug-associated cues following extinction. At this time, the probability that the cues predict cocaine delivery is the lowest. While the present findings demonstrate that time-locked DA responses are highly plastic over short time periods, future studies are nec- essary to determine whether these signals are modulated with other more long-duration drug-seeking para- digms that mirror human drug withdrawal and relapse.

Spontaneous Dopamine Transients Are Induced Figure 4. Phasic Dopamine in the NAc within ±10 s of the Operant by the Pharmacological Effects of Cocaine Response during Reinstatement of Cocaine Self-Administration While two different types of dopamine signals occurred (A) Pre- and postresponse dopamine time locked to the operant that are associated with the operant response for co- response for a single trial during reinstatement. The lever press caine, spontaneous dopamine transients occurred ran- response occurs att=0,theblack bar indicates the duration of domly in time, but their frequency correlated with cal- the 0.33 mg cocaine infusion, and the shaded bar indicates the presentation of the drug-associated cues (lasts for 20 s after the culated cocaine levels. Because they occur in both lever press response). Asterisks indicate signals that are dopamine. drug- and self-administration-naive animals (Stuber et Inset shows the cyclic voltammogram from the trial (solid line) com- al., 2005) as well as animals with a history of cocaine pared to the voltammogram for electrically evoked dopamine re- self-administration, they arise from a pharmacologi- lease from the same animal. Linear regression analysis revealed a cal effect of the drug. Similarly, both noncontingent high correlation (r2 = 0.88) between the voltammograms, indicative (Schwarz et al., 2004) and self-administered (Wise et that the measured signal was dopamine. (B and C) Mean (±SEM) preresponse (B) or postresponse (C) dopa- al., 1995) cocaine led to an increase in tonic dopamine mine concentrations during the five trials of reinstatement, the levels, as measured by microdialysis, and are linearly average concentration during late extinction (“E”), or the reinstating related to cocaine concentrations (Nicolaysen et al., prime (“P”), compared to maintenance (“M”). **p % 0.01 compared 1988). Indeed, increases in tonic dopamine occurring to maintenance. over minutes may simply be due to a summation of all spontaneous dopamine transients. Furthermore, spon- It is important to note that a clear dissociation exists taneous dopamine transients may be able to diffuse between pharmacological levels of cocaine and the oc- greater distances and interact with a greater population currence of postresponse dopamine transients. First, of dopamine receptors in the presence of cocaine. As modeled cocaine concentrations did not correlate with such, the significance of spontaneous events may be the amplitude of these events. Second, examination of to activate a larger neuronal network in the presence Figure 4C shows that no postresponse transients were of cocaine compared to nondrug conditions, in which observed during late extinction, following the cocaine spontaneous dopamine transients are observed much prime, and relative to the first lever press during rein- less frequently (Roitman et al., 2004). statement. However, calculated cocaine levels are dra- Although spontaneous dopamine transients do not matically different at these time points (see Figure 5D). appear to be time locked to any overt behavior or envi- Finally, during early extinction (by trial 4) calculated co- ronmental stimuli, they may play an important role by caine concentrations were similar to levels modeled encoding for the hedonic aspects of drug-taking. Con- during the first lever press in reinstatement, yet postre- sistent with this, locomotor activity, which in this study sponse dopamine transients were still observed at this was associated with the rate of lever pressing during point in extinction (Figure 3C), as opposed to their ab- extinction is maximal following a decline in spontane- sence during trial 1 of reinstatement (Figure 4C). Collec- ous dopamine transient frequency (Figure 6D). This is Neuron 666

Figure 5. Brain Cocaine Concentrations and Frequency of “Spontaneous” Dopamine Transients Decrease during Extinction (A–C) Traces showing spontaneous dopamine transients during maintenance (A), late extinction (B), or reinstatement (C) from the same animal. Asterisks denote voltammograms at that time which have an r2 R 0.75 compared to stimulated dopamine release. (D) Calculated brain cocaine concentrations during maintenance, extinction, and reinstatement of cocaine self-administration. The animal was reinstated to self-administer cocaine by administering a 0.33 mg priming infusion of cocaine paired with a 20 s exposure to the drug-associated cues (denoted by the arrow). (E) Frequency of spontaneous dopamine transients during maintenance, extinction, and reinstatement from the same animal used for the calculated brain cocaine concentrations (D).

also consistent with the “threshold hypothesis” pro- ant response for cocaine, natural rewards, and the posed by Wise et al. (1995), in which falling tonic dopa- stimuli that are associated with these (Carelli, 2004). mine concentrations were thought to trigger successive Furthermore, within-session extinction experiments con- operant responses for the drug. ducted during cocaine self-administration (Carelli and Ijames, 2000) show that NAc neurons exhibiting postre- The Role of Dopamine Signaling sponse patterned discharges are attenuated during ex- within NAc Microcircuits tinction and again become phasically active during In this study, we describe dopamine signals that occur reinstatement in much the same way that phasic post- during three different phases of cocaine self-admin- response dopamine signaling is seen in the NAc. Fur- istration. While each of these are distinctly modulated thermore, NAc neurons exhibiting preresponse pat- by the dissociation of cocaine from the drug-associ- terned discharges are not altered during extinction ated stimuli during extinction, little is known about the similar to preresponse dopamine transients reported function dopamine transients play within the NAc to in- here. Whether phasic dopamine signaling is directly af- fluence subsequent cellular activity. NAc neurons show fecting cells within the NAc remains unknown, but un- distinct patterned discharges time locked to the oper- derstanding this will further elucidate the neurobiologi-

Figure 6. Declining Cocaine Concentrations and Dopamine Transient Frequency Precede the Increase in Behavioral Activity (A) Average (mean ± SEM) cocaine concentrations across all animals during the extinction phase. (B) Average (mean ± SEM) spontaneous dopamine transient frequency across all animals during the extinction phase. (C) Linear regression analysis showing the correlation of spontaneous dopamine transient frequency with brain cocaine concentrations (from [A] and [B]) over the extinction phase. (D) Average (mean ± SEM) locomotor activity across all animals over the extinction phase. Time 0 is the beginning of the extinction phase immediately after the last maintenance response. Modulation of Dopamine by Extinction/Reinstatement 667

cal function of dopamine within the NAc, and the role 400 V/s), stored to a PC using software written in LabVIEW (Na- it plays in reinforcement and drug addiction. tional Instruments, Austin, TX). Dopamine release was optimized within the NAc core by adjusting the vertical position of the working Experimental Procedures electrode (0.1 mm increments). All recording sites had a signal-to- noise ratio of electrically evoked (24 biphasic pulses, 60 Hz, 120 ␮ Cocaine Self-Administration A, 2 ms/phase) dopamine release of at least 30. The electrode Male Sprague-Dawley (Harlan, Raleigh, NC) rats (280–300 g; n = 8) was then locked in place, and the voltammetric waveform was ap- were housed individually and kept on a 12:12 light cycle (lights on plied for an additional 45–60 min (Phillips et al., 2003a). w at 07:00). Rats had ad libitum access to food, while water intake Following equilibration of the voltammetric electrode ( 1 hr), the was restricted to 30 ml/day to maintain body weight. Rats were experiment was begun by turning on white noise and illuminating anesthetized with ketamine hydrochloride (100 mg/kg i.m.) and xy- the cue light above the lever, signaling drug availability. Rats typi- lazine hydrochloride (20 mg/kg i.m.) and implanted with a chronic cally responded two to four times in quick succession, then three indwelling SILASTIC catheter into their right jugular vein as de- distinct experimental phases followed: maintenance, extinction, scribed previously (Carelli and Deadwyler, 1996). A syringe pump and reinstatement. The maintenance phase consisted of five lever was connected to a liquid swivel system in sound-attenuated ex- press responses in which each response was reinforced with a 0.33 perimental chambers, which enabled intravenous infusion of co- mg infusion of cocaine (delivered over 6 s), paired with a 20 s pre- caine. An additional nine animals were removed from the study; sentation of the drug-associated cues (as described above). Fol- four were removed due to faulty i.v. catheters, four had no detecta- lowing maintenance, the extinction phase of the experiment began, ble electrically evoked DA release at the beginning or end of the and heparinized saline was substituted for cocaine. During extinc- experiment, and one would not initiate lever pressing. tion, each lever press response resulted in a 6 s infusion of saline Following a 1 week recovery period, rats were trained daily to paired with a 20 s presentation of the drug-associated cues. This self-administer i.v. cocaine in 2 hr sessions. At the beginning of caused a dramatic increase in the rate of responding (see Figure 1) each self-administration session, rats were placed in the operant and eventually the termination of lever presses. The extinction chambers (Med Associates, St. Albans, VT), connected to the sy- phase of the experiment was determined to be complete following ringe pump, and a cue light located 6.5 cm above the lever was 30 min without a lever press. The reinstatement phase was initiated illuminated to signal drug availability. Each lever depression re- by noncontingently priming each animal with an infusion of cocaine sulted in a 6 s cocaine infusion (FR-1 schedule; 0.33 mg/infusion (0.33 mg/6 s) paired with the drug-associated cues (20 s). In seven dissolved in sterile heperanized saline) via a computer-controlled of eight rats, a single priming infusion was sufficient to reinstate syringe pump. Cocaine infusions were paired with the immediate responding on the lever (the remaining rat was reinstated with three onset of a tone/house light stimulus (65 dB, 2900 Hz) lasting for priming infusions). Following the priming infusion, rats were al- 20 s. During the 20 s postresponse interval, responding on the lever lowed to respond five times during reinstatement in which each had no programmed consequences. Stable self-administration be- response resulted in an infusion of cocaine (0.33 mg/6 s) paired havior was typically achieved by 10 to 15 training sessions. Once with a 20 s presentation of the drug-associated cues. stable self-administration behavior was achieved, rats were surgi- After the experiment, the VTA was again electrically stimulated cally prepared for voltammetric recording. Behavioral events during to ensure that viable dopamine release could still be detected at self-administration were logged on a PC with Med-PC software the carbon fiber electrode. The carbon fiber electrode was then (Med Associates, St. Albans, VT). A separate PC for voltammetry removed, cleaned, and calibrated with 1 ␮M dopamine in a flow data acquisition was synchronized with Med-PC via a series of injection analysis system (Phillips et al., 2003a). TTL pulses. Signal Identification and Correction Voltammetric Recording Chemical identification for dopamine was obtained from the back- Implantation and voltammetric recording procedures were carried ground-subtracted cyclic voltammograms (Heien et al., 2003; Phil- out as previously described (Phillips et al., 2003a). Rats were anes- lips and Wightman, 2003). Typical noise levels were equivalent to thetized with ketamine hydrochloride (100 mg/kg i.m.) and xylazine 6 nM dopamine. Only signals larger than this were evaluated for hydrochloride (20 mg/kg i.m.) and placed in a stereotaxic frame. the presence of dopamine. Collected data files were evaluated with A guide cannula (Bioanalytical Systems, West Lafayette, IL) was a program written in LabVIEW. To examine the cyclic voltammo- positioned above the NAc core (+1.3 mm AP, +1.3 mm ML, with its grams within ±10 s of the lever press, a local minimum in the cur- tip −2.5 mm DV; all coordinates relative to bregma). An Ag/AgCl rent (0.5 s duration) at the potential where dopamine is oxidized reference electrode was placed contralateral to the guide cannula. was found during the 10 s preceding each lever press. The cyclic All items were secured to the skull with machine screws and cra- voltammograms in this interval were subtracted from the remainder nioplastic cement. A detachable micromanipulator containing a of this set. To evaluate each cyclic voltammogram in the 20 s in- carbon fiber electrode (75–100 ␮m length cylinders, T-650; Amoco, terval around the lever press for the presence of dopamine, it was Greenville, SC) was inserted into the guide cannula, and the compared to a template cyclic voltammogram obtained from stim- electrode was lowered into the dorsal NAc core. A bipolar stimulat- ulated dopamine release, and a correlation coefficient for the sim- ing electrode was placed directly above the ventral tegmental area ilarity of their shape was obtained. In vitro data analysis revealed (−5.2 mm AP, +1.0 mm ML, and −7.7 to −8.8 mm DV). It was lowered that a cutoff of r2 R 0.75 distinguished dopamine from interfer- in 0.1 mm increments until electrically evoked (60 biphasic pulses, ences (Heien et al., 2003) such as ascorbic acid, DOPAC, ionic 60 Hz, 120 ␮A, 2 ms/phase) dopamine release was detected at the changes in the extracellular space (e.g., changes in local pH [Ven- carbon fiber electrode. The stimulating electrode was then secured ton et al., 2003]), and noise. Typically cyclic voltammograms near in place with cranioplastic cement, and the carbon fiber electrode the lever press had a correlation coefficient of r2 R 0.80. Current was removed and replaced with a stylet. Following voltammetric recorded at the potential where dopamine is oxidized was pH cor- surgery, all rats were allowed to recover to their presurgery body rected as described previously (Stuber et al., 2005). Amplitude of weight. dopamine transients time locked to the lever press was calculated as described in detail previously (Stuber et al., 2005). Briefly, for Extinction/Reinstatement Procedure Following recovery from surgery, rats were retrained for an addi- postresponse dopamine transients, the first transient that occurred tional 2–5 days to reestablish their self-administration behavior. On immediately after the lever press response was used for data the day of the experiment, a new carbon fiber electrode was low- analysis (Stuber et al., 2005). ered into the NAc core. The carbon fiber and Ag/AgCl electrodes were connected to a head-mounted voltammetric amplifier at- Identification of Spontaneous Dopamine Transients tached to a commutator (Med Associates, St. Albans, VT) located Spontaneous dopamine transients were identified using an algo- at the top of the test chamber. Voltammetric recordings were made rithm written in LabVIEW. The average of five voltammograms was every 100 ms by applying a triangular waveform (−0.6 V to +1.4 V, subtracted from a voltammogram recorded 1 s later, and this pro- Neuron 668

cedure was followed sequentially for each cyclic voltammogram in Carelli, R.M., and Deadwyler, S.A. (1996). Dose-dependent transi- the file. Template voltammograms obtained from stimulated dopa- tions in nucleus accumbens cell firing and behavioral responding mine release were statistically compared to the resulting sub- during cocaine self-administration sessions in rats. J. Pharmacol. tracted voltammograms using linear regression analysis. Sponta- Exp. Ther. 277, 385–393. neous dopamine signals were identified if their voltammograms Carelli, R.M., and Ijames, S.G. (2000). Nucleus accumbens cell fir- 2 R had an r 0.75 compared to stimulated dopamine release (Heien ing during maintenance, extinction, and reinstatement of cocaine et al., 2003). Spontaneous dopamine transients were considered self-administration behavior in rats. Brain Res. 866, 44–54. to be separate events if they were separated by at least 0.5 s by voltammograms in which dopamine was not detected. Crombag, H.S., Grimm, J.W., and Shaham, Y. (2002). Effect of dopamine receptor antagonists on renewal of cocaine seeking by reex- Modeling Brain Cocaine Concentrations posure to drug-associated contextual cues. Neuropsychopharma- Brain cocaine concentrations throughout maintenance, extinction, cology 27, 1006–1015. and reinstatement were computed using the model for rats chroni- Di Chiara, G., and Imperato, A. (1988). A. Drugs abused by humans cally treated with i.v. cocaine developed by Pan et al. (1991). Briefly, preferentially increase synaptic dopamine concentrations in the the brain cocaine concentration, Cbrain, is expressed by the equa- mesolimbic system of freely moving rats. Proc. Natl. Acad. Sci. −αt −βt tion Cbrain =A*(e −e ), where A = 9.65 (a multiplicative factor USA 85, 5274–5278. α β included the injected cocaine dose), = 0.097, = 0.642, and t is Di Ciano, P., and Everitt, B.J. (2004). Contribution of the ventral the time in min since the cocaine infusion. The time course of each tegmental area to cocaine-seeking maintained by a drug-paired cocaine infusion was calculated independent of the others. The conditioned stimulus in rats. Eur. J. Neurosci. 19, 1661–1667. actual brain cocaine concentration was obtained by summing the Everitt, B.J., Parkinson, J.A., Olmstead, M.C., Arroyo, M., Robledo, cocaine concentrations that each infusion contributed in 30 s intervals using a macro written in Microsoft Excel. P., and Robbins, T.W. (1999). Associative processes in addiction and reward. The role of amygdala-ventral striatal subsystems. Ann. Locomotor Activity Analysis N Y Acad. Sci. 877, 412–438. Locomotor activity was quantified offline by analysis of videotapes Fiorillo, C.D., Tobler, P.N., and Schultz, W. (2003). Discrete coding from the extinction/reinstatement sessions. The operant chamber of reward probability and uncertainty by dopamine neurons. Sci- was divided into four equal quadrants, and an activity count of ence 299, 1898–1902. one was made every time the animal’s head completely entered a Heien, M.L., Phillips, P.E., Stuber, G.D., Seipel, A.T., and Wightman, different quadrant. Locomotor activity was analyzed in 2 min bins, R.M. (2003). Overoxidation of carbon-fiber microelectrodes en- consistent with the length of voltammetric data files. hances dopamine adsorption and increases sensitivity. Analyst 128, 1413–1419. Histology Hyman, S.E., and Malenka, R.C. (2001). Addiction and the brain: Following completion of experiments, rats were deeply anesthe- the neurobiology of compulsion and its persistence. Nat. Rev. Neu- tized (ketamine, 150 mg/kg; xylazine, 20 mg/kg), and a stainless rosci. 2, 695–703. steel electrode (50 ␮m tip radius), housed in the same micromanipulator used during the experiment, was lowered to the recording Ito, R., Dalley, J.W., Howes, S.R., Robbins, T.W., and Everitt, B.J. site, and an electrolytic lesion was made (50 ␮A, 10 s). Rats were (2000). Dissociation in conditioned dopamine release in the nucleus then transcardially perfused with physiological saline, followed by accumbens core and shell in response to cocaine cues and during 4% paraformaldehyde. Brains were stored in 4% paraformalde- cocaine-seeking behavior in rats. J. Neurosci. 20, 7489–7495. hyde for at least 3 days and frozen, and 50 ␮m sections were made Jones, S.R., Garris, P.A., and Wightman, R.M. (1995). Different ef- using a cryostat. Brain sections were mounted on slides, stained fects of cocaine and nomifensine on dopamine uptake in the Cau- with thionin, and coverslipped. All recording sites were verified to date-Putamen and nucleus accumbens. J. Pharmacol. Exp. Ther. be located in the core region of the NAc based on the atlas of 274, 396–403. Paxinos and Watson (1997). McClure, S.M., Daw, N.D., and Montague, P.R. (2003). A computational substrate for incentive salience. Trends Neurosci. 26, 423– Statistical Analysis 428. All behavioral and voltammetric data were analyzed in GraphPad Prism and InStat (GraphPad Software, San Diego, CA). One-way Nicolaysen, L.C., Pan, H.T., and Justice, J.B., Jr. (1988). Extracellu- repeated measures ANOVAs were performed on data from each of lar cocaine and dopamine concentrations are linearly related in rat the three experimental phases. Upon conformation of a main effect, striatum. Brain Res. 456, 317–323. Dunnett’s test for multiple comparisons post hoc analysis was per- O’Brien, C.P., Childress, A.R., McLellan, A.T., and Ehrman, R. (1993). formed for all voltammetric data that were compared to the Developing treatments that address classical conditioning. NIDA average concentration during the maintenance phase. For all other Res. Monogr. 135, 71–91. post hoc analyses, Tukey’s test for multiple comparisons was used. Pan, H.T., Menacherry, S., and Justice, J.B., Jr. (1991). Differences All results were considered statistically significant if p % 0.05. in the pharmacokinetics of cocaine in naive and cocaine-experi- enced rats. J. Neurochem. 56, 1299–1306. Acknowledgments Paxinos, G., and Watson, C. (1997). The Rat Brain in Stereotaxic Coordinates (San Diego, CA: Academic Press). The authors would like to thank M.L.A.V. Heien, P.E.M. Phillips, and Phillips, P.E., and Wightman, R.M. (2003). Critical guidelines for vali- the UNC Department of Chemistry Electronics Facility. This work dation of the selectivity of in-vivo chemical microsensors. Trends was supported by grants from the National Institute on Drug Abuse: in Analytical Chemistry 22, 509–514. DA10900 (R.M.W.), DA017318 (R.M.C. and R.M.W.), and DA015923 Phillips, P.E., Robinson, D.L., Stuber, G.D., Carelli, R.M., and Wight- (G.D.S.). man, R.M. (2003a). Real-time measurements of phasic changes in extracellular dopamine concentration in freely moving rats by fast- Received: December 22, 2004 scan cyclic voltammetry. Methods Mol. Med. 79, 443–464. Revised: March 11, 2005 Phillips, P.E., Stuber, G.D., Heien, M.L., Wightman, R.M., and Carelli, Accepted: April 10, 2005 R.M. (2003b). Subsecond dopamine release promotes cocaine Published: May 18, 2005 seeking. Nature 422, 614–618.

References Redish, A.D. (2004). Addiction as a computational process gone awry. Science 306, 1944–1947. Carelli, R.M. (2004). Nucleus accumbens cell firing and rapid dopa- Roitman, M.F., Stuber, G.D., Phillips, P.E.M., Wightman, R.M., and mine signaling during goal directed behaviors in rats. Neurophar- Carelli, R.M. (2004). Dopamine operates as a subsecond modulator macology 47, 180–189. of food seeking. J. Neurosci. 24, 1265–1271. Modulation of Dopamine by Extinction/Reinstatement 669

Schultz, W. (1998). Predictive reward signal of dopamine neurons. J. Neurophysiol. 80, 1–27. Schultz, W. (2004). Neural coding of basic reward terms of animal learning theory, game theory, microeconomics and behavioural ecology. Curr. Opin. Neurobiol. 14, 139–147. Schultz, W., Dayan, P., and Montague, P.R. (1997). A neural substrate of prediction and reward. Science 275, 1593–1599. Schwarz, A.J., Zocchi, A., Reese, T., Gozzi, A., Garzotti, M., Varnier, G., Curcuruto, O., Sartori, I., Girlanda, E., Biscaro, B., et al. (2004). Concurrent pharmacological MRI and in situ microdialysis of cocaine reveal a complex relationship between the central hemody- namic response and local dopamine concentration. Neuroimage 23, 296–304. Stuber, G.D., Roitman, M.F., Phillips, P.E.M., Carelli, R.M., and Wightman, R.M. (2005). Rapid dopamine signaling in the nucleus accumbens during contingent and noncontingent cocaine administration. Neuropsychopharmacology 30, 853–863. Venton, B.J., Michael, D.J., and Wightman, R.M. (2003). Correlation of local changes in extracellular oxygen and pH that accompany dopaminergic terminal activity in the rat caudate-putamen. J. Neu- rochem. 84, 373–381. Weiss, F., Martin-Fardon, R., Ciccocioppo, R., Kerr, T.M., Smith, D.L., and Ben-Shahar, O. (2001). Enduring resistance to extinction of cocaine-seeking behavior induced by drug-related cues. Neu- ropsychopharmacology 25, 361–372. Wise, R.A. (2004). Dopamine, learning and motivation. Nat. Rev. Neurosci. 5, 483–494. Wise, R.A., Newton, P., Leeb, K., Burnette, B., Pocock, D., and Jus- tice, J.B., Jr. (1995). Fluctuations in nucleus accumbens dopamine concentration during intravenous cocaine self-administration in rats. Psychopharmacology (Berl.) 120, 10–20. Wu, Q., Reith, M.E., Kuhar, M.J., Carroll, F.I., and Garris, P.A. (2001). Preferential increases in nucleus accumbens dopamine after sys- temic cocaine administration are caused by unique characteristics of dopamine neurotransmission. J. Neurosci. 21, 6338–6347.