The Journal of Neuroscience. December 1992. 72(12): 45954610 Neuronal Activity in Monkey Ventral Striatum Related to the Expectation of Reward Wolfram Schultz, Paul Apicella,” Eugenio Scarnati,b and Tomas Ljungbergc lnstitut de Physiologie, Universitk de Fribourg, CH-1700 Fribourg, Switzerland Projections from cortical and subcortical limbic structures The anatomical substrate underlying these functions may con- to the basal ganglia are predominantly directed to the ventral sist in the conjunction of afferents from limbic structures and striatum. The present study investigated how the expecta- mesencephalicdopamine neurons. Major limbic structures in tion of external events with behavioral significance is re- monkeys, such asthe anterior cingulate gyms, orbitofrontal cor- flected in the activity of ventral striatal neurons. A total of tex, and amygdala, project to the ventral striatum, including the 420 neurons were studied in macaque monkeys performing nucleus accumbens,in a particularly dense and interdigitating in a delayed go-no-go task. Lights of different colors in- fashion, whereas their projections to the dorsal striatum are structed the animal to do an arm-reaching movement or re- more sparseand scattered(Baleydier and Mauguiere, 1980; Par- frain from moving, respectively, when a trigger light was ent et al., 1983; Russchenet al., 1985; Selemonand Goldman- illuminated a few seconds later. Task performance was re- Rakic, 1985). The amygdala is involved in the association of inforced by liquid reward in both situations. A total of 60 external stimuli with primary and secondaryreinforcers for sus- ventral striatal neurons showed sustained increases of ac- taining performance in learning tasks (Gaffan and Harrison, tivity before the occurrence of individual task events. In 43 1987; Gaffan et al., 1988). Interactions in the ventral striatum of these neurons, activations specifically preceded the de- between afferents from the amygdala and from dopamine neu- livery of reward, independent of the movement or no-move- rons appear to be necessaryfor mediating the effectsof stimulus- ment reaction. In a series of additional tests, these activa- reward associationson behavior (Cador et al., 1989). The re- tions were time locked to the subsequent reward, inforcing effects of electrical brain stimulation and of major disappeared within a few trials when reward was omitted, drugs of abuseapparently involve the dopaminergic neurotrans- and were temporally unrelated to mouth movements. Changes mission in the ventral striatum (Fibiger and Phillips, 1986; Wise in the appetitive value of the reward liquid modified the mag- and Bozarth, 1987). For example, the reinforcing effects of her- nitude of activations, suggesting a possible relationship to oin are reduced by 6-hydroxydopamine-induced lesionsof do- the hedonic properties of the expected event. Activations paminergic fibers in nucleus accumbensof rats (Spyraki et al., also occurred when reward was delivered in a predictable 1983). manner outside of any behavioral task. These data suggest Few studies have investigated the neurophysiological sub- that neurons in the ventral striatum are activated during states strates underlying the behavioral role of the primate ventral of expectation of individual environmental events that are striatum. Recent investigations showedthat dorsal and ventral predictable to the subject through its past experience. The caudate neurons are activated when different kinds of food mor- prevalence of activations related to the expectation of re- sels are shown to the animal (Nishino et al., 1984) and that ward suggests that ventral striatal neurons have access to ventral striatal neurons respond to external stimuli associated central representations of reward and thereby participate in with reward through prior conditioning (Williams, 1989). Neu- the processing of information underlying the motivational rons in both the ventral and dorsal parts of the striatum respond control of goal-directed behavior. to the delivery of primary liquid reward at the animal’s mouth, theseresponses being unrelated to mouth movements (Apicella The ventral striatum of the mammalian brain may be involved et al., 1991 b). Thesereward responsesoccur twice as frequently in evaluating the hedonic properties of external stimuli and in in ventral as compared to dorsal parts of striatum and suggest sustaining behavioral reactions toward goalsof primary interest. that information about the reception of reward reachesprefer- entially the ventral striatum. Single-neuronstudies in behaving monkeys reported that dor- Received Sept. 30, 1991; revised May 8, 1992; accepted June 11, 1992. sal striatal neuronsshow sustainedchanges in activity selectively We thank F. Tinguely, A. Schwarz, and J. Corpataux for technical assistance. during the expectation of external signalsof behavioral signif- The study was supported by the Swiss NSF (Grants 3.473-0.86, 31-28591.90) icance, during the preparation of limb or eye movements, and the Fyssen Foundation, the Formation pour la Recherche M&licale, the United Parkinson Foundation, and the Italian CNR. during the expectation of reward (Alexander, 1987; Schultz and Correspondence should be addressed to Dr. W. Schultz at the above address. Romo, 1988; Hikosaka et al., 1989; Alexander and Crutcher, a Present address: Laboratoire de Neurobiologie Cellulaire et Fonctionnelle, CNRS, F-l 3274 Marseille, France. 1990; Apicella et al., 1992). Thus, striatal neuronshave access b Present address: Department ofBiomedicalTechnology, Laboratory ofHuman to central representationsof environmental events that are pre- Physiology, School of Medicine, University of L’Aquila, I-67 100 L’Aquila, Italy. dictable to the subject through its past experience. These activ- = Present address: Department of Pharmacology, Karolinska Institute, Stock- holm, Sweden. ities may constitute important components of neuronal pro- Copyright 0 1992 Society for Neuroscience 0270-6474/92/124595-16$05.00/O cessesunderlying the organization of behavioral output by the 4596 Schultz et al. l Ventral Striatum and Expectation of Reward basal ganglia. The objective of the present study was to inves- defined period began with reward delivery and ended with the instruc- tigate whether expectation- and preparation-related activity could tion light of the subsequent trial. In the simultaneous go-no-go task, instruction and trigger lights were also be found in the ventral striatum, to which external signals illuminated at the same time. The animal received the information and behavioral events such activity could be related, and how about the go or no-go situation (green or red instruction light, respec- the activity could contribute to the proposed motivational func- tively) together with the trigger signal that elicited the appropriate be- tions of the ventral striatum. Monkeys performed in a behav- havioral reaction. This eliminated the preparatory period between in- ioral task composed of separate time periods during which ex- struction and trigger while maintaining the period of expectation of reward after the trigger. Similar to the delayed go-no-go task, reward ternal signals were expected, behavioral reactions were prepared, was delivered after a delay following lever touch in go trials and trigger and reward as the common outcome of different behavioral onset in no-go trials, respectively. This task was only used on neurons reactions was expected. This task structure allowed discrimi- that were activated during the delayed go-no-go task. nation of distinct expectation-related activities among the dif- Additional tests were occasionally employed. In a variation of the delayed go-no-go task, reward was delivered simultaneously with il- ferent task components. lumination of the trigger stimulus. This eliminated the period of reward expectation after the trigger stimulus. In another test, the tube con- ducting reward liquid to the spout at the animal’s mouth was shut off. Materials and Methods This eliminated the delivery of reward while maintaining the noise of The study was performed on three male Macaca fascicularis monkeys the solenoid valve. This modification was only noticeable to the animal (3.5-3.8 kg weight). Animals performed under computer control in sev- by the absence of liquid arrival. In a further test, we determined whether eral variationsof a go-no-gotask for obtainingliquid reward.Activity neuronal activations would continue when the trigger stimulus was kept of singleneurons was recorded during task performance with moveable present before reward delivery. The trigger light remained illuminated microelectrodes while monitoring arm and mouth muscle activity through until lever touch (go trials) or reward delivery (no-go), instead of being chronically implanted electrodes. Upon termination of recording, elec- turned off 400 msec after its illumination. This test served to assess trode positions were histologically reconstructed on brain sections. whether neuronal activity was driven by sensory input or was possibly Methods were similar to those employed previously (Schultz and Romo, related to visual working memory. In another test, the animal received 1990; Apicella et al., 199 1 b). a drop of liquid reward once every 8 set without being engaged in any Behavioral procedures. The behavioral apparatus was positioned in specific task, the resting key being removed and lights unused. the right half of the frontal wall of a completely enclosed primate chair. Data acquisition and analysis.