Neuropsychopharmacology (2018) 43, 680–689 © 2018 American College of Neuropsychopharmacology. All rights reserved 0893-133X/18 www.neuropsychopharmacology.org Review Drive and Reinforcement Circuitry in the Brain: Origins, Neurotransmitters, and Projection Fields Roy A Wise*,1 and Ross A McDevitt1 1 Intramural Research Program, National Institute on Drug Abuse, National Institute on Health, Baltimore, MD, USA Brain stimulation has identified two central subsets of stimulation sites with motivational relevance. First, there is a large and disperse set of sites where stimulation is reinforcing, increasing the frequency of the responses it follows, and second, a much more restricted set of sites — — where along with reinforcement stimulation also has drive-like effects, instigating feeding, copulation, predation, and other motivated acts in otherwise sated or peaceful animals. From this work a dispersed but synaptically interconnected network of reinforcement circuitry is emerging: it includes afferents to the ventral tegmental area and substantia nigra; the dopamine systems themselves; glutamatergic afferents to the striatum; and one of two dopamine-receptor-expressing efferent pathways of the striatum. Stimulation of a limited subset of these sites, including descending inhibitory medial forebrain bundle fibers, induces both feeding and reinforcement, and suggests the possibility of a subset of fibers where stimulation has both drive-like and reinforcing effects. This review stresses the common findings of sites and connectivity between electrical and optogenetic studies of core drive and reinforcement sites. By doing so, it suggests the biological importance of optogenetic follow-up of less-publicized electrical stimulation findings. Such studies promise not only information about origins, neurotransmitters, and connectivity of related networks, by covering more sensory and at least one putative motor component it also promotes a much deeper understanding of the breadth of motivational function. Neuropsychopharmacology (2018) 43, 680–689; doi:10.1038/npp.2017.228; published online 1 November 2017 INTRODUCTION the same or by independent substrates (Coons and Cruce, 1968; Deutsch et al, 1962; Gratton and Wise, 1988a, b,; Early studies of the effects of electrical brain stimulation in Huston, 1971; Mendelson, 1970). These studies have not yet freely moving animals identified two motivational effects. identified differences between reinforcement and drive First, diencephalic stimulation at many sites can be fibers. However, because electrical stimulation preferentially reinforcing, controlling the acts that it reliably follows activates fibers of passage (Ranck, 1975), electrical stimula- (Olds, 1956; Olds and Milner, 1954). Second, stimulation tion studies had limited success in identifying the directly can have drive-like effects, energizing a variety of species- activated fibers or connections of either effect. typical, biologically primitive acts, such as eating, drinking, In the present paper we integrate these electrical stimula- copulation, or attack, in otherwise sated or quiescent animals tion findings with recent optogenetic studies that can now (Andersson and Wyrwicka, 1957; Delgado and Anand, 1953; identify cells of origin, neurotransmitters, and synaptic Hess, 1957; Roberts and Carey, 1965; Wasman and Flynn, targets of the stimulated fibers. These studies, taken together, 1962). Stimulation at the same sites within the medial begin to sketch a core structure for an integrated mesence- forebrain bundle (MFB) often has both drive-like and phalic, diencephalic, and telencephalic circuitry subserving reinforcing effects (Caggiula and Hoebel, 1966; Margules motivational function. The optogenetic study of additional — and Olds, 1962; Mogenson and Stevenson, 1966; Roberts and structures currently identified only by electrical stimulation — Carey, 1965), whereas stimulation at a larger number of is hoped to broaden the list of anatomical substrates and to extraneous sites is reinforcing without inducing goal-directed deepen our understanding of motivational function. behaviors. That rats should find stimulation reinforcing when it also makes them hungry or thirsty seemed paradoxical (Wise, 2013) and led to a range of experiments TERMINOLOGY designed to explore whether the two effects were mediated by Here we separate two motivational effects of stimulation, one associated with the animal’s state of mind before it earns a *Correspondence: Dr RA Wise, Intramural Research Program, National Institute on Drug Abuse, National Institute on Health, 251 Bayview reward and one associated with the state of mind after the Boulevard, Baltimore, MD 21224, USA, Tel: +443-562-4660, reward has been earned and is being experienced. We will E-mail: [email protected] use drive-like to refer to the former and reinforcing to refer Received 21 May 2017; revised 8 August 2017; accepted 13 September to the latter. We have avoided, elsewhere in this paper, the 2017; accepted article preview online 6 October 2017 use of the more familiar term ‘reward’ because it confuses the Drive and reinforcement circuitry RA Wise and RA McDevitt 681 two meanings. The noun (‘a’ reward) refers to the object or fibers preferentially (Gallistel et al, 1981); only very few and effect to be sought, whereas the verb (‘to’ reward) refers to very local, dopaminergic fibers are activated at traditional the action of obtaining and experiencing its effects. stimulation parameters (Yeomans, 1989; Yeomans et al, Reinforcing effects are defined as the consequences of 1988). Rather, the major contributions to the directly required behavior. This is measured by allowing an animal to activated reinforcing effects of MFB electrical stimulation control the stimulation. The gold standard is operant self- are now believed to be low-threshold, small, and descending stimulation, where a specific response (ie, a lever-press or a myelinated fibers (Gallistel et al, 1981; Gratton and Wise, nose-poke) results in a reinforcing event. In this case there is 1988a) one or more synapses upstream from the midbrain a fixed amount of reinforcement for each action. In the dopamine neurons (Wise, 1980; Yeomans, 1982). traditional situation, there are fixed quanta of reinforcement, With optogenetic methods, however, dopaminergic cell and reinforcements can be counted. In the case of unsensed bodies of VTA or SNc can be selectively activated, and incentives such as earned brain stimulation, where there is activation of these neurons is reinforcing (Ilango et al, 2014a; no external stimulus object and continuous reinforcement Ilango et al, 2014b; Kim et al, 2012; McDevitt et al, 2014; can be offered (Wise, 2002), an alternative paradigm allows Pascoli et al, 2015; Rossi et al, 2013; Stauffer et al, 2016; Tsai an animal to control both onset and offset of stimulation. et al, 2009; Wang et al, 2017; Witten et al, 2011). Two This paradigm was originally called ‘shuttlebox stimulation’ subpopulations of dopaminergic ventral midbrain neurons and is referred to in the optogenetics literature as ‘real-time have been targeted—one that expresses the dopamine place preference.’ Care must be taken in this paradigm to synthesis enzyme tyrosine hydroxylase (TH) but includes differentiate between stimulus-onset effects and stimulus- some midline GABAergic neurons that do not release offset effects; rats, having initiated the stimulation, will often dopamine (Lammel et al, 2015; Stamatakis et al, 2013), and work to discontinue it as if it had become aversive (Bower another that expresses the dopamine transporter but and Miller, 1958; Mendelson and Freed, 1973; Roberts, excludes some dopaminergic neurons near the midline 1958a; Steiner et al, 1969). Furthermore, because continuous (Darvas et al, 2014; Li et al, 2013). Stimulation of each is stimulation may impair the ability of the animal to remain in reinforcing (for DAT-targeted animals see Kim et al (2012) or return to the non-stimulation zone, locomotor artifacts and Wang et al (2017)). The reinforcing effects of can greatly influence measures in this situation. One optogenetic activation of TH-expressing midbrain neurons workaround is using optogenetic stimulation where an are attenuated by ventral striatal microinjections of D1-type animal is given fixed periods of stimulation or fixed periods or D2-type dopamine antagonists (Steinberg et al, 2014). between stimulations. However, genetic disruption of glutamate co-release from DAT-expressing neurons does not impact their reinforcing abilities (Wang et al, 2017). Consistent with this finding, rats SITES AND SUBSTRATES will lever-press for direct microinjections of D1 and D2 VentralMidbraintoStriatum agonists into this brain region (Ikemoto et al, 1997). The dorsal striatum is richly innervated by dopaminergic fibers Mesolimbic and nigrostriatal dopamine systems. Phar- from the SNc, and stimulation of the SNc cell bodies macological challenge in early electrical stimulation studies produces reward measures equivalent to those produced by suggested that the midbrain dopamine systems were a ventral tegmental stimulation (Ilango et al, 2014b). The other common substrate of MFB reinforcement and motivation. projections from these systems remain to be studied, but it The reinforcing (Fouriezos and Wise, 1976; Lippa et al, 1973) has been confirmed that stimulation of dopaminergic fibers and feeding-inducing (Phillips and Nikaido, 1975) effects of projecting to the shell of the ventral striatum is reinforcing