Neuromodulatory Systems and Their Interactions: a Review of Models, Theories, and Experiments
Total Page:16
File Type:pdf, Size:1020Kb
UC Irvine UC Irvine Previously Published Works Title Neuromodulatory Systems and Their Interactions: A Review of Models, Theories, and Experiments. Permalink https://escholarship.org/uc/item/7th2d1rf Authors Avery, Michael C Krichmar, Jeffrey L Publication Date 2017 DOI 10.3389/fncir.2017.00108 License https://creativecommons.org/licenses/by/4.0/ 4.0 Peer reviewed eScholarship.org Powered by the California Digital Library University of California REVIEW published: 22 December 2017 doi: 10.3389/fncir.2017.00108 Neuromodulatory Systems and Their Interactions: A Review of Models, Theories, and Experiments Michael C. Avery 1* and Jeffrey L. Krichmar 2, 3 1 SNL-R, Systems Neurobiology Laboratory, Salk Institute for Biological Studies, La Jolla, CA, United States, 2 Department of Cognitive Sciences, University of California, Irvine, Irvine, CA, United States, 3 Department of Computer Science, University of California, Irvine, Irvine, CA, United States Neuromodulatory systems, including the noradrenergic, serotonergic, dopaminergic, and cholinergic systems, track environmental signals, such as risks, rewards, novelty, effort, and social cooperation. These systems provide a foundation for cognitive function in higher organisms; attention, emotion, goal-directed behavior, and decision-making derive from the interaction between the neuromodulatory systems and brain areas, such as the amygdala, frontal cortex, hippocampus, and sensory cortices. Given their strong influence on behavior and cognition, these systems also play a key role in disease states and are the primary target of many current treatment strategies. The fact that these systems interact with each other either directly or indirectly, however, makes it difficult to understand how a failure in one or more systems can lead to a particular symptom or pathology. In this review, we explore experimental evidence, as well as focus on computational and theoretical models of neuromodulation. Better understanding of Edited by: neuromodulatory systems may lead to the development of novel treatment strategies for Michael E. Hasselmo, Boston University, United States a number of brain disorders. Reviewed by: Keywords: neuromodulation, computational neuroscience, computational modeling, brain disorders, Daniel W. Wesson, neuromodulatory systems University of Florida, United States Wolfgang Stein, Illinois State University, United States INTRODUCTION *Correspondence: Michael C. Avery The mammalian neuromodulatory system consists of small pools of neurons (on the order of [email protected] thousands in the rodent and tens of thousands in the human) located in the brainstem, pontine nucleus, and basal forebrain, which can have a powerful effect on cognitive behavior. Ascending Received: 02 August 2017 neuromodulatory systems include noradrenergic, serotonergic, dopaminergic, and cholinergic Accepted: 14 December 2017 projections from the brainstem and basal forebrain regions to broad areas of the central nervous Published: 22 December 2017 system (Briand et al., 2007). Neuromodulators signal risks, rewards, novelty, effort, and social Citation: cooperation. These systems provide a basis for many higher cognitive functions; attention, Avery MC and Krichmar JL (2017) decision-making, emotion, and goal-directed behavior result from the interaction between Neuromodulatory Systems and Their Interactions: A Review of Models, the neuromodulatory systems and brain areas, such as the anterior cingulate, frontal cortex, Theories, and Experiments. hippocampus, sensory cortex, and striatum (Figure 1). In this review, we explore experimental Front. Neural Circuits 11:108. evidence, with a strong focus on computational and theoretical models of neuromodulation. We doi: 10.3389/fncir.2017.00108 discuss how these models might increase our understanding of brain disorders. Frontiers in Neural Circuits | www.frontiersin.org 1 December 2017 | Volume 11 | Article 108 Avery and Krichmar Modeling Neuromodulatory Systems and Their Interactions FIGURE 1 | Neuromodulatory system interactions and their role in disease. This figure illustrates how serotonergic (blue), cholinergic (red), noradrenergic (green), and dopaminergic (purple) systems are highly connected to one another as well as cortical and subcortical structures. Their malfunction has been associated with a host of neurological and psychiatric conditions as indicated above each brain region. Gray arrows denote recurrent connections. DOPAMINERGIC SYSTEM to the striatum, how they compete to drive striatum responses, and how phasic and tonic dopamine levels might regulate these The dopaminergic neuromodulatory system has been extensively brain regions. Understanding these upstream effects is a critical studied and is involved in nearly every aspect of brain function component as we develop a circuit-level understanding of brain from cognition to behavior (Schultz, 1997; Schultz et al., 1997, disorders that are thought to result from abnormal dopaminergic 2000; Berridge, 2004, 2012; Hyman et al., 2006; Durstewitz activity. and Seamans, 2008). Dopamine originates in either the ventral Dopamine neurons, in turn, send projections to the tegmental area (VTA) or substantia nigra pars compacta (SNc). striatum, thalamus, amygdala, hippocampus, and prefrontal A substantial amount of research has gone into understanding cortex, demonstrating the “feedback” nature of this circuit. the circuits that regulate dopamine neuron firing as well as Dopaminergic neurons originating in the SNc project to the the downstream effects of dopamine release. In particular, we dorsal striatum. Abnormalities in this pathway can lead to motor know that the VTA and SNc are strongly influenced by the disorders including Parkinson’s disease. Two distinct areas in the striatum and subcortical structures such as the lateral habenula VTA project to either the ventral striatum (mesolimbic) or to the and pedunculopontine tegmental nucleus. It has been shown that prefrontal cortex (mesocortical). The effect that dopamine has on the phasic increase and dip in dopamine response are due to its downstream target depends on the post-synaptic receptor and the activation of the pedunculopontine tegmental nucleus and the firing mode of the DA neuron. Phasic release of dopamine lateral habenula, respectively (Matsumoto and Hikosaka, 2007; in the striatum, for example, preferentially activates D1 receptors Hong and Hikosaka, 2014). Phasic increases also may be due to on striatal Medium Spiny Neurons (MSNs) and increases their collicular or other sensory or non-sensory inputs to VTA/SNc activity (direct pathway). Increases in tonic dopamine, on the when a salient event is identified (Redgrave and Gurney, 2006). other hand, are thought to activate D2 receptors (D2R) in the Direct and indirect pathways in the striatum disinhibit and striatum, which inhibit MSNs in the striatum (indirect pathway). inhibit dopamine neuron firing, respectively, and are themselves It has recently been shown, however, that phasic DA can also lead modulated by cortical and limbic inputs. Prefrontal and to increases in inhibitory post-synaptic currents in D2R-MSN hippocampal inputs to the striatum disinhibit the VTA leading to neurons (Marcott et al., 2014), suggesting the role of tonic and an increase in phasic and tonic activity, respectively (Takahashi phasic dopamine may be more complex than originally thought. et al., 2011; Murty et al., 2017). It has been hypothesized Increases in both phasic and tonic activity would, therefore, that an abnormal increase in glutamatergic input to striatum lead to an increase in the direct pathway and a decrease in the leads to excess dopamine in the striatum and may account for indirect pathway, which would ultimately cause a strong release symptoms of schizophrenia (de la Fuente-Sandoval et al., 2011). of inhibition on the thalamus. Computational models of the basal ganglia have also shed light The effects of tonic and phasic dopamine in the prefrontal on the role dopamine plays in Parkinson’s disease (Moustafa and cortex appears to be opposite of the striatum. D1 receptors Gluck, 2011; Moustafa et al., 2013; Balasubramani et al., 2015). in the prefrontal cortex are preferentially activated by tonic Still, many questions remain regarding cortical and limbic inputs dopamine and have an inverted-U dose-dependent response Frontiers in Neural Circuits | www.frontiersin.org 2 December 2017 | Volume 11 | Article 108 Avery and Krichmar Modeling Neuromodulatory Systems and Their Interactions on superficial neurons (discussed below), whereas D2 receptors REINFORCEMENT LEARNING MODELS are activated by phasic dopamine and increase the activity of subcortically-projecting neurons in deep layers. This suggests Reinforcement learning is a machine learning method that that D2 receptors play a preferential role in behavior and concerns itself with finding the appropriate actions that reward processing, whereas D1-expressing neurons are involved maximize future reward. Formally, the theory aims to find an in working memory and attentional modulation of visual optimal function, or policy, (P) for mapping states (S) into cortices (Noudoost and Moore, 2011; Gee et al., 2012; Puig actions (A) that maximize the sum of future reward. Temporal and Miller, 2014). Interestingly, the temporal dynamics of the difference learning methods, such as the actor-critic model, solve phasic responses of dopaminergic cells resemble those found in this problem by computing a reward prediction error signal a machine learning method known as