Control by neuromodulation: a tutorial. Rodolphe Sepulchre, Timothy O’Leary, Guillaume Drion, and Alessio Franci Abstract— This tutorial provides an introduction to the topic results from the interaction of many nonlinear elements, of neuromodulation as an important control paradigm for computational models are needed to understand them. How natural and artificial neuronal networks. We review how neu- realistic do these models need to be, and what data are romodulation modulates excitability, and how neuromodulation interacts with homeostasis. We stress how modulating nodal needed to constrain these models? How will modulation alter excitability provides a robust and versatile control principle to these processes? ” and “How can highly modulated circuits dynamically reconfigure the connectivity of rhythmic circuits be stable in the face of parameter changes brought about by and to shape the spatio-temporal synchrony of large popula- modulation?” The central questions of neuromodulation are tions. all control theoretic in nature: How do neuronal circuits cope Index Terms— Neural circuits, neuromodulation, excitability, with uncertainty, heterogeneity, and variability ? How do homeostasis, robustness, adaptation. neuronal circuits dynamically reconfigure to process sensory signals and execute motor tasks ? Medical neuromodulation I. INTRODUCTION is largely regarded as the future of neuroengineering and investigated as a potential treatment for a rapidly expanding Neuromodulation is an important control principle of range of pathologies. neuroscience. In a physiological context, neuromodulation Traditionally, the emphasis in modeling the nervous sys- designates the regulation of the electrical activity of neuronal tem has been on neurotransmission rather than neuromodu- circuits by ignaling molecules called neuromodulators. They lation. The focus of neurotransmission is on the excitatory include ACh, dopamine, norepinephrine, GABA, glycine, or inhibitory synaptic transmission of electrical signals in glutamate, serotonin, histamine, octopamine, and neuropep- neuronal networks. In contrast, neuromodulation is about the tides. In a medical context, neuromodulation designates the permanent modulation (another name for control) of cellular increasing body of technologies aiming at interacting with or synaptic properties of neurons by neuromodulators. Neu- the electrical activity of neuronal circuits. Minimally invasive romodulators are released by specific sets of neurons. Those technologies have been rapidly developing in the recent neuromodulatory neurons are localized in specific regions years. They include electromagnetic stimulation (e.g. deep- of the brains but they project broadly. Neuromodulation brain stimulation as a treatment for Parkinson’s disease), can radically alter the processing properties of neurons and pharmacological drug delivery, and optogenetics. The mecha- synapses over short time scales and large spatial scales. nisms of medical neuromodulation are still poorly understood Neuromodulators are major players in the control of neuronal and largely empirical, but, ultimately, they can be regarded circuits and suggest a diversity of possible behaviors for a as an external interaction with the internal physiological given connectome. mechanisms of neuromodulation. The focus of this tutorial is to introduce some of the Neuromodulation is a natural entry point to neuroscience basic mechanisms of neuromodulation in a control theoretic for control theorists and control engineers. A theory of language. We minimize the reference to biological details and physiological neuromodulation is ultimately a control theory provide cartoon illustrations of general principles that could of neuronal circuits. One of the earliest books on neuromodu- also be of potential importance in artificial neuromorphic lation is entitled Neuromodulation: The Biochemical Control circuits or artificial neural networks. of Neuronal Excitability [35]. It could serve as a title of The aim of this tutorial is to inspire control theorists this tutorial. The recent review [48] by Eve Marder should and control engineers. Specific examples will be selected to speak to control theorists as a remarkable invitation to study illustrate that neuromodulation raises novel control questions neuronal circuits. The following two quotes of the paper are that directly connect the classical language of circuit theory illustrative: “Because the output of all biological circuits to some of the most fascinating challenges of neuroscience and neuroengineering. TO and RS are with the Control Group at the Department of Engineering, University of Cambridge, Trumpington Street, Cambridge CB2 1PZ, United Kingdom. ftimothy.oleary, II. A BRIEF HISTORY OF NEUROMODULATION [email protected]. GD is with the Institut Irrespective of background, many engineers will have an Montefiore, Universite de Liege, 4000 Liege Sart-Tilman, Belgium. AF is with National Autonomous University of Mexico, Science informal picture of a neuron as an excitable electrical device, Faculty, Department of Mathematics, Coyoacan,´ D.F. Mexico.´ coupled in a circuit to sensory organs, muscles and other [email protected] neurons. This view emerged early in the history of physi- The research leading to these results has received funding from the European Research Council under the Advanced ERC Grant Agreement ology, where the pioneering work of Galvani (1737-1798) Switchlet n.670645. and Ramon y Cajal (1852-1934) respectively revealed the electrical basis of neural activity and unpicked the intricate at the synapse, many neuromodulators diffuse long distances ‘wiring’ of the brain as a network of interconnected cells. and are capable of targeting populations of neurons. More- Later work in the early 20th century revealed that neurons over, the mode of action of many neuromodulators is funda- signal in discrete pulses, or action potentials [1], [43]. It was mentally different to classical neurotransmitters. Instead of not obvious at the time that the interconnections - synapses merely exciting or inhibiting neurons, neuromodulators – as - between circuit elements were chemical in nature: neurons their name suggests – modulate the biophysical properties of influence their neighbours by secreting tiny puffs of amino target neurons. acids, peptides and other small molecules [6], [7], [16], [70]. In later sections we will clarify the kinds of biophysical Neurotransmitters act as a chemical key, binding to recep- properties that are affected by neuromodulators. Informally, tor proteins that are structurally maintained in close apposi- these can include synaptic conductances or the intrinsic con- tion to their site of release. Receptor proteins, in turn, are ductances that control the excitable behaviour of a neuron. physical pores that selectively allow ions to pass when open. Neuromodulators can thus directly reconfigure the wiring The action of synaptic transmission can be conveniently diagram of a circuit by changing interconnection strengths. and accurately modelled as a variable conductance in series They can ‘switch’ the nodes in the circuit from generating with a potential difference that represents the electrochemical discrete pulses to oscillations. They can also alter the de- driving force of different ionic species. This driving force can gree of excitability of neurons, making otherwise quiescent be negative or positive with respect to the resting membrane circuits highly active. voltage of a neuron. Synaptic transmission may thus excite Contemporary neuroscience research has been transformed or inhibit a target. Similarly, the maximal conductance of a by experimental methods that allow neural circuits in more synapse may vary. complex organisms, including humans, to be mapped in Since the 1930s it was known that there are multiple increasing detail. Modern connectomics allows exhaustive types of neurotransmitters [6] and distinct types of neurons mapping of the connectivity of entire nervous systems. that secrete specific neurotransmitters. Accordingly, there are Similarly, recording and stimulation technology allows mea- multiple types of neurotransmitter receptors, each with its surement and control of neurons at a level of precision own biophysical characteristics. To a loose approximation, that permits intentions, sensations and actions to be reliably one can summarize this picture of neural circuits as a decoded in awake, behaving animals. collection of excitable elements that are interconnected via This wave of technological progress understandably fuels positive or negative (excitatory or inhibitory) synapses. promises that neural circuits in ‘higher’ organisms can be un- This picture of a neural circuit as a network of excitable derstood mechanistically. This excitement mirrors the hopes elements with signed interconnections is very appealing. It that were ignited over 40 years ago in small invertebrate suggests that circuit principles in electrical engineering can circuits. Paradoxically, the same lessons about neuromod- be directly applied to understand the function of any neural ulation are being rediscovered [38], [44]. The confounds, circuit, provided the connectivity (or wiring diagram) and surprises and shifts in thinking that followed the discovery of the signs and relative strengths of synapses are known [66], neuromodulation are playing out for a second time on a much [69]. bigger scale. Progress in neural engineering as well as basic The 70s saw a surge in interest in mapping