Neural Networks 15 (2002) 719–730 www.elsevier.com/locate/neunet 2002 Special issue Acetylcholine in cortical inference Angela J. Yu*, Peter Dayan Gatsby Computational Neuroscience Unit, University College London, 17 Queen Square, London WC1N 3AR, UK Received 5 October 2001; accepted 2 April 2002 Abstract Acetylcholine (ACh) plays an important role in a wide variety of cognitive tasks, such as perception, selective attention, associative learning, and memory. Extensive experimental and theoretical work in tasks involving learning and memory has suggested that ACh reports on unfamiliarity and controls plasticity and effective network connectivity. Based on these computational and implementational insights, we develop a theory of cholinergic modulation in perceptual inference. We propose that ACh levels reflect the uncertainty associated with top- down information, and have the effect of modulating the interaction between top-down and bottom-up processing in determining the appropriate neural representations for inputs. We illustrate our proposal by means of an hierarchical hidden Markov model, showing that cholinergic modulation of contextual information leads to appropriate perceptual inference. q 2002 Elsevier Science Ltd. All rights reserved. Keywords: Acetylcholine; Perception; Neuromodulation; Representational inference; Hidden Markov model; Attention 1. Introduction medial septum (MS), diagonal band of Broca (DBB), and nucleus basalis (NBM). Physiological studies on ACh Neuromodulators such as acetylcholine (ACh), sero- indicate that its neuromodulatory effects at the cellular tonine, dopamine, norepinephrine, and histamine play two level are diverse, causing synaptic facilitation and suppres- characteristic roles. One, most studied in vertebrate systems, sion as well as direct hyperpolarization and depolarization, concerns the control of plasticity. The other, most studied in all within the same cortical area (Kimura, Fukuda, & invertebrate systems, concerns the control of network Tsumoto, 1999). Behavioral experiments indicate that responses. For instance, a single, recurrently connected ACh is involved in a wide variety of cognitive functions assembly of neurons can exhibit multiple dynamical modes such as perception, selective attention, associative learning, (Pflu¨ger, 1999), as neuromodulators alter the excitabilities and memory (Everitt & Robbins, 1997; Hasselmo, 1995; of individual neurons and the amplitudes of synaptic Holland, 1997). potentials (Marder, 1998). These two roles have also been Hasselmo and colleagues proposed that cholinergic (and brought together, notably in the theoretical and experi- perhaps other) neuromodulation controls read-in to, and mental studies of Hasselmo and his colleagues, into the read-out from, recurrently-connected, attractor-like mem- neuromodulatory control of plasticity in recurrently con- ories, such as that in area CA3 of the hippocampus. Such nected neural networks (Hasselmo, 1995; Hasselmo & attractor networks (Amit, 1989) fail if the recurrent Bower, 1993). This work sits with that on dopamine (e.g. connections are operational during storage, since new Schultz, Dayan and Montague, 1997) in proposing compu- memories lose their specific identity by being forced to tationally specific roles for neuromodulation. map onto existing memories retrieved through the recurrent Hasselmo and his colleagues (Hasselmo, 1995; Hasselmo dynamics. Hasselmo and colleagues suggested, and col- & Bower, 1993) focused on cholinergic neuromodulatory lected direct experimental evidence, that cholinergic influences over learning and memory in the hippocampus neuromodulation during storage could selectively suppress and cortex. ACh is delivered to the cortex and hippocampus but plasticize the recurrent connections (and perhaps the from a small number of nuclei in the basal forebrain (BF): perforant path connections) onto CA3 cells and selectively boost the feedforward mossy fiber inputs from the dentate * þ þ Corresponding author. Tel.: 44-20-7679-1195; fax: 44-20-7679- gyrus. During recall, the recurrent connections should play a 1173. E-mail addresses: [email protected] (A.J. Yu), dayan@gatsby. fuller part, being comparatively boosted through a lower ucl.ac.uk (P. Dayan). level of ACh, allowing associative retrieval. The degree of 0893-6080/02/$ - see front matter q 2002 Elsevier Science Ltd. All rights reserved. PII: S0893-6080(02)00058-8 720 A.J. Yu, P. Dayan / Neural Networks 15 (2002) 719–730 ACh release would reflect the unfamiliarity of the input, and using it correctly in concert with bottom-up information thereby act as a gate to learning. This mechanism has been from the sensory input (Grenander, 1995; Helmholtz, 1896; widely adopted, for instance in our own work (Ka´li and Neisser, 1967). Dayan, 2000) to help understand how spatial place cells in For simplicity, we only consider one of the most basic CA3 might result from a learned surface attractor network. forms of top-down contextual information, namely that Hasselmo and his colleagues have also demonstrated that coming from the recent past. That is, we consider a series of ACh has similar physiological and functional effects in the sensory inputs whose internal representations are individu- piriform cortex (Linster & Hasselmo, 2001), which is ally ambiguous. Disambiguation comes via top-down important for olfactory memory. information based on a slowly changing overall state of Lesions studies in classical conditioning tasks provide the environment. Here, only temporal context is relevant; additional insight into ACh’s role in the cortex. Animals are there is no spatial context. The resulting model (see also known to learn faster about stimuli whose consequences Becker, 1999) is a form of Hidden Markov Model (HMM). remain uncertain (Pearce & Hall, 1980). Through an The HMM captures the way that sensory inputs are extensive series of selective lesion experiments in rats, generated or synthesized. We consider the inferential task Holland and his colleagues have demonstrated that the of recognition or analysis in which the representation for cholinergic projection from the nucleus basalis magno- each input is determined. We compare an approximate cellularis (nucleus basalis of Meynert in primates) to the recognition model based on cholinergic neuromodulation, parietal cortex is essential for this sort of faster learning with the exact recognition model (Rabiner, 1989), in a case (Holland, 1997; Holland & Gallagher, 1999). These data chosen so that the exact model is computationally tractable. have been interpreted, using the theoretical viewpoint of Our HMM (Fig. 1(A) and (B)) consists of three pieces. statistical learning models, as implying that the ACh signal One, zt; is the overall state of the environment at time t, reports the unfamiliarity of the stimuli, or the uncertainty in which we also call the context. Changes to zt are its predictions (Dayan, Kakade, & Montague, 2000). stochastically controlled by a transition matrix T zt21zt ; In this paper, we present a theory of cortical cholinergic whose entries ensure that the context changes rarely. The function in perceptual inference based on combining the second piece is yt; which is determined stochastically on physiological evidence that ACh can differentially modulate each time-step, in a way that depends on the current state of synaptic transmission to control states of cortical dynamics, the environment. The third piece, the observed input xt, with theoretical ideas about the information carried by the depends stochastically on yt: The inferential task is to ACh signal. Crudely speaking, perception involves inferring represent inputs x in terms of the y values that were the most appropriate representation for sensory inputs. This responsible for them. However, the relationship between yt inference is influenced by both top-down inputs, providing and xt is such that this is ambiguous, so top-down contextual information, and bottom-up inputs from sensory information from the likely states of zt; i.e. the likely processing. We propose that ACh reports on the uncertainty context, is important to find the correct representation for xt. associated with top-down information, and has the effect of Fig. 1(A) shows the probabilistic contingencies among the modulating the relative strengths of these two input sources. variables. Fig. 1(B) shows the same contingencies in a Many cognitive functions affected by ACh levels can be different way, and specifies the particular setting of recast in the conceptual framework of representational parameters used to generate the examples found in the inference. remainder of the paper. In Section 2, we present a simple hierarchical hidden More formally, the context is a discrete, hidden, random Markov model that casts sensory perception in the variable zt, whose stochastic temporal dynamics are theoretical framework of representational inference. As we described by a Markov chain with transition matrix T zt21zt as demonstrate in Section 3, approximate inference in such a 8 > model could be mediated by cortical cholinergic inner- < g if zt ¼ zt21; vation. A summary of relevant experimental data and P½z lz ; T ¼ 1 2 g ð1Þ t t21 zt21zt :> proposals for new experiments is presented in Section 4. otherwise; nz 2 1 where nz is the number of all possible states of z, and g is the 2. Hidden Markov models and perceptual inference probability of persisting in one text. When g is close to 1, as is the case in the example of Fig. 1(B), the context tends to