Opening the gate to working memory

David Badrea,b,1 aDepartment of Cognitive, Linguistic, and Psychological Sciences and bBrown Institute for Brain Science, Brown University, Providence, RI 02912

n a changing and complex world, we of the neurotransmitter dopamine (DA) these “context-dependent” trials, the cue require the ability to sustain pursuit from the ventral tegmental area (VTA) was encoded as a context. I of a goal, while also updating our and substantia nigra (SN) to the PFC. Following this logic, comparison of goals as the need arises. For exam- Thus, DA acts as a gating signal, timing context-dependent vs. context-independent ple, we are capable of rapidly shifting when the PFC accepts new input. In this cues provided a means of testing working- tasks, such as stopping work on a paper to model, the SN/VTA learns when to gate memory updating. First, in fMRI, greater have an involved discussion with a col- from the positive or negative outcomes bilateral DLPFC activation accompanied league. However, we are also able to avoid that follow gating. context-dependent vs. context-independent distracting cues, such as ignoring the un- Several aspects of the DA-updating hy- cues. Next, single-pulse transcranial mag- read e-mail messages incrementing in our pothesis are supported empirically. For netic stimulation (TMS) was applied to inbox while we write. The mechanisms each participant’sDLPFCinalocusthat by which the brain achieves an adaptive ’ showed that participant’s maximal updating balance between flexibility and stability D Ardenne et al. provide effect. When the TMS pulse was applied to remain the basis of much current inves- an important step right DLPFC 150 ms after onset of the cue, tigation in cognitive neuroscience. In subsequent performance was disrupted in PNAS, D’Ardenne et al. (1) provide evi- forward in our context-dependent trials. This timing aligns dence for two key pieces to this puzzle. with a previous event-related potential They demonstrate that the dorsolateral understanding of study of context updating (18). Disruption prefrontal cortex (DLPFC) is necessary was not observed following stimulation of for flexible updating of contexts to control working-memory left DLPFC or stimulation of right DLPFC behavior and that activity in cells of the at offsets earlier than 150 ms or when en- dopaminergic midbrain may signal an updating. coding of the context was not required updating response. (context-independent trials). Thus, right One solution to the flexibility vs. stabil- DLPFC is necessary for context updating. example, it is established that the PFC is ity paradox has been to hypothesize a In a third experiment, D’Ardenne et al. necessary for controlled behavior (2, 8) working-memory “gate.” Such models as- (1) provide evidence for a link between sume that working memory, supported by and that PFC neurons exhibit delay period SN/VTA and DLPFC updating. Using a PFC, maintains contextual information activity (9, 10) and selectivity to task rules high-resolution fMRI technique, greater to modulate thought and action (2). (11). Similarly, evidence from functional activation for context-dependent than for Critical to such a system is a mechanism by MRI (fMRI) has implicated DLPFC in context-independent conditions was again which useful contextual information is gating responses (12, 13). observed in right DLPFC, but also in the updated into working memory and dis- Moreover, working-memory perfor- SN/VTA. Moreover, the activation in mance has been linked to DA. Midbrain SN/VTA correlated with behavior and tracting information is kept out; in other fi words, a working-memory gate. When the DA cells re transiently to cues during with the activation in right DLPFC. Thus, gate is open, available information can working-memory encoding (14). In PFC, these observations provide support for the enter working memory. When the gate is extracellular DA levels increase during dopamine-updating hypothesis in that closed, the current contents of working working-memory tasks (15), and perfor- not only is SN/VTA responsive to the pre- memory are sustained, while irrelevant mance suffers following DA depletion sentation of stimuli that predict reward, but information is kept out. (16). In general, higher DA levels in PFC also its activity is coordinated with frontal Having separate maintenance and drive stability over distractibility (17). regions necessary for updating. gating mechanisms is computationally ef- However, no evidence shows that DLPFC D’Ardenne et al. (1) provide an impor- ficient (3). And, although there are is necessary for flexible updating of con- tant step forward in our understanding of proposals of general form recurrent net- text information triggered by phasic re- working-memory updating. Of course, an works with diverse neural responses (akin sponses in SN/VTA. Using a combined- interesting result also raises a number of i to those in PFC) that can exhibit self- methods approach, D’Ardenne et al. (1) open questions. For example, ( ) Is up- ii gating behavior (4), the most influential provide evidence to fill this dating global or selective? ( ) Are there iii models of working-memory updating as- fundamental gap. multiple gating mechanisms? ( ) What is sume separate gating and maintenance In three experiments, human partic- the relationship between gating and the mechanisms (5, 6). Specifically, the PFC is ipants saw pairs of letters presented se- functional organization of frontal cortex? iv proposed to support noise-resistant main- quentially and responded on the basis of ( ) What is the reason for the right later- ’ tenance, such as via recurrence in a net- the identity of the letters. The first letter of ality of the effects observed by D Ardenne v work of active neurons (7). Gating is each pair was the cue and the second letter et al. (1)? ( ) What is the nature of the achieved by delivery of a brief stimulation was the probe. For some trials, the ap- interaction between episodic and working- vi that can elicit hysteresis in the resting propriate response entirely depended memory during updating? And ( )what network or make an active ensemble on the probe identity. These “context- available to new inputs. independent” trials did not require main- A prominent example of such a mecha- tenance of the cue as a context in working Author contributions: D.B. wrote the paper. nism was proposed by Braver and Cohen memory. In other trials, the appropriate The author declares no conflict of interest. (5). In their model, encounter with con- response to a probe was conditioned on See companion article on page 19900. textual information elicits phasic delivery the identity of the preceding cue. Thus, in 1E-mail: [email protected].

19878–19879 | PNAS | December 4, 2012 | vol. 109 | no. 49 www.pnas.org/cgi/doi/10.1073/pnas.1216902109 Downloaded by guest on September 27, 2021 COMMENTARY

is the relationship between the updating portant role in these models, as well. memory to influence behavior, “output function studied here and working-memory However, here SN/VTA is crucial for gating.” Given its advantages, some com- capacity? In what follows, the first two of training the striatal gating responses. putational models of cognitive control in- these questions are elaborated further, as Thus, these models separate learning and clude input and output gating (6, 22). For they relate to the nature of updating. gating into separate components. example, in a model of abstract rule The results of D’Ardenne et al. (1) are learning, output gating allowed a hierar- Is Working-Memory Updating Global or consistent with both global and selective chical corticostriatal network to gain a Selective? gating models. However, several other re- learning advantage over a nonhierarchical A potential limitation of assuming a global cent results have highlighted the impor- network and better matched human par- fl updating signal, such as that arising from tance of striatum in exibility and stability ticipants (23). By contrast, a hierarchical SN/VTA (5), is that it updates everything in in working memory (20). Also, recent network that leveraged input gating did not working memory. There are no means to fMRI data have provided support for se- show this benefit(alsoseeref.24).Notably, selectively hold one contextual representa- lective gating in separate regions of frontal this nested output gating model (23) pre- tion in working memory while updating cortex based on context abstraction, im- dicted an organization of frontostriatal a second one. However, selective updating plicating striatum but also parietal cortex connectivity that has been partly supported of this type is likely adaptive in a number of in gating (21). Thus, it remains open by fMRI (25) and by diffusion tractography circumstances, and so it is important to whether gating within working memory is know whether PFC gating can be selective selective or global and what systems might (26). However, the empirical evidence in or whether tasks requiring selective updat- support one or the other. brain and behavior for output gating of ing must be solved in other ways (such as working memory is primarily indirect (27, by relying on episodic memory). Are There Multiple Gates on Working 28). Thus, it remains open whether output At least one class of computational Memory? gating is supported by a similar DA gating models, such as those of O’Reilly and Gating can involve more than just deciding system to that of input gating. ’ Frank (6), highlights the striatum in de- what information to allow into working D Ardenne et al. (1) provide important livering selective gating inputs to PFC. memory. Indeed, it may be the case that evidence regarding the neural substrates These models assume that, through elab- we also want to select which maintained of working-memory updating and the im- orated frontostriatal-thalamic loops (19), representation is permitted to have an in- portance of interactions between DLPFC striatum can update recurrent activity in fluence over behavior. Thus, it may be and the SN/VTA for this function. These organized ensembles of frontal neurons, useful to distinguish gathering contextual results provide a basis for new inves- while information in other ensembles is information, “input gating,” from allowing tigations into the neural mechanisms of sustained. DA and SN/VTA play an im- the information resident in working flexible, goal-directed behavior.

1. D’Ardenne K, et al. (2012) Role of prefrontal cortex 11. Wallis JD, Anderson KC, Miller EK (2001) Single neu- ing basal ganglia and cortex. Annu Rev Neurosci 9: and the midbrain dopamine system in working mem- rons in prefrontal cortex encode abstract rules. Nature 357–381. ory updating. Proc Natl Acad Sci USA 109:19900–19909. 411(6840):953–956. 20. Cools R (2011) Dopaminergic control of the striatum 2. Miller EK, Cohen JD (2001) An integrative theory of 12. Postle BR (2005) Delay-period activity in the prefrontal for high-level cognition. Curr Opin Neurobiol 21(3): prefrontal cortex function. Annu Rev Neurosci 24: cortex: One function is sensory gating. J Cogn Neurosci 402–407. – 167 202. 17(11):1679–1690. 21. Nee DE, Brown JW (2012) Rostral-caudal gradients of 3. Hochreiter S, Schmidhuber J (1997) Long short-term 13. Nee DE, Brown JW (2012) Dissociable frontal-striatal abstraction revealed by multi-variate pattern analysis memory. Neural Comput 9(8):1735–1780. and frontal-parietal networks involved in updating hi- of working memory. Neuroimage 63(3):1285–1294. 4. Rigotti M, Ben Dayan Rubin D, Wang XJ, Fusi S (2010) erarchical contexts in working memory. Cereb Cortex, 22. Kriete T, Noelle DC (2011) Generalization benefits of Internal representation of task rules by recurrent dy- in press. output gating in a model of prefrontal cortex. Connect namics: The importance of the diversity of neural re- 14. Schultz W, Apicella P, Ljungberg T (1993) Responses of Sci 23(2):119–129. sponses. Front Comput Neurosci 4:24. monkey dopamine neurons to reward and conditioned 5. Braver TS, Cohen JD (2000) On the control of control: 23. Frank MJ, Badre D (2012) Mechanisms of hierarchical stimuli during successive steps of learning a delayed The role of dopamine in regulating prefrontal function reinforcement learning in corticostriatal circuits 1: response task. J Neurosci 13(3):900–913. and working memory. Attention and Performance Computational analysis. Cereb Cortex 22(3):509–526. 15. Watanabe M, Kodama T, Hikosaka K (1997) Increase of XVIII, eds Monsell S, Driver J (MIT Press, London), pp 24. Reynolds JR, O’Reilly RC (2009) Developing PFC repre- extracellular dopamine in primate prefrontal cortex 713–737. sentations using reinforcement learning. Cognition during a working memory task. J Neurophysiol 78(5): 6. O’Reilly RC, Frank MJ (2006) Making working memory 113(3):281–292. 2795–2798. work: A computational model of learning in the pre- 25. Badre D, Frank MJ (2012) Mechanisms of hierarchical 16. Brozoski TJ, Brown RM, Rosvold HE, Goldman PS (1979) frontal cortex and basal ganglia. Neural Comput 18(2): reinforcement learning in cortico-striatal circuits 2: Ev- – Cognitive deficit caused by regional depletion of do- 283 328. idence from FMRI. Cereb Cortex 22(3):527–536. 7. Durstewitz D, Seamans JK, Sejnowski TJ (2000) Neuro- pamine in prefrontal cortex of rhesus monkey. Science – 26. Verstynen TD, Badre D, Jarbo K, Schneider W (2012) computational models of working memory. Nat Neuro- 205(4409):929 932. Microstructural organizational patterns in the hu- sci 3(Suppl):1184–1191. 17. Cools R (2008) Role of dopamine in the motivational man corticostriatal system. JNeurophysiol107(11): 8. Stuss DT, Benson DF (1987) The frontal lobes and con- and cognitive control of behavior. Neuroscientist 14(4): 2984–2995. trol of cognition and memory. The Frontal Lobes Re- 381–395. 27. Farrell S, Lelièvre A (2012) The dynamics of access to visited, ed Perecman E (IRBN Press, New York), pp 18. Lenartowicz A, Escobedo-Quiroz R, Cohen JD (2010) 141–158. Updating of context in working memory: An event-re- groups in working memory. J Exp Psychol Learn Mem 9. Fuster JM, Alexander GE (1971) Neuron activity related lated potential study. Cogn Affect Behav Neurosci 10 Cogn 38(6):1659–1674. to short-term memory. Science 173(3997):652–654. (2):298–315. 28. Kiyonaga A, Egner T, Soto D (2012) Cognitive control 10. Goldman-Rakic PS (1995) Cellular basis of working 19. Alexander GE, DeLong MR, Strick PL (1986) Parallel over working memory biases of selection. Psychon memory. Neuron 14(3):477–485. organization of functionally segregated circuits link- Bull Rev 19(4):639–646.

Badre PNAS | December 4, 2012 | vol. 109 | no. 49 | 19879 Downloaded by guest on September 27, 2021