DOCSLIB.ORG

Prefrontal Mechanisms Underlying Sequential Tasks by Feng-Kuei

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Prefrontal Mechanisms Underlying Sequential Tasks by Feng-Kuei Prefrontal Mechanisms Underlying Sequential Tasks By Feng-Kuei Chiang A dissertation submitted in partial satisfaction of the requirements for the degree of Doctor of Philosophy in Psychology in the Graduate Division of the University of California, Berkeley Committee in charge: Professor Joni Wallis, Chair Professor Richard B. Ivry Professor Ming Hsu Summer 2017 Prefrontal Mechanisms Underlying Sequential Tasks Copyright c 2017 by Feng-Kuei Chiang Abstract Prefrontal Mechanisms Underlying Sequential Tasks by Feng-Kuei Chiang Doctor of Philosophy in Psychology University of California, Berkeley Professor Joni Wallis, Chair For decades, mechanisms of cognitive behaviors have been studied in a simple form of stimulus- and action-outcome associations. Those seminal studies serve as funda- mental frameworks and enable us to explore how neural activities in brain represent the increasingly complex and temporarily-extended associations in sequential tasks. The prefrontal cortex (PFC) has long been suspected to play an important role in cognitive control, in the ability to orchestrate thought and action in accordance with internal representation. In particular, PFC \top-down" processes serve as an internal signal to guide high-level cognitive functions, such as working memory (WM), abstract rules, or goal-directed decision-making. Several models have described how task information, in- cluding supra- and super-ordinate information, is organized in prefrontal cortex, but it remains unclear precisely how this cognitive information maps onto neurophysiological functions. To explore these issues, we devised primate versions of two tasks that tax sequential behavior: a spatial self-ordered search task and a hierarchical reinforcement learning (HRL) task. These tasks examine how sequential behavior interacts with WM and reinforcement learning (RL), respectively. We examined how prefrontal neurons en- coded task-related information across these two cognitive tasks. Our results show that prefrontal neurons are capable of adaptively regulating the precision with which infor- mation is encoded. In the spatial self-ordered search task, lateral prefrontal neurons have spatiotemporal mnemonic fields, in that their firing rates are modulated both by the spatial location of future selection behaviors and the temporal organization of that behavior. Furthermore, the precision of this tuning can be dynamically modulated by the demands of the task. In the HRL task, prefrontal neurons are involved in inte- grating the abstract subject values. Especially, we found that the firing rate of a small population of neurons encoded pseudo-reward prediction errors and these neurons were restricted to anterior cingulate cortex. Taken together, our findings suggest that pre- frontal neurons encode not only basic information associated with external stimuli, but also high-level information that is used to organize task relevant behaviors. 1 To my parents, family, and friends All is Well i Contents 1 Introduction 1 1.1 Outline of thesis . 2 1.2 Prefrontal cortex (PFC) . 3 1.2.1 Lateral prefrontal cortex (LPFC) . 3 1.2.2 Orbitofrontal cortex (OFC) . 5 1.2.3 Anterior cingulate cortex (ACC) . 7 1.2.4 Closing remarks on anatomy . 9 1.3 The role of PFC in working memory . 9 1.4 The role of PFC in reinforcement learning . 10 1.4.1 Historical perspective . 11 1.4.2 Reinforcement learning and hierarchical behaviors . 11 2 General Methods 19 2.1 Overview . 19 2.2 Behavioral training materials and methods . 19 2.2.1 Subjects . 19 2.2.2 Behavioral training . 20 2.2.3 Materials and methods for training . 20 2.3 Neurophysiological techniques . 21 2.3.1 Isolation of recording sites . 21 2.3.2 Surgery . 22 2.4 Recordings . 23 2.4.1 Materials and methods for neurophysiology . 23 2.5 Statistical analysis . 24 3 Spatiotemporal encoding of search strategies by prefrontal neurons 28 3.1 Introduction . 28 3.2 Methods . 29 3.3 Results . 30 3.3.1 Task performance . 30 3.3.2 Behavioral strategies . 31 3.3.3 Neurophysiological analysis . 32 3.3.4 Encoding of spatial information . 33 ii 3.3.5 Effects of behavioral strategy on spatial encoding . 34 3.4 Discussion . 35 3.4.1 Role of prefrontal cortex in the sequential organization of behavior . 35 3.4.2 Contribution of PFC to working memory . 36 3.4.3 Role of PFC in cognitive control . 37 4 Neuronal encoding in prefrontal cortex during hierarchical reinforcement learning 48 4.1 Introduction . 48 4.2 Methods . 49 4.2.1 Subjects and behavioral task . 49 4.2.2 Neurophysiological procedures . 51 4.2.3 Statistical methods . 51 4.3 Results . 54 4.3.1 Behavioral task performance . 54 4.3.2 Neural encoding . 55 4.4 Discussion . 55 5 Conclusion 67 5.1 Summary of results . 67 5.2 Remaining questions . 68 5.2.1 Errors in sequential tasks . 68 5.2.2 Adaptation from prediction errors . 69 5.3 Future directions . 69 5.4 Closing remarks . 70 6 Bibliography 71 iii List of Figures 1.1 Hierarchical representation of a routine sequential task. From Humphreys and Forde, 1998. 14 1.2 (A) Flow of control in one step of a sequential task, with blue representing the increased involvement of supervisory control and red representing increased involvement of schematic control during a single step. (B) Representation of the multiple, hierarchical levels that can characterize sequences. Each step in more concrete motor sequences or more abstract task sequences may engage supervisory or schematic control and the interaction between them. From Desrochers et al. (2015). 15 1.3 The medial, lateral, and orbital surfaces of the prefrontal cortex. Monkey outlines taken from Carmichael and Price (1996) and Petrides and Pandya (1999); human outline taken from Ongur and Price (2000). 16 1.4 Medial and orbital networks of the prefrontal cortex taken from Carmichael and Price (1996). Adapted and re-printed with permission from Wallis (2012). 17 1.5 Overview of Content-Specific Activity during Working Memory Delays in the Macaque (Left) and Human (Right) Brain. Icons indicate persistent stimulus- selective activity for each stimulus type indicated by the icon (see legend) at the respective locations. Both left- and right-sided effects are shown on the left hemisphere. A full list of individual studies is reported in the supple- mental information from Christophel, et al., 2017. Brain areas are identified by abbreviations: AC, auditory cortex; ERC, entorhinal cortex; EVC, early visual cortex; FEF, frontal eye fields; FG, fusiform gyrus; hMT+, human analog to MT/MST; IPS, intraparietal sulcus; IT, inferior temporal cortex; LOC, lateral occipital complex; lPFC, lateral prefrontal cortex; PM, premotor cortex; PPC, posterior parietal cortex. 18 2.1 Experimental set up used to control behavioral events and record neurophys- iological data. 25 2.2 Magnetic resonance imaging (MRI) scans illustrating the coronal slice from the middle of our recording locations illustrating potential electrode paths. Example on the top is from subject R in WM task. Example on the bottom is from subject Q in HRL task. Possible electrode tracks are highlighted in white. Brain areas recorded from are highlighted in red (LPFC), green (ACC), and blue (OFC). 26 iv 2.3 Cluster isolation of neuronal waveforms. On the left are 32 channels, of which many have neurons on them, i.e., detected waveforms. A sample channel is zoomed in in the middle panel to reveal three distinct waveforms. Those waveforms are decomposed into components in Plexon's online sorter, and each waveform is plotted as a single dot in the right panel. Clusters of wave- forms are then isolated manually and entered into neuronal analyses. 27 3.1 (A) Spatial self-ordered search task. (B) Each configuration consisted of 6 targets (green filled circle), which were selected from 36 possible locations (gray filled circle). We ensured that targets were approximately balanced across the display by requiring that the centroid of the configuration (red cross) was located within ± 3◦ fixation window (black circle). The inter- target distances were spaced at least 6◦ to avoid overlap of the eye position detection window around the target. (C) Number of unique sequences of target selection per configuration. 39 3.2 Behavioral performance. (A) Distribution of the number of incorrect saccades per trial. (B) Observed and expected error rates plotted as a function of which saccade in the sequence was performed. The observed error rate was significantly higher than the expected error rate for the last saccade in both subjects (binomial test, ∗ ∗ ∗p < 0:001). (C) Reaction times as a function of saccade. Boxplots indicate the 25th, 50th, and 75th percentiles of the sRT distribution. Red dots indicated the mean sRT. 40 3.3 (A) For each block of trials, we calculated how many times a particular target was selected by a particular saccade. We then plotted this data according to the most common sequence by which targets were selected. (B) Representative blocks, illustrating how the pseudocolor plots changed as a function of the stereotype index. Blocks above the line are from subject R, while blocks below the line are from subject Q. (C) Increased values of the stereotype index were correlated with better behavioral performance (fewer incorrect saccades) in both subjects. 41 3.4 Recording locations. (A) MRI of a coronal slice through the frontal lobe of subject R. Red region in each hemisphere denotes the area of the LPFC inves- tigated. White lines depict electrode paths. (B) We measured the anterior- posterior position from the interaural line (x-axis), and the lateral-medial position relative to the lip of the ventral bank of the principal sulcus (0 point on y-axis). Gray shading indicates unfolded sulci. Diameter of the circles in- dicate the number of recordings from a given location. SA = superior arcuate sulcus; IA = inferior arcuate sulcus; P = principal sulcus. 42 3.5 (A) Percentage of selective neurons that encoded different predictors in each epoch.
Load more

Recommended publications
  • Erin L. Rich, MD, Phd Tel
    Erin L. Rich, MD, PhD tel. (212) 824-9304 Hess CMS Building [email protected] Room 10-115 1470 Madison Ave New York, NY 10029 Professional 2017 – present Assistant Professor (Tenure-track), Department of Neuroscience and Friedman Brain Institute, Icahn School of Medicine at Mount Sinai Date of appointment: April 1, 2017 2015 – 2017 Assistant Project Scientist, Helen Wills Neuroscience Institute, University of California, Berkeley 2014 – 2017 Affiliate Researcher, Department of Neurological Surgery, University of California, San Francisco 2010 – 2015 Post-doctoral Fellow, University of California, Berkeley Mentor: Joni Wallis, PhD 2014 – 2017 Co-mentor: Edward Chang, MD, UC San Francisco Education MD 2010, Mount Sinai School of Medicine, New York, NY Medical Scientist Training Program (MSTP), 2002 – 2010 PhD 2008, Mount Sinai School of Medicine, New York, NY Department of Neuroscience BS 2002, Tufts University, Medford, MA Majors in Biology and Classical Studies Awards & 2019 – 2021 Brain and Behavior Research Foundation NARSAD Young Research Investigator Grant The Neural Basis of Expectation and Bias in Large-scale Support Brain Networks 2018 – 2021 Whitehall Foundation Research Grant Neural mechanisms of self- generated mnemonic strategies 2018 – 2022 Pew Scholars Program in Biomedical Sciences, The neural basis of expectation and bias in large-scale orbitofrontal networks 2018 Schneider-Lesser Foundation Junior Faculty Fellowship 2015 – 2020 K08 Mentored Clinical Scientist Research Development Award, NIH/NIDA, Multi-Scale Orbitofrontal
    [Show full text]
  • Covert Shift of Attention Modulates the Value Encoding in the Orbitofrontal Cortex Yang Xie1,2, Chechang Nie1,2, Tianming Yang1*
    RESEARCH ARTICLE Covert shift of attention modulates the value encoding in the orbitofrontal cortex Yang Xie1,2, Chechang Nie1,2, Tianming Yang1* 1Institute of Neuroscience, Key Laboratory of Primate Neurobiology, CAS Center for Excellence in Brain Science and Intelligence Technology, Shanghai Institutes for Biological Sciences, Chinese Academy of Sciences, Shanghai, China; 2University of Chinese Academy of Sciences, Beijing, China Abstract During value-based decision making, we often evaluate the value of each option sequentially by shifting our attention, even when the options are presented simultaneously. The orbitofrontal cortex (OFC) has been suggested to encode value during value-based decision making. Yet it is not known how its activity is modulated by attention shifts. We investigated this question by employing a passive viewing task that allowed us to disentangle effects of attention, value, choice and eye movement. We found that the attention modulated OFC activity through a winner-take-all mechanism. When we attracted the monkeys’ attention covertly, the OFC neuronal activity reflected the reward value of the newly attended cue. The shift of attention could be explained by a normalization model. Our results strongly argue for the hypothesis that the OFC neuronal activity represents the value of the attended item. They provide important insights toward understanding the OFC’s role in value-based decision making. DOI: https://doi.org/10.7554/eLife.31507.001 Introduction Imagine you are standing before a fruit stand and trying to buy some apples. Facing a box of apples, *For correspondence: you usually will pick up one apple at a time, evaluate it, decide whether you will have it or leave it, [email protected] and then move on to the next apple until you have chosen enough.
    [Show full text]
  • UC Berkeley UC Berkeley Electronic Theses and Dissertations
    UC Berkeley UC Berkeley Electronic Theses and Dissertations Title Prefrontal Mechanisms Underlying Sequential Tasks Permalink https://escholarship.org/uc/item/7510v08r Author Chiang, Feng-Kuei Publication Date 2017 Peer reviewed|Thesis/dissertation eScholarship.org Powered by the California Digital Library University of California Prefrontal Mechanisms Underlying Sequential Tasks By Feng-Kuei Chiang A dissertation submitted in partial satisfaction of the requirements for the degree of Doctor of Philosophy in Psychology in the Graduate Division of the University of California, Berkeley Committee in charge: Professor Joni Wallis, Chair Professor Richard B. Ivry Professor Ming Hsu Summer 2017 Prefrontal Mechanisms Underlying Sequential Tasks Copyright c 2017 by Feng-Kuei Chiang Abstract Prefrontal Mechanisms Underlying Sequential Tasks by Feng-Kuei Chiang Doctor of Philosophy in Psychology University of California, Berkeley Professor Joni Wallis, Chair For decades, mechanisms of cognitive behaviors have been studied in a simple form of stimulus- and action-outcome associations. Those seminal studies serve as funda- mental frameworks and enable us to explore how neural activities in brain represent the increasingly complex and temporarily-extended associations in sequential tasks. The prefrontal cortex (PFC) has long been suspected to play an important role in cognitive control, in the ability to orchestrate thought and action in accordance with internal representation. In particular, PFC \top-down" processes serve as an internal signal to guide high-level cognitive functions, such as working memory (WM), abstract rules, or goal-directed decision-making. Several models have described how task information, in- cluding supra- and super-ordinate information, is organized in prefrontal cortex, but it remains unclear precisely how this cognitive information maps onto neurophysiological functions.
    [Show full text]
  • Talk & Poster Abstracts
    TALK &POSTER ABSTRACTS WWW.RLDM.ORG TABLE OF CONTENTS PREFACE 3 INVITED SPEAKER ABSTRACTS 4 CONTRIBUTED SPEAKER ABSTRACTS 9 MONDAY POSTER ABSTRACTS 14 TUESDAY POSTER ABSTRACTS 74 PROGRAM COMMITTEE 132 2 Preface Welcome to Reinforcement Learning and Decision Making 2019! Over the last few decades, reinforcement learning and decision making have been the focus of an incredible wealth of research in a wide variety of fields including psychology, animal and human neuroscience, artifi- cial intelligence, machine learning, robotics, operations research, neuroeconomics and ethology. All these fields, despite their differences, share a common ambition—understanding the information processing that leads to the effective achievement of goals. Key to many developments has been multidisciplinary sharing of ideas and findings. However, the com- monalities are frequently obscured by differences in language and methodology. To remedy this issue, the RLDM meetings were started in 2013 with the explicit goal of fostering multidisciplinary discussion across the fields. RLDM 2019 is the fourth such meeting. Our primary form of discourse is intended to be cross-disciplinary conversations, with teaching and learn- ing being central objectives, along with the dissemination of novel theoretical and experimental results. To accommodate the variegated traditions of the contributing communities, we do not have an official pro- ceedings. Nevertheless, some authors have agreed to make their extended abstracts available, which can be downloaded from the RLDM website. We would like to conclude by thanking all past organizers, speakers, authors and members of the program committee. Your hard work is the bedrock of a successful conference. We hope you enjoy RLDM2019. Satinder Singh, General chair Catherine Hartley and Michael L.
    [Show full text]
  • About Cosyne
    Program Summary All times are EST (Eastern Standard Time, GMT-5) Tuesday, 23 February 11:00a Cosyne Tutorial: Recurrent Neural Networks for Neuroscience 03:00p Paths for leadership in promoting the careers of scientists Wednesday, 24 February 08:00a Poster Session 1a 10:00a Session 1: 3 accepted talks 11:40a Session 2: 4 accepted talks 01:00p Simons Foundation Bridge to Independence Awards: Facilitating the transition from postdoc to junior faculty 02:00p Session 3: 3 accepted talks 03:00p Poster Session 1b 05:00p Mini-symposium on the history of race and racism in science 07:00p Poster Session 1c Thursday, 25 February 08:00a Poster Session 2a 10:00a Session 4: 3 accepted talks 11:40a Session 5: 4 accepted talks 02:00p Session 6: 3 accepted talks 03:00p Poster Session 2b 05:00p Expanding horizons: Diverse perspectives in computational neuroscience 07:00p Poster Session 2c Friday, 26 February 08:00a Poster Session 3a 10:00a Session 7: 3 accepted talks 11:40a Session 8: 4 accepted talks 02:00p Session 9: 3 accepted talks 03:00p Poster Session 3b 07:00p Poster Session 3c COSYNE 2021 i ii COSYNE 2021 About Cosyne About Cosyne The annual Cosyne meeting provides an inclusive forum for the exchange of experimental and theoretical/computational approaches to problems in systems neuroscience. To encourage interdisciplinary interactions, the main meeting is arranged in a single track. A set of invited talks are selected by the Executive Committee and Organizing Commit- tee, and additional talks and posters are selected by the Program Committee, based on submitted abstracts.
    [Show full text]