On the Use of Cognitive Maps

On the use of cognitive maps adam johnson University of Minnesota This is to certify that I have examined this copy of a doctoral dissertation by Adam Conrad Johnson and have found that it is complete and satisfactory in all respects, and that any and all revisions required by the final examining committee have been made. Name of Faculty Adviser Signature of Faculty Adviser Date Graduate school On the use of cognitive maps A dissertation submitted to the faculty of the graduate school of the University of Minnesota by Adam Conrad Johnson in partial fulfillment of the requirements for the degree of doctor of philosophy Adviser: A. David Redish June 2008 c Adam Conrad Johnson 2008 Acknowledgements I am profoundly grateful to many people for their contributions to my graduate studies and this thesis. Dave Redish fundamentally shaped this work. Even though he was surprised by many of these experimental findings and theoretical conclusions, he provided an environment of support and challenge within which these theoretical perspectives developed and these observations were made. His willingness to chase a ideas and to challenge dogma made science exciting and interesting where it might have otherwise been a simple collection of facts. The members of the Redish lab - Mallika Arudi, Deborah Bang, Dan Bernal, Chris Boldt, Alex Colvin, Giuseppe Cortese, John Ferguson, Maniezheh Firouzi, Mandy Huber, Jadin Jackson, Steve Jensen, Sarah Jutila, Monica Kumar, Zeb Kurth-Nelson, Morgan Little, Beth Masimore, Matt van der Meer, Susan Nwoke, Neil Schmitzer-Torbert, Kelsey Seeland - all provided fundamental contributions to this work, through technical support, experimental assistance or discussions. I would like to highlight Matt van der Meer’s questions, discussions and all-around resourcefulness which prompted and refined many of these ideas. Paul Schrater provided important theoretical discussions and feedback along with much statistical assistance. J. Bruce Overmier and the ”Mental time travel in animals” seminar provided much of the conceptual fodder that connected a variety of disparate ideas and observations. Bruce also provided much encouragement to go back to many original sources including Tolman’s observations on vicarious trial and error. My interactions with Paul Schrater and Bruce Overmier developed through the Center for Cognitive Sciences and has been a source of inspiration and answers throughout my education. The Cognitive Critique reading group, particularly Celia Gershonson, also provided valuable feedback on early ideas and drafts of the paper on cognitive maps. Edvard and May-Britt Moser and the Center for the Biology of Memory deserve special mention for their time and support. Although no experiments from their lab are presented i in the thesis, many of the ideas were refined in their lab. Gerit Pfuhl and Robert Beigler provided much support and many discussions on cognitive maps. David Crowe, Alexandra Basford, Thomas Naselaris and Matt Chafee provided important points of perspective along the way. From the start, David Crowe challenged me to think deeply about the difference between an experimental result and an experimental insight. Finally, I have been amazingly blessed and continually surprised by the grace of my family - Holly, Isabelle and Elise Johnson - as I wrestled with these experiments and ideas. Funding sources This work was supported by NIH (MH68029), NSF (IGERT9870633), a training fellowship from the Center for Cognitive Sciences (NIH-T32HD007151), fellowships from 3M and the graduate school (a first year graduate student fellowship and a doctoral dissertation fellowship) and a Fulbright scholarship. ii To I and E who teach me how to see. iii Abstract Cognitive maps were proposed as an alternative to stimulus-response explanations of animal behavior. Although the concept of cognitive maps advanced treatments of complex animal behavior, it has remained resistant to theoretical definition. A simplified perspective on cognitive maps that focused on spatial behavior and the construction of spatial maps has provided an important approach to understanding the role of the hippocampus in spatial behavior and spatially modulated neural activity, particularly within the hippocampus. However, this perspective leaves open many questions on how spatial maps and neural activities within the hippocampus are used and how they contribute to selection of adaptive actions. A reinforcement learning approach to animal behavior was used to develop a theory of cognitive map function. Reinforcement learning provides a theoretical framework within which the components of cognitive map function can be readily defined and explored. This approach addresses long-standing criticisms of cognitive map theory by explicit mapping of stimuli to action via specific, albeit behaviorally unobservable, computations. Flexible behavior associated with cognitive maps implies the use of transition models in reinforcement learning algorithms. In contrast to model-free algorithms that depend on current experience only, model-based reinforcement algorithms represent sensory or state informa- tion beyond the modeled animal’s current sensory experience. As a result, model-based reinforcement learning provides a principled approach to analysis of neural representations and the dynamic processes that support cognition. Neurophysiological recordings in the hippocampus showed that apparent noise present in spatially modulated place cell activity could be explained as coherent spatial representations that deviated from the animal’s position on the maze. These non-local representations were associated with fast spatial representation dynamics and were typically found when the animal was at feeder locations or choice points. Non-local representations at choice points shifted forward of the animal to potential future spatial positions and were associated with iv theta and gamma local field potential activity. Forward-shifted spatial representations were associated with vicarious-trial-and-error behaviors and were task and experience dependent. In sum, these results suggest how cognitive maps in the hippocampus can contribute to selection of adaptive actions through the construction of past events and potential future experiences. v Contents 1 Cognition and cognitive maps 1 1.1Problemstatement................................ 1 1.2Earlyobservationsandtheformationofcognitivemaps........... 2 1.2.1 Tolman and cognitive maps ....................... 3 1.2.2 Subsequent readings and perspectives . ............. 10 1.2.3 Currentbehavioralapproachestocognitivemaps........... 11 1.2.4 Conclusions................................ 18 1.3Cognitivemapsandthehippocampus..................... 20 1.3.1 Spatialbehaviorandthehippocampus................. 22 1.3.2 Placecells................................. 26 1.3.3 Cellassemblies.............................. 28 1.3.4 Mentaltimetravel,spatialimageryandepisodicmemory...... 31 1.3.5 Conclusions................................ 33 1.4Multiplememorysystems............................ 34 1.4.1 Outcomevaluationwithininstrumentaltasks............. 35 1.4.2 Instrumentalcontingencies....................... 36 1.4.3 Goalsandoutcomes........................... 39 1.4.4 Conclusions................................ 41 1.5 A modern theoretical/statistical/Bayesian approach to cognitive maps . 41 1.5.1 Experimentalevidence.......................... 45 1.5.2 Discussion................................. 47 1.6Conclusions.................................... 48 2 Reinforcement learning 49 2.1Basicformalisms................................. 50 2.2Model-freereinforcementlearning........................ 51 vi 2.3Model-basedreinforcementlearning....................... 53 2.3.1 Multiplememorysystems........................ 53 2.3.2 Arbitration of behavioral control – Daw et al. (2005) . ...... 54 2.3.3 Memorydynamicsinreinforcementlearning.............. 57 2.3.4 Analysis of working and episodic memory – Zilli and Hasselmo (2008) 58 2.4Discussion..................................... 62 3 State dynamics and learning 64 3.1Introduction.................................... 64 3.2Model....................................... 65 3.2.1 Reinforcementlearning.......................... 65 3.2.2 Thepracticesignal............................ 66 3.2.3 Analysis.................................. 68 3.3Results....................................... 68 3.4Discussion..................................... 69 3.4.1 Assumptions............................... 69 3.4.2 Predictions................................ 70 3.4.3 Openquestions.............................. 71 3.5Conclusion.................................... 72 4 Statistical methods 77 4.1Neuralrepresentations.............................. 77 4.2Analysisofneuralrepresentations........................ 78 4.2.1 Encodingandtuningcurves....................... 78 4.2.2 Populationdecoding........................... 79 4.2.3 Decodingatfasttimescales....................... 80 4.2.4 Memoryandcognitionasnon-localrepresentations.......... 81 4.3Consistencyanalysis............................... 85 4.3.1 Basicconsistencyanalysis........................ 85 4.3.2 Consistencyandcoherency....................... 86 4.3.3 ABayesianapproachtoconsistency.................. 88 4.3.4 Multiplemodelsinhippocampus.................... 89 4.3.5 Conclusions................................ 93 vii 5 Experiments 95 5.1Methods...................................... 95 5.1.1 Subjects.................................. 95 5.1.2 Behavioraltraining...........................

Load more