Signed Reward Prediction Errors Drive Declarative Learning and Are Encoded in the Ventral Striatum
Total Page：16
File Type：pdf, Size：1020Kb
SIGNED REWARD PREDICTION ERRORS DRIVE DECLARATIVE LEARNING AND ARE ENCODED IN THE VENTRAL STRIATUM Word count: 11147 Anna Snoeck Student number: 01502341 Promotor: Dr. Cristian Buc Calderon Co-promotor: Dr. Esther De Loof Supervisor: Kate Ergo Master’s Dissertation II submitted to Ghent University in partial fulfilment of the requirements for the degree of Master of Science in Psychology, main subject Theoretical and Experimental Psychology May 19, 2020 Academic year: 2019 – 2020 II Preamble Concerning COVID-19 This Master’s dissertation was written during the COVID-19 restrictions, namely during the so-called Belgian ‘lockdown’ from March to May 2020. Since Ghent University was closed during this period, the Faculty of Psychology and Educational Sciences asked its students to specify what kind of impact these restrictions had on our dissertation and its writing process. Concerning the data acquisition, all data was already collected last year (March-April 2019). Since we were all obligated to stay at home, meetings with my promotor and co- promotor were remotely. By video call we further discussed the behavioural and neural data analyses, and for quick questions about the writing process itself email communication was the way to go. Both my promotor and co-promotor were extremely responsive and helpful. In sum, except from not meeting my (co-)promotor in person, the COVID-19 restrictions did not really have a practical impact on finishing this Master’s dissertation. This preamble was drawn up by the student and the (co-)promotor, and is approved by both. III Corona Verklaring Vooraf Deze masterproef werd geschreven tijdens de COVID-19 maatregelen, meer bepaald tijdens de zogenaamde Belgische 'lockdown' van maart tot mei 2020. Aangezien de UGent gedurende deze periode gesloten was, vroeg de Faculteit Psychologie en Pedagogische Wetenschappen haar studenten om te specificeren welke impact deze maatregelen hadden op ons onderzoek en het schrijfproces. Wat de dataverzameling betreft, alle data werd vorig jaar (maart-april 2019) al verzameld. Omdat we nu eenmaal verplicht waren om thuis te blijven, verliepen de gesprekken met mijn promotor en copromotor vanop afstand. Via video-oproep bespraken we de details omtrent de data-analyse, en snelle vragen over het schrijven zelf werden gesteld via e-mail. Zowel mijn promotor als copromotor waren buitengewoon responsief en behulpzaam. Kortom, hoewel ik mijn (co)promotor niet persoonlijk heb kunnen spreken dit jaar, toch hadden de COVID-19 maatregelen niet echt een praktische impact op het afronden van deze masterproef. Deze preambule werd in overleg tussen de student en de promotor opgesteld en door beide goedgekeurd. IV Acknowledgements I would first like to thank my promotor Dr. Cristian Buc Calderon, my co-promotor Dr. Esther De Loof, my supervisor Kate Ergo and principal investigator Prof. Verguts for giving me the opportunity to contribute to their project, for introducing me to the field, and for their guidance and knowledge that got me to the end of my Master’s in Theoretical and Experimental Psychology. Honestly, I could not have wished for any better supervisors. Esther, you not only taught me how to code, but you also massively expanded my knowledge of and skills in statistics and data analysis. I cannot thank you enough for putting up with my endless questions, for encouraging me to keep going – no matter how tough it might get – and for your friendship created during our many real life and virtual videocall meetings. In all honesty, I have no clue how I can fully express my gratitude towards you. Cris, I cannot imagine how difficult it must have been to teach a Master’s student how to acquire fMRI data, let alone how to analyse it. I want to thank you for introducing me to cognitive neuroimaging, for teaching me how to analyse this kind of data (from across the ocean), and for being patient with me and always willing to help me out with whatever question I had. I also wish to extend my gratitude to Pieter Vandemaele and Dr. Bart Aben for their technical assistance during our many fMRI testing sessions. Many thanks to my parents, Anja and Johan, and grandparents, Cecile and Walter, without whom I would not be where I am today. They have been, and continue to be, an incredible source of inspiration and encouragement, as have my siblings Pieter-Jan, Marie and Sarah. There are so many friends who have made my life as a student in Ghent such an incredible and unforgettable experience. Particular thanks to Alison, Bieke and Gusta, who have been an irreplaceable source of support, laughter and friendship from the very first day. Thanks also go to my friends and class mates Britt, Elke, Hannah, Oona, Rien, Sarah, Selin and Toon for bonding over our common interest (and struggles) in experimental psychology. Finally, I am indebted to Jan for his unconditional support and belief in me during the hardest moments, and when I fell out of love with all of it. V Abstract The role of reward and its prediction errors within the learning literature is a topic of rising interest. Although reward prediction errors (RPEs) have been frequently investigated within procedural learning, its role within declarative learning remains rather obscure. Recent studies have started focusing on RPEs within declarative learning on a behavioural level and on the neural level using EEG. However, no attempt has been made yet to explore its neural underpinnings. Therefore, the objective of this study was to explore the neural basis of RPEs in a declarative learning paradigm using fMRI. We investigated the behavioural and neural effect of RPEs on recognition in a novel face-word association task, in which two types of RPEs were differentiated, namely signed RPEs (SRPEs) and unsigned RPEs (URPEs). Our behavioural data revealed a significant effect of SRPEs on recognition, in which recognition performance linearly increased with SRPE. Additionally, we observed increased bilateral activation in the ventral striatum as SRPEs increased. Based on these findings we conclude that SRPEs drive declarative learning, and are encoded in the ventrial striatum. Keywords: reward prediction error, declarative learning, episodic memory, ventral striatum, functional magnetic resonance imaging VI Nederlandstalige Samenvatting De rol van beloning en beloningsvoorspellingsfouten binnen de leerliteratuur is een onderwerp van toenemende belangstelling. Hoewel beloningsvoorspellingsfouten (reward prediction errors, RPE's) vaak onderzocht worden tijdens procedureel leren, blijft hun rol tijdens declaratief leren eerder onduidelijk. Recente studies zijn zich beginnen toeleggen op het onderzoeken van RPE's binnen declaratief leren op gedragsniveau en op neuraal niveau met behulp van EEG. Er zijn echter nog geen pogingen ondernomen om de neurale basis ervan te onderzoeken. Om deze reden was het doel van dit onderzoek om de neurale basis van RPE's te verkennen in een declaratief leerparadigma met behulp van fMRI. We onderzochten het gedrags- en neurale effect van RPE's op herkenning in een nieuwe gezichts- woord-associatietaak, waarin twee soorten RPE’s konden worden onderscheiden, namelijk signed RPE's (SRPE's) en unsigned RPE's (URPE's). Uit onze gedragsdata bleek een significant effect van SRPE's op herkenning, waarbij de herkenningsprestaties lineair verbeterden met de SRPE. Bovendien namen we een verhoogde bilaterale activatie in het striatum ventrale waar, naarmate SRPE's toenamen. Op basis van deze bevindingen concluderen we dat SRPE's declaratief leren stimuleren en hun oorsprong vinden in het striatum ventrale. Kernbegrippen: beloningsvoorspellingsfouten, declaratief leren, episodisch geheugen, striatum ventrale, functionele kernspintomografie VII Table of Contents Introduction 1 Learning 1 Definition and types. 1 Role of reward in learning. 2 Reward Prediction Errors (RPEs) 2 Definition. 2 Signed versus unsigned reward prediction errors. 3 Neural signatures of RPE. 4 RPEs in Declarative Learning 5 Behavioural effect 5 Neural signature. 5 The Present Study 6 Method 8 Participants 8 Design 8 Celebrities knowledge task. 8 Training session. 9 Learning task. 12 Functional localizer. 13 Recognition task. 13 Stimuli and Apparatus 15 fMRI data collection 15 Procedure 15 VIII Data Analysis 16 Behavioural analysis. 16 fMRI analysis. 16 Preprocessing. 16 First level analysis. 17 Second level analysis. 18 Results 19 Behavioural Results 19 Recognition accuracy: SRPE. 19 Recognition accuracy: reward vs. reward expectation. 20 Certainty and SRPE. 21 fMRI Results 22 SRPE contrast. 22 Discussion 24 Conclusion 27 References 28 Appendix 35 Appendix 1. List of Celebrity Names (410) 35 Appendix 2. List of Swahili Words (280) 39 IX List of Figures Figure 1. Memory Classification: Declarative vs. Procedural Memory. 1 Figure 2. Scheme of Learning by a Reward Prediction Error (RPE). 3 Figure 3. Celebrity Knowledge Task. 9 Figure 4. Training Session and Learning Task. 12 Figure 5. Choice-Feedback Training Session and Learning Task. 12 Figure 6. Functional Localizer Task. 13 Figure 7. Recognition Task. 14 Figure 8. Recognition Accuracy: SRPE Effect. 19 Figure 9. Recognition Accuracy in Function of Reward and RPEs. 20 Figure 10. Certainty and SRPE. 21 Figure 11. Spatial Visualisation of SRPE contrast. 23 X List of Tables Table 1. Summary of ANOVA Type III Tests Per Model. 22 Table 2. Summary of the Activation Cluster. 23 XI List of Abbreviations Abbreviations Description ANOVA Analysis of Variance ART Artifact Detection Toolbox BIDS Brain Imaging Data