Conjunctive Coding of Complex Object Features
Total Page:16
File Type:pdf, Size:1020Kb
Conjunctive Coding of Complex Object Features by Jonathan Erez A thesis submitted in conformity with the requirements for the degree of Doctor of Philosophy Graduate Department of Psychology University of Toronto © Copyright by Jonathan Erez 2015 Conjunctive Coding of Complex Object Features Jonathan Erez A thesis submitted in conformity with the requirements for the degree of Doctor of Philosophy Graduate Department of Psychology University of Toronto 2015 Abstract Critical to perceiving an object is the ability to bind its constituent features into a cohesive representation, yet the manner by which the visual system integrates object features to yield a unified percept remains unknown. Another important aspect of object perception is the ability to represent complex objects across different viewing angles, despite the drastic variability in appearance caused by shifting viewpoints. This dissertation describes a novel application of multivoxel pattern analysis (MVPA) of neuroimaging data that allows a direct investigation of whether neural representations integrate object features into a whole that is different from the sum of its parts and whether these representations are view invariant. Results showed that patterns of activity throughout the ventral visual stream (VVS), extending anteriorly into the perirhinal cortex (PRC), discriminated between the same sets of features when combined into different objects. Despite this sensitivity to the unique conjunctions of features, this activity was invariant to the viewpoints from which the conjunctions were presented. These results suggest that the manner in which our visual system processes complex objects depends on the explicit coding of the feature conjunctions comprising them, and that these representations are invariant to large changes in visual appearance caused by shifting viewpoints. To test whether experience with the objects changed the pattern of conjunctive representations across the VVS, participants ii were scanned in a separate fMRI study following an extensive behavioural training period. Results showed that both view-dependent and view-invariant conjunctive representations shifted to more posterior regions of the VVS, suggesting that training affected how unique feature conjunctions are represented across the VVS. These results support the representational- hierarchical view of cortical organization and contribute to our understanding of object perception and memory. iii Acknowledgments First and foremost, I would like to thank my advisor, Dr. Morgan Barense, for her support and guidance throughout my PhD. Thank you for your patience, generosity and for encouraging me to become a better researcher and writer. I would also like to thank the members of my thesis committee, Dr. Susanne Ferber and Dr. Brad Buchsbaum for their valued input and encouragement during this process. In addition, I wish to thank Dr. Rhodri Cusack for his theoretical and technical expertise. Many thanks to my labmates, Rachel Newsome, Lok-Kin Yeung, Danielle Douglas, and Celia Fidalgo and the rest the members of the Barense Lab for their friendship and for fruitful discussions over the years. Special thanks to Felicia Zhang and Will Kendall for their assistance with the experiments in this thesis, and thanks to all of the participants that agreed to be scanned, and rescanned. I also would like to thank NSERC, OGS, and the University of Toronto for their generous financial support of this research, as well as the hot dog van outside Sid Smith for staying open late most evenings to benefit from the aforementioned funding. I am grateful for my wife, Noam, for her unwavering support during this process. Your love and patience made this possible. Lastly, I’d like to thank my parents who inspired me to pursue my passions and instilled in me a thirst for knowledge. iv Contents Acknowledgments.......................................................................................................................... iv List of Tables ................................................................................................................................. ix List of Figures ................................................................................................................................. x List of Appendices ........................................................................................................................ xii General Introduction ....................................................................................................................... 1 1.1 The binding problem ............................................................................................................. 1 1.2 The view-invariance problem................................................................................................ 4 1.3 How might the brain solve these problems? ......................................................................... 7 1.4 fMRI MVPA ....................................................................................................................... 12 1.5 Thesis overview................................................................................................................... 15 Chapter 2: Neural representations of feature conjunctions composing complex objects ............. 17 2.1 Introduction ......................................................................................................................... 17 2.2 Materials and Methods ........................................................................................................ 23 Participants ............................................................................................................................ 23 Experimental Stimuli and Design ......................................................................................... 23 Tasks ..................................................................................................................................... 24 fMRI Data Acquisition ......................................................................................................... 27 Multi-Voxel Pattern Analysis (MVPA) ................................................................................ 27 Control analysis to test for sub-additivity of the BOLD signal ............................................ 33 v 2.3 Results and Discussion ........................................................................................................ 35 Chapter 3: Neural representations of feature conjunctions composing complex objects: View Invariance ...................................................................................................................................... 42 3.1 Introduction ......................................................................................................................... 42 3.2 Materials and Methods ........................................................................................................ 44 Participants ............................................................................................................................ 44 Experimental Stimuli and Design ......................................................................................... 44 Image similarity analysis. ..................................................................................................... 44 Experimental task.................................................................................................................. 48 3.3 Results and Discussion ........................................................................................................ 48 Chapter 4: Experience modifies the conjunctive neural representations necessary for complex object perception: behavioural training ........................................................................................ 53 4.1 Introduction ......................................................................................................................... 53 4.2 Materials and Methods ........................................................................................................ 57 Participants ............................................................................................................................ 57 Materials ............................................................................................................................... 57 Design and Behavioural Procedure ....................................................................................... 59 Data analysis ......................................................................................................................... 61 4.3 Results and discussion ......................................................................................................... 62 Visual search performance across training sessions ............................................................. 62 vi Effect of viewpoint congruency between sample target and choice target ........................... 65 4.4 General Discussion .............................................................................................................. 70 Chapter 5: Experience modifies the conjunctive neural representations necessary for complex object perception ........................................................................................................................... 74 5.1. Introduction .......................................................................................................................