On the neuronal activity in the human brain during visual recognition, imagery and binocular rivalry Thesis by Gabriel Kreiman In partial fulfillment of the requirements for the degree of Doctor of Philosophy California Institute of Technology Pasadena, California 2002 (Defended August 29, 2001) ii © 2002 Gabriel Kreiman All Rights Reserved iii To all those teachers who taught me to enjoy learning iv v Acknowledgments Haec ego non multis scribo, sed tibi: satis enim magnum alter alteri theatrum sumus1. All of the work described in the following chapters would not have been possible without the enthusiastic help of a large and nice group of colleagues and friends. First of all, I would like to thank all the patients who participated in these experiments. We have not paid them and they have cooperated for the advancement of science. Candi Maechtlen and Irene Wainwright have provided valuable and prompt help throughout these years in several of the fundamental details that keep things going. Irene has also provided excellent editorial assistance in several of our publications. I have been fortunate to be able to interact and learn from a large number of teachers, colleagues and friends throughout my entire life. While a number of them have not contributed directly to this thesis work, it seems evident that they have enormously influenced my education and my preparation. Among them, I would like to especially mention my chess teacher Cesar Corte who was perhaps among the first who showed me the way to indulge in the wonderful task of solving new problems. The long hours of scientific exploration clearly have a root in me in the uncountable hours that we spent in front of a chessboard trying to figure out the solution to a problem or the best move. I would also like to express my gratitude to Rebeca Muller, Daniel Goldstein, Alberto Kornblihtt, Ernersto Timmermann, Rodolfo Ugalde and the math Olympiad team in Buenos Aires. I would like to also mention the continuous support from my family Raul, Lidia, Lauri, Elias, Haydee, Dora, Mariela and friends Damian Scherlis, Dan Rozenfarb, Diego Tisman and Dario Winter. When I arrived to Caltech, I learnt the first steps in Neuroscience during my rotation in the lab of Mark Konishi to whom I am greatly indebted. Fabrizio Gabbiani immersed me in the wonderful world of quantitative spike train analysis. I would like to 1 I write this not to the many, but to you only, for you and I are surely enough of an audience for each other. vi also thank Erin Schuman who introduced me to the hippocampus and taught me the first steps in electrophysiology. A number of people from UCLA and Caltech have helped me with the recordings including Anatol Bragin, Charles Wilson, Rick Staba, Erick Behnke, Jack Morrow and Eve Isham. Laurent Itti built the push buttons used in all our experiments. The data obtained during sleep (Chapter 5) were recorded by the skilful and laborious endeavor of Charles Wilson and Rick Staba. Jonathan Lin, a Surf student from Caltech, provided fundamental help for the analysis and study of the correlations during sleep. Most of the experiments to try to understand the psychophysics of flash suppression (briefly mentioned in Appendix 2) were carried out by SURF student Nick Knouf and by the enthusiastic collaboration of Pasadena High School student Shaun Lee. Michael Herzog was extremely kind in his discussions and suggestions about flash suppression, particularly in facilitating his programs to measure orientation discrimination. He was also key in the genesis of the idea to objectively measure suppression (see Appendix 2). The comparison of different methods of spike sorting (Appendix 1) was carried out by Angela Yu and Jason Davis. Maneesh Sahani and John Pezaris provided their algorithm (spikesort) for comparison purposes. Part of the spiker software developed by Angela Yu and myself was written by Christophe Pouzat. The experiment using repetitive transcranial magnetic stimulation during binocular rivalry (Appendix 3) was carried out in Harvard Medical School in collaboration with Julian Keenan and Alvaro Pascual Leone. Geraint Rees and Nikos Logothetis have been very kind in providing advice for the binocular rivalry and flash suppression experiments (Chapter 6). I would also like to thank a number of people for scientific discussions and for insightful comments to our writings: Mariela Zirlinger, Dan Rozenfarb, Terry Sejnowski and Francis Crick. My thesis committee, conformed by Mark Konishi, Richard Andersen, Shim Shimojo and John Allman (in addition to Itzhak Fried and Christof Koch) has provided valuable feedback. vii The support of my wife Mariela Zirlinger was also key to the completion of all this work. Itzhak Fried is the neurosurgeon at UCLA who has made all of these experiments recording the activity of single neurons possible. He has been a pioneer in the technique of chronic single unit recordings in epileptic patients. His scientific enthusiasm has been highly motivating and encouraging at the same time of helping the patients with their clinical problems. This is remarkable in itself but, in addition to that, Itzhak has proved to be extremely helpful in the design, analysis and discussion of our experiments. Finally, the inspiring driving force behind all the work here described is my advisor Christof Koch. His enthusiasm for research is contagious, imparting the basement in Beckman Institute a strong scientific momentum. Together with Francis Crick, he has had the insight that we can and we should rigorously study the neuronal correlates of consciousness. His curiosity and motivation has pushed the field of consciousness research (among other fields) into the mainstream of neuroscience exploration. He has created an excellent environment to learn and explore new questions. He has made the already interesting adventures of scientific exploration during my doctorate work even more fascinating, enlightening them with new views, new questions, new solutions and new challenges every time. I am immensely grateful to him for allowing me the opportunity to work throughout these years in the lab and for his continuous motivation and encouragement. viii ix Abstract How does the neuronal activity in our brains give rise to our perceptions? We recorded the electrophysiological activity of over one thousand individual neurons in the human brain during object recognition, binocular rivalry, visual imagery and sleep. Subjects were patients with intractable epilepsy implanted with depth electrodes in targets including the amygdala, entorhinal cortex and hippocampus to localize the seizure focus for potential surgical resection. This has allowed us to explore the neuronal responses during visual processing in humans at an unprecedented level of spatial and temporal resolution. We observed a high degree of selectivity in the responses to complex visual stimuli. Some units were selective to categories of pictures including faces, houses, objects, famous people and animals while others responded only to one or a few stimuli, suggesting a sparse representation of visual information in the medial temporal lobe. Most of the selective neurons modulated their responses depending on the subject's percept during flash suppression. To further explore the correlation between perception and neuronal activity we investigated the vivid images that can be voluntarily generated in our minds in the absence of concomitant visual input. Our study revealed neuronal correlates of visual imagery and supports a common substrate for the processing of visual input and recall. Since visual memory is also prominent during dreams, we investigated the neuronal responses during different stages of the sleep-wake cycle. We observed an increase in synchrony during slow wave sleep compared to the wake and rapid-eye- movement sleep states. Our results suggest that neuronal activity in the human medial temporal lobe correlates with perception, shows a strong degree of invariance to changes in the input and could be involved in processing, storing and recalling visual information. x xi Table of Contents Acknowledgments .............................................................................................................. v Abstract.............................................................................................................................. ix Table of Contents...............................................................................................................xi List of Figures.................................................................................................................. xix List of Tables ................................................................................................................. xxiv List of abbreviations ....................................................................................................... xxv 1 Introduction...................................................................................................................... 1 1.1. Informal definitions: statement of the questions ..........................................................................1 1.2. Brief history of theoretical ideas regarding perception ................................................................3 1.3. Why an experimental approach is necessary................................................................................6 1.4. Brief comments about epilepsy ....................................................................................................7