Spatio-Temporal Representations and Analysis of Brain Function from fMRI DISSERTATION Presented in Partial Fulfillment of the Requirements for the Degree Doctor of Philosophy in the Graduate School of The Ohio State University By Firdaus Janoos, B.E., M.S. Graduate Program in Computer Science ***** The Ohio State University 2011 Dissertation Committee: Prof. Raghu Machiraju, PhD., Adviser Prof. Steffen Sammet, M.D.,PhD. Dr. Istvan´ Akos´ Morocz,´ M.D.,PhD. Prof. Michael V. Knopp, M.D.,PhD. Prof. Lee Potter, PhD. © Copyright by Firdaus Janoos 2011 ABSTRACT Understanding the highly complex, spatially distributed and temporally organized phenom- ena entailed by mental processes using functional MRI is an important research problem in cognitive and clinical neuroscience. Classically, the analysis of functional Magnetic Res- onance Imaging (fMRI) has focused either on the creation of static maps localizing the metabolic fingerprints of neural processes or on studying their temporal evolution in a few pre-selected regions in the human brain. However, it is widely acknowledged that cognition recruits the entire brain and that the underlying mental processes are fundamentally spatio- temporal in nature. By neglecting either the temporal dimension or the spatial entirety of brain function, such methods must necessarily compromise on extracting and representing all the information contained in the data. In this thesis, I present new paradigms and an accompanying suite of tools to facilitate a timeresolved exploration of mental processes as captured by fMRI. The first part of the thesis describes a method for visualizing the metabolic activity recorded in the data and a method for studying the timing differences in the recruitment of the different functional modules during a task. In the next part a state-space formalism is used to model the brain transitioning through a sequence of mental states as it solves a task, enabling study of the spatial distribution of activity along with its temporal structure. Efficient algorithms for estimating the parameters, state-sequence and the hemodynamic behavior of the brain have ii been developed. In addition to revealing the mental patterns of an individual subject, such a generative model enables comparing mental processes between subjects in their entirety, not just as spatial activation maps. The methods developed here were applied to fMRI studies for developmental disorders such as dyslexia and dyscalculia (i.e. math learning disability) and for visuo-spatial work- ing memory. I show the types of inferences possible with these methods in analyzing and differentiating mental capabilities and the neuro-scientific conclusions that they provide. iii This thesis is dedicated to my parents for their unconditional love and unswerving support during this long and sometimes arduous journey. iv ACKNOWLEDGMENTS I would like thank Raghu for his many years of guidance – intellectual and philosophical, his pragmatic wisdom, his forbearance, his friendship and his genuine solicitude. I am grateful to Steffen for sharing the deep knowledge of MRI and radiology and for his in- fectious and friendly spirit. I owe special gratitude to Pisti for continuously reminding me to keep the big picture in mind and not developing algorithms for algorithms’ sake, for his wild but brilliant visions, and most of all for his warmth and humanism. Getting through graduate school would have been harder, if it were not for the the support of my friends, who are too many to list here. Among these, I am especially thankful to Kishore, Okan and Shantanu for sharing this experience with me. Most of all, I have to thank Zeenat for giving me the impetus to finally graduate. v VITA 2001 . B.E. Computer Science, University of Pune, India 2009 . M.Sc. Computer Science, Ohio State University, USA 2009–present . PhD Candidate, Ohio State University, USA PUBLICATIONS Research Publications F. Janoos, R. Machiraju and I.A.´ Morocz,´ “Spatio-temporal Models of Cognitive Processes with fMRI,” NeuroImage, In review. T.K. Dey, F. Janoos and J.A. Levine, “Meshing interfaces of multi-label data with Delaunay refinement,” Engineering with Computers, In review. F. Janoos, R. Machiraju, S. Sammet, M.V. Knopp and I.A.´ Morocz,´ “Unsupervised Learn- ing of Brain States from fMRI Data,” Proceedings of 13th International Conference on vi Medical Image Computing and Computer Assisted Intervention (MICCAI), Vol. 6362, 201– 208, 2010. F. Janoos, M.O. Irfanoglu, O. Afacan, R. Machiraju, S.K. Warfield, L.L. Wald and I.A.´ Morocz,´ “Brain State Identification from fMRI Using Unsupervised Learning,” Proceedings of the 16th Annual Meeting of the Organization for Human Brain Mapping (OHBM), 2010. O. Afacan, D.H. Brooks, F. Janoos, W.S. Hoge and I.A.´ Morocz,´ “Multi-shot high-speed 3D-EPI fMRI using GRAPPA and UNFOLD,” Proceedings of the 16th Annual Meeting of the Organization for Human Brain Mapping (OHBM), 2010. C. Lehr, M.O. Irfanoglu, F. Janoos, M.V. Knopp and S. Sammet, “Disease Progression in Multiple Sclerosis: Correlations to Diffusion Tensor Imaging Features,” Proceedings of 18th Annual Meeting of The International Society for Magnetic Resonance in Medicine (ISMRM) , 2010. M.O. Irfanoglu,R. Machiraju, F. Janoos, M.V. Knopp and S. Sammet, “Effect of Gradient Resolution in Diffusion Tensor Imaging on the Appearance of Multiple Sclerosis Lesions at 3T,” Proceedings of 18th Annual Meeting of The International Society for Magnetic Resonance in Medicine (ISMRM) , 2010. F. Janoos, R. Machiraju, S. Sammet, M.V. Knopp, S.K. Warfield and I.A.´ Morocz,´ “Mea- suring Effects of Latency in Brain Activity with fMRI,” Proceedings of IEEE Symposium on Bio-medical Imaging (ISBI), 2010. K. Mosaliganti, F. Janoos, A. Gelas, R. Noche, N. Obholzer, R. Machiraju and S. Megason, “Anisotropic Plate Diffusion Filtering for Detection of Cell Membranes in 3D Microscopy Images,” Proceedings of IEEE Symposium on Bio-medical Imaging (ISBI), 2010. vii F. Janoos, B. Nouansengsy, R. Machiraju, H.W. Shen, S. Sammet, M. Knopp and I.A.´ Morocz,´ “Visual Analysis of Brain Activity from fMRI Data,” Computer Graphics Forum, Vol.28(3), 903-910, June 2009. K. Mosaliganti, F. Janoos, O. Irfanoglu, R. Ridgway, R. Machiraju, K. Huang, J. Saltz, G. Leone and M. Ostrowski, “Tensor Classification of N-point Correlation Function fea- tures for Histology Tissue Segmentation,” Medical Image Analysis, Vol. 13(1), 156–166, Feb. 2009. F. Janoos, K. Mosaliganti, X. Xu, R. Machiraju and S.T.C. Wong, “Robust 3D Reconstruc- tion and Identification of Dendritic Spines from Optical Microscopy Imaging,” Medical Image Analysis, Vol. 13(1), 167–179, Feb. 2009. F. Janoos, B. Nouansengsy, X. Xu, R. Machiraju, K. Huang and S.T.C. Wong, “Classifica- tion and Uncertainty Visualization of Dendritic Spines from Optical Microscopy Imaging,” Computer Graphics Forum, Vol. 27(3), 879– 886, Sep. 2008. K. Mosaliganti, F. Janoos, R. Sharp, R. Ridgway, R. Machiraju, K. Huang, P. Wenzel, A. de Bruin, G. Leone and J. Saltz, “Detection and Visualization of Surface-Pockets to enable Phenotyping Studies,” IEEE Transactions on Medical Imaging, Vol. 26(9), 1283– 90, Sep. 2007. K. Mosaliganti, J. Chen, F. Janoos, R. Machiraju, W. Xia, X. Xu and K. Huang, “Auto- mated Quantification of Colony Growth in Clonogenic Assays,” Proceedings of Medical Image Analysis with Applications in Biology (MIAAB), 2007. F. Janoos, S. Singh, O. Irfanoglu, R. Machiraju and R. Parent, “Activity Analysis Using Spatio-Temporal Trajectory Volumes in Surveillance Applications,” Proceedings of IEEE Symposium on Visual Analytics Science and Technology (VAST), 3–10, Nov. 2007. viii F. Janoos, O. Irfanoglu, K. Mosaliganti, R. Machiraju, K. Huang, P. Wenzel, A. de Bruin and G. Leone, “Histology Image Segmentation using the N-Point Correlation Functions,” Proceedings of IEEE Symposium on Biomedical Imaging (ISBI), 300 – 303, Apr. 2007. K. Mosaliganti, F. Janoos, X. Xu, R. Machiraju, K. Huang and S.T.C. Wong, “Temporal Matching of Dendritic Spines in Confocal Microscopy Images of Neuronal Tissue Sec- tions,” Proceedings of the 9th International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI), 106–113, 2006. F. Janoos, R. Machiraju, R. Parent, J.W. Davis and A. Murray, “Sensor Orientation for Coverage Optimization for Surveillance Applications,” Proceedings of IS&T/SPIE Sympo- sium on Electronic Imaging, vol. 6491, 1–12, Jan 2007. Instructional Publications F. Janoos, R. Machiraju, S. Singh and I.A.´ Morocz,´ “Spatio-temporal Representations and Decoding Cognitive Processes from fMRI,” Ohio State Univ. Tech. Report OSU-CISRC- 9/10-TR19, 2010. F. Janoos, R. Machiraju, S. Sammet, I.A.´ Morocz,´ M.V. Knopp and S.K. Warfield, “Linear Models for fMRI with Varying Hemodynamics,” Ohio State Univ. Tech. Report OSU- CISRC-9/10-TR20, 2010. F. Janoos, O. Irfanolgu, R. Machiraju and I.A.´ Morocz,´ “Visualizing Brain Activity from fMRI Data,” Ohio State Univ. Tech. Report OSU-CISRC-9/10-TR21, 2010. T.K. Dey, F. Janoos and J.A. Levine, “Meshing interfaces of multi-label data with Delaunay refinement,” Ohio State Univ. Tech. Report OSU-CISRC-8/09-TR40, 2008. ix FIELDS OF STUDY Major Field: Computer Science and Engineering Studies in: Medical Image Analysis Prof. Raghu Machiraju Machine Learning Prof. Yoonkyung Lee Computer Graphics Prof. Tamal K. Dey x TABLE OF CONTENTS Page Abstract . ii Dedication . iv Acknowledgments . .v Vita ........................................... vi List of Tables . xvii List of Figures . xviii List of Algorithms . xxi Introduction ......................................1 1 Background of Problem . .1 2 Research Statement . .5 3 Outline of Solution . .6 4 Organization of Thesis . .8 I Background 10 Chapters: 1. Background: fMRI Principles . 11 1.1 Nuclear Magnetic Resonance . 12 1.2 Magnetic Resonance Imaging . 14 1.3 Functional Magnetic Resonance Imaging . 16 1.3.1 The BOLD Contrast . 17 xi 1.3.2 Relationship between BOLD and Physiology . 18 1.3.3 fMRI Noise . 20 2. Background: fMRI Methods . 22 2.1 Neuroscience Principles . 23 2.2 fMRI Methods Taxonomy . 25 2.3 Pre-processing . 25 2.3.1 Motion Correction .