Goal Selection as a Control Strategy in a Brain-Computer Interface A DISSERTATION SUBMITTED TO THE FACULTY OF THE GRADUATE SCHOOL OF THE UNIVERSITY OF MINNESOTA BY Audrey Nicole Smith Royer IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF DOCTOR OF PHILOSOPHY Bin He, Adviser September, 2011 © Audrey S Royer 2011 Acknowledgements I would like to acknowledge the other people who contributed to the works contained in this thesis. My advisor Bin He supported me throughout the process. In section 2.1, Andrew McCullough and Cristina Rios assisted with data collection. In section 2.2, Minn Rose contributed intellectually and helped gather data; I am also grateful to Alex Doud, Han Yuan, Dan Rinker, and Cristina Rios for assistance in data collection and many useful discussions. In chapter 3, Alex Doud and Minn Rose contributed intellectually; I am also grateful to Dan Rinker for data collection assistance, Han Yuan for useful discussions, Keith Jamison for assistance with Matlab, and the Blender community. In chapter 4, I am grateful to Alex Doud, Minn Rose, and Keith Jamison for assistance in data collection and many useful discussions. I would like to thank all my subjects for their time and cooperation. Several groups of people contributed to my overall intellectual development. I would like to thank my thesis committee for their guidance along this path: James Ashe, Geoff Ghose, Sheng He, and David Redish. I would like to thank everyone from the Biomedical Functional Imaging and Neuroengineering Laboratory, both past and present. The many people in lab over time provided intellectually stimulating discussions and technical assistance. i Dedication This dissertation is dedicated to my loving and patient husband, David Royer ii Abstract A brain-computer interface (BCI) translates signals recorded directly from the brain into commands that control an external device, such as a computer cursor, wheelchair, or neuroprosthetic. BCIs promise to help the nearly 6 million people who live with paralysis by allowing them to interact with the world in ways they are no longer able. BCIs can also be used by able bodied individuals to extend their capabilities. BCIs differ widely in how they implement the translation from raw brain signal to device command. Two competing control strategies, goal selection and process control, differ in how much the BCI assists the user. In process control, the user controls every step of the process and receives minimal to no assistance from the system. Other terms for process control include low- level control or continuous control. In goal selection, the user only needs to determine the goal and the system executes the process to achieve that goal. Other terms for goal selection include high-level control or shared control. This thesis presents the first studies directly comparing goal selection and process control. We found in these studies that the goal selection based paradigms were easier to learn, had a decreased training period, and provided improved speed, accuracy, and information transfer in both the simple and more complex applications studied. This thesis also extends our understanding of the neurophysiology while using a sensorimotor rhythm based BCI. When individual trial data were analyzed and not averaged as is typically done in the literature, we found that duration of sensorimotor rhythm modulation was more correlated to successful use than amplitude of modulation. Additionally, we found that correct modulation that led to either a single hit or overall high accuracy was the same between the two control strategies. This shows that the improved performance in these studies while using the goal selection based paradigms was more attributable to the difference in device command instead of the difference in raw brain signal. By understanding neurophysiology and applying that knowledge to BCI design, we can make a better BCI. iii Table of Contents Acknowledgements..................................................................................................i Dedication..................................................................................................................ii Abstract.....................................................................................................................iii Table of Contents......................................................................................................iv List of Tables.............................................................................................................vi List of Figures..........................................................................................................vii Chapter 1 Introduction ............................................................................................... 1 1.1 Brain-Computer Interfaces ............................................................................... 1 1.2 Electroencephalography ................................................................................... 4 1.3 Motor Control and Sensorimotor Rhythms ...................................................... 5 1.4 Goal Selection and Process Control ............................................................... 10 1.5 The Organization of this Thesis ..................................................................... 15 Chapter 2 Goal Selection vs. Process Control in a Simple Task ............................. 17 2.1 Trained and Naive Subjects ........................................................................... 17 2.1.1 Introduction ............................................................................................. 17 2.1.2 Methods ................................................................................................... 19 2.1.3 Results .................................................................................................... 25 2.1.4 Discussion ............................................................................................... 35 2.2 Tracking Naive Subjects through the Learning Process ................................ 39 2.2.1 Introduction ............................................................................................. 39 2.2.2 Methods ................................................................................................... 41 2.2.3 Results ..................................................................................................... 49 2.2.4 Discussion ............................................................................................... 54 2.2.5 Conclusion ............................................................................................... 61 Chapter 3 Goal Selection in a More Complicated Application................................ 62 3.1 Introduction .................................................................................................... 62 3.2 Methods .......................................................................................................... 63 3.3 Results ............................................................................................................ 70 iv 3.4 Discussion ...................................................................................................... 75 3.5 Further Analysis on the Impact of the Cone of Guidance .............................. 78 3.5.1 Introduction ............................................................................................. 78 3.5.2 Methods ................................................................................................... 79 3.5.3 Results ..................................................................................................... 80 3.5.4 Conclusion ............................................................................................... 88 Chapter 4 Analyzing the Underlying Neural Signal ................................................ 90 4.1 Introduction .................................................................................................... 90 4.2 Methods .......................................................................................................... 92 4.3 Results .......................................................................................................... 100 4.4 Discussion .................................................................................................... 110 4.5 Electrodes and Frequencies Not Used for Control ....................................... 115 4.5.1 Introduction ........................................................................................... 115 4.5.2 Methods ................................................................................................. 116 4.5.3 Results ................................................................................................... 116 4.5.4 Discussion ............................................................................................. 120 Chapter 5 Discussion ............................................................................................. 123 References .............................................................................................................. 132 Copyright Permissions ........................................................................................... 139 v List of Tables Table 2.1 Paradigm program parameters ................................................................. 23 Table 2.2 Significant differences for number of hits ............................................... 27