The Pennsylvania State University the Graduate School

The Pennsylvania State University The Graduate School

PERSONALIZED BRAIN-COMPUTER INTERFACES FOR AMYOTROPHIC

LATERAL SCLEROSIS

A Dissertation in Engineering Science and Mechanics by Andrew Geronimo

Submitted in Partial Fulﬁllment of the Requirements for the Degree of

Doctor of Philosophy

May 2015 The dissertation of Andrew Geronimo was reviewed and approved∗ by the following:

Steven J. Schiff Professor of Engineering Science and Mechanics, Neurosurgery, and Physics Dissertation Advisor, Chair of Committee

Zachary Simmons Professor of Neurology and Humanities

Patrick Drew Assistant Professor of Engineering Science and Mechanics and Neurosurgery

Rick Gilmore Associate Professor of Psychology

Bruce J. Gluckman Associate Professor of Engineering Science and Mechanics, Neurosurgery, and Biomedical Engineering

Judith A. Todd Department Head, P. B. Breneman Chair, Professor of Engineering Science and Mechanics

∗Signatures are on ﬁle in the Graduate School. ABSTRACT

Brain-computer interfaces (BCIs) are a potential last line of communication for those in the late stages of amyotrophic lateral sclerosis (ALS). Following the precedent seen in the field of personalized medicine, this thesis proposal focuses on tailoring BCI devices to personal factors which influence disease outcomes. These factors include the physical, cognitive, and behavioral presentations of the patient as well as the contribut- ing genetic factors. It is with this type of patient-centered personalization that I aim to establish and improve communication in a larger portion of BCI users. I show that previously unused features reflecting task vigilance can be used to increase BCI speed in certain individuals. I also show that psychological changes associated with ALS, rather than physical symptoms, can affect the desire and ability to use a BCI communication system. Analysis of BCI data indicates a frontal shift and delayed timing of discriminable features during a P300 task for ALS patients. Furthermore, patients with cognitive impairment uniquely benefit from BCI features capturing functional connectivity compared to the traditional power features used in a motor-imagery task. In light of the methods for personalization defined in this work, I provide outlook on possible avenues for future BCI development, along with some thoughts on the ethical guidelines for implementation of these systems as assistive communication tools.

iii CONTENTS

List of Figures vii

List of Tables ix

Acknowledgments x

Chapter 1 Introduction 1

Chapter 2 Literature Review 4 2.1 Brain-Computer Interfaces ...... 4 2.1.1 Signiﬁcant Early Contributions to the Field ...... 4 2.1.2 Recording methods ...... 8 2.1.3 BCI paradigms and associated brain phenomena ...... 11 2.1.4 The BCI pipeline ...... 16 2.2 Amyotrophic Lateral Sclerosis ...... 24 2.2.1 Signs and Symptoms of ALS ...... 25 2.2.2 Brain Imaging in ALS ...... 27 2.2.3 Assistive and Augmentative Communication in ALS ...... 29 2.3 Unsolved issues with BCIs ...... 30 2.3.1 Asynchronous Use ...... 30 2.3.2 BCI Inefﬁciency and Predictors of Success ...... 32 2.3.3 Limitations of the locked-in ...... 33

iv Chapter 3 Gating mechanisms for BCI 35 3.1 Evoked Responses to Asymmetric Cueing [1] ...... 36 3.2 Use of single trial gating signals to optimize motor-imagery BCI [2] . . 39 3.2.1 Introduction ...... 39 3.2.2 Methods ...... 40 3.2.3 Results ...... 50 3.2.4 Discussion ...... 54

Chapter 4 Personalization - User to System 58 4.1 Desire for BCI use among ALS participants is affected by behavioral health [3] ...... 59 4.1.1 Introduction ...... 59 4.1.2 Methods ...... 60 4.1.3 Results ...... 63 4.1.4 Discussion ...... 68 4.2 Cognitive impairment negatively impacts BCI use [4] ...... 71 4.2.1 Introduction ...... 71 4.2.2 Methods ...... 72 4.2.3 Results ...... 75 4.2.4 Discussion ...... 80 4.3 Repeats of hexanucleotide G4C2 in C9ORF72 correlate with quality of BCI performance [5] ...... 84 4.3.1 Introduction ...... 84 4.3.2 Methods ...... 86 4.3.3 Results ...... 87 4.3.4 Discussion ...... 92

Chapter 5 Personalization - System to User 93 5.1 Feature Optimization ...... 94 5.1.1 Introduction ...... 94 5.1.2 Methods ...... 97 5.1.3 Results ...... 99 5.1.4 Discussion ...... 106 5.2 Features describing neural connectivity for BCI ...... 107 5.2.1 Event-related coherence changes ...... 107 5.2.2 State estimation based on neural ﬁeld modeling ...... 114

v Chapter 6 Conclusions, Future Directions, and Ethical Considerations 130 6.1 Personalized deployment and design of BCI devices in ALS ...... 130 6.1.1 Prospects for gating ...... 130 6.1.2 System Targeting ...... 131 6.1.3 System Personalization ...... 132 6.1.4 Alternative features ...... 134 6.2 Ethical Considerations ...... 136 6.2.1 Making judgments about BCI use ...... 137 6.2.2 Informed Consent ...... 138 6.2.3 Issues arising from continuous BCI use ...... 139

Appendix A Supplementary Materials 141

Bibliography 147

vi LIST OF FIGURES

2.1 Berger’s EEG ...... 5 2.2 The Farwell and Donchin speller ...... 6 2.3 Directional tuning of neurons in the motor cortex ...... 7 2.4 Motor-imagery paradigm ...... 15 2.5 P300 paradigm ...... 17 2.6 EEG artifact examples ...... 19 2.7 EEG reference types ...... 20 2.8 Time- and frequency-based features ...... 22 2.9 Heterogeneity of ALS ...... 26 2.10 Brain changes in the ALS-FTD continuum ...... 28

3.1 Event-related lateralization ...... 38 3.2 Phase and amplitude extraction via Hilbert Transform ...... 43 3.3 Construction of the VEP template ...... 45 3.4 VEP template for all subjects ...... 46 3.5 Flowchart of the gating analysis ...... 49 3.6 Results of gating by right hemisphere variables ...... 51 3.7 Accuracy and bit rates from the gating procedure ...... 54

4.1 ALS clinic study procedures ...... 62 4.2 Initial patient survey responses ...... 65 4.3 Patient BCI requirements ...... 66 4.4 Interest in BCI functions is affected by cognition and behavior . . . . . 66 4.5 Logistic regression model for participation in the BCI pilot study . . . . 67 4.6 Success of BCI use inﬂuences patient opinion on the follow-up survey . 68 4.7 BCI accuracy among patients and controls ...... 77 4.8 P300 quality of ALS patients ...... 78 4.9 Regression of BCI quality on patient factors ...... 79 4.10 Motor-imagery quality of ALS patients ...... 80

vii 4.11 Elevated high-frequency power occurs in cognitively impaired patients . 81 4.12 Older patients perform better with P300 ...... 82 4.13 Histogram of GGGGCC repeat lengths within C9ORF72 ...... 87 4.14 Disease factors do not correlate with sub-threshold expansions . . . . . 87 4.15 Repeat length is inversely related to P300 quality...... 90 4.16 Repeat length is inversely related to motor-imagery quality...... 91

5.1 P300 error among patient participants by classifier complexity . . . . . 100 5.2 P300 error by electrodes used in classification ...... 102 5.3 P300 error generated from reduced electrode sets ...... 103 5.4 P300 error by time points used in classification ...... 103 5.5 Motor-imagery error by electrode, averaged over participant groups . . . 104 5.6 Motor-imagery error generated from reduced electrode sets ...... 106 5.7 Differential mu power between right and left trials ...... 110 5.8 Differential coherence in the mu band between left and right trials . . . 111 5.9 Classification results from power and coherence features ...... 112 5.10 2D representation of field-, state-, and sensor-space ...... 118 5.11 Modifications to the estimation framework ...... 121 5.12 Replication of kernel estimates in the Freestone framework ...... 123 5.13 Reconstruction of the simulated field ...... 124 5.14 Spectral analysis of field reconstruction ...... 124 5.15 Estimation fidelity of the EEG fields ...... 125 5.16 Spatial frequencies in the sensor and basis space for the EEG fields . . . 126 5.17 Estimated kernels from motor-imagery EEG data ...... 127

viii LIST OF TABLES

3.1 Results of the permutation test with the three gating variables ...... 52 3.2 Simulated gating results ...... 53

4.1 ALS patient characteristics ...... 64 4.2 Characteristics of ALS patients and control participants in the BCI study 76

5.1 Parameters used in the neural ﬁeld estimation framework ...... 120

ix ACKNOWLEDGMENTS

I would first like to thank my advisor, Dr. Steven Schiff for his constant encouragement and guidance. I am also indebted to Dr. Zachary Simmons, without whom my past and future work in the ALS Clinic would not be possible. For first recognizing and nurtur- ing my potential, I thank Dr. Mst Kamrunnahar. Thanks to my committee members, Dr. Patrick Drew, Dr. Rick Gilmore, and Dr. Bruce Gluckman. You challenged me to think beyond my own little world and have elevated the quality of this work. Thanks to the nurses, specialists, and staff who welcomed me as a part of the team, especially Travis Haines, Judy Lyter, Heidi Runk, Beth Stephens, and Susan Walsh. To my fellow graduate students who have lent me their heads, both literally and figuratively, you have succeeded in helping me keep my own. This work would not be possible without the following funding sources: NIH grant K25NS061001(Kamrunnahar), the endowment funds of Harvey F. Brush, the Paul and Harriet Campbell Fund for ALS Research, the ALS Association Greater Philadelphia Chapter, and many other private donations to the Penn State Hershey ALS Center.

A special thanks to all the patients who gave of their precious time, your excitement brought meaning and joy to my work.

Last, to my parents and family, even though your eyes glazed over at even the briefest of research descriptions, I owe you all my success.

x CHAPTER 1

INTRODUCTION

“Does the cosmos contain keys for opening my diving bell? A subway line with no terminus? A currency strong enough to buy my freedom back? We must keep looking.”

— Jean-Dominique Bauby, The Diving Bell and the Butterﬂy: A Memoir of Life in Death

In the winter of 1995, French editor Jean-Dominique Bauby suffered a stroke which left him nearly completely paralyzed. Following months of adjustment to the reality of his physical condition, he painstakingly wrote The Diving Bell and the Butterfly with the help of a translator, using only the blink of an eye. Twenty years later, the field of brain-computer interfacing is on the cusp of an answer to the question posed in the final pages of his memoir. As an emerging field of scientific study, brain-computer interface (BCI) research has experienced a divergence of nomenclature during its development, with systems labeled brain-response interfaces, thought-translation devices, brain-machine interfaces, and direct-neural interfaces, to name a few. Although there is no academic society or governing entity which formally organizes BCI researchers, the community set out at the 4th International Workshop on BCI to establish a definition, and collectively agreed on four main points [6]. A BCI:

• Must directly record activity from the brain 2

• Utilize at least one intentionally-modulated brain signal • Processing of this signal must occur in real-time • Feedback must be given to the user to communicate success in the task.

Aside from their important uses as tools of scientific inquiry, the potential applications of BCIs are widespread, encompassing novel forms of biofeedback for purposes ranging from immersive entertainment to vigilance monitoring. Much of the research into BCI technology has been with the goal of enhancing the quality of life for individuals with conditions causing neurological dysfunction by aiding in motor control [7], communication [8] and rehabilitation [9]. The focus of this dissertation will center on the use of BCI for augmentative and alternative (AAC) communication for those with neurological disorder, specifically those with amyotrophic lateral sclerosis (ALS). This disease causes progressive degeneration of motor neurons, leading to muscle weakening, and in some cases, to locked-in syndrome similar to what was experienced by Bauby. The physical limitations of this patient population necessitate the development of a practical tool which can augment communication and mobility. Until very recently, BCI-AAC systems studied in healthy individuals and ALS patients were created with the assumption of homogeneous, strictly motor limitations across users. As one group of researchers note, this assumption couldn’t be further from the truth. “Disease heterogeneity in ALS has multiple dimensions, including genetic origin, site of onset, rate of decline, the presence of cognitive impairment and the relative degree of upper and lower motor neuron involvement” [10]. In light of this, it may be of little surprise that ALS patients continue to under-perform in BCI compared to neurologically healthy individuals, and in critical situations often achieve no control whatsoever. As will be described in this thesis, nothing short of a patient-specific approach to BCI design will be suitable for achieving robust control in the majority of patients. In the following chapters, I will describe how the heterogeneity observed in ALS patients describes a pattern of device utility. Even more critically, I outline methods to personalize the BCI system to account for specific motor and non-motor limitations of the disease. I do this because I hope to see BCI technology become a valuable and effective means of communication for those living with ALS, and hope to implement personalized BCI methods into device design in the future. 3

The following work builds upon the successes of scalp-based BCI communication discussed in Chapter 2, while addressing the unique challenges to deployment faced by the ALS population. Early work in my thesis focused on BCI device personalization applied to healthy, young individuals (Chapter 3), while my later work focused on the challenges of patient-to-device and device-to-patient personalization for ALS users (Chapters 4 & 5). The ﬁndings emerging from these studies are presented in short below.

• Previously unused brain signatures can be employed as gating mechanisms as part of a hybrid BCI, in order to increase the accuracy and speed of the BCI system by identifying periods of task vigilance and cortical excitability.

• Behavioral abnormalities occurring in some ALS patients reduce their interest in BCI adoption.

• Cognitive deﬁciencies associated with ALS can lead to poor performance in standard BCI protocols. This poses a clinical challenge, as patients with cognitive deﬁcits are just as eager to use a BCI for communication.

• Preliminary evidence exists that genetic screening of the ALS-linked C9ORF72 genetic expansion could form a predictor of BCI system success.

• Patterns of feature extraction differ between control and patient participants on two BCI tasks. Furthermore, we show how patterns of control signals diverge for ALS patients who experience cognitive impairment.

The ﬁnal chapter addresses some of the future questions and design implementations that make up the next step in this line of work. Furthermore, a portion of the discussion is dedicated to ethical matters pertaining to my present and future work in this ﬁeld. CHAPTER 2

LITERATURE REVIEW

2.1 Brain-Computer Interfaces

2.1.1 Signiﬁcant Early Contributions to the Field

Although the field of BCI is relatively new, the neuroscientific and engineering discoveries that preceded its definition have a much longer history. The contributions to the field of BCI, which have grown at an increasing rate since the 1990’s, rise from a foundation built in electroencephalography, neurophysiology, and computer science. Comprehen- sive reviews on the discoveries which preceded the technical and conceptual foundation of BCIs are available [8, 11]. Here I overview some of the work that has gone into the development of tools for studying the electrical phenomena of the brain at the level of the scalp, the definition and refinement of paradigms used to infer information about a mental state from neural activity, and the progress in computational and statistical methods for making these inferences. By the 19th century, there was a growing body of knowledge about the electrical impulses associated with living tissue, and electrical studies of the brain had been conducted in a number of animal species. Hans Berger, a German neuropsychiatrist, is credited as the first to document the nature of the human electroencephalogram (EEG) in 1929 [12]. Berger used a galvanometer in his early studies, more advanced than those used by his predecessors, which allowed him more sensitivity and bandwidth in 5 his recordings. Although he is most famously credited for showing the occipital alpha rhythm (Figure 2.1) and blocking of this rhythm, he would also describe the EEG characteristics of sleep, consciousness, hypoxia, and various brain disorders in human subjects [12, 13]. A substantial review is given on the developments of EEG that occurred throughout the world after the popularization of the technique by Berger [12]. By the 1950s, EEG was a concept well known in medicine and the academic world; nearly all university hospitals had at least one EEG machine [12]. During this time, EEG flour- ished as a tool used to study epileptic foci, the neurological correlates of sleep, analysis of evoked potentials, as well as leading to the development of more invasive bioelectrical sensors. In the last 60 years, the clinical use of EEG has diminished due to the avail- ability of superior non-invasive techniques for structural and functional brain imaging, but researchers still employ EEG for its applicability to human subjects and desirable temporal resolution, in order to related the electrical activity of the brain to the complex human behavior. The birth of proper BCI research likely rose out of the study of operant conditioning, where the voluntary regulation of EEG characteristics in the presence of feedback had been explored by researchers since the 1960’s. This type of biofeedback was subsequently realized as a potential BCI by Birbaumer and colleagues [11]. His group showed that slow cortical potentials (SCPs) were able to be trained by subjects to be increased or reduced at will, providing at least rudimentary brain control of a computer system [14]. Around a similar time, a separate class BCIs emerged, which were based on the user’s innate response to rare visual stimuli. Farwell and Donchin created the now famous and perhaps most widely successful system, the P300 speller, (Figure 2.2) [15] which utilized the electrical potential evoked by rare target stimuli to identify the user’s letter of intent. This system achieved an increased range of outputs for the user at a faster speed. It represented a dramatic increase in the usability of such a system for a broader

Figure 2.1: Berger’s recording of the electroencephalogram, along with a 10 Hz sine wave reference [12]. 6 spectrum of users, without the need for a lengthy conditioning process. A third system based on the imagination of motor movements was pioneered primarily in the work of Pfurtscheller and Aranibar, who identified the suppression of cortical rhythms that accompanied self paced movements [16]. From these observations, BCI systems based on the imagination of movement were also developed, functioning on a similar principal of cortical desynchronization. Unlike the SCP systems before it, motor-imagery BCIs have been shown to be capable of producing multidimensional control through the use of different mental imageries. Other advances in recording technologies and computation have contributed significantly to the field. Berger performed his recordings using a single pair of electrodes connected to a galvanometer, recording on photographic paper [12]. The sophistication of our tools today allow us to process enormous amounts of data from sensors that provide an image of the brain in far greater detail. Indeed, Figure 2.2: The speller matrix used to evoke some of the most fantastic results come P300 potentials for communication in the from intracranial BCI systems. In 1982, first of its type BCI [15]. Georgopoulos et al. published a landmark study in the field of single unit recordings, showing that primary motor cortex neurons are tuned preferentially to movement in one direction (Figure 2.3, [17]). Their recordings from the motor cortex of the monkey showed that direction of movement was coded in these individual cells, and movement trajectories could be reconstructed from their firing rates alone. They later showed that the population coding of many neurons in the motor cortex were responsible for the range of movements in 3-D space [18]. The work of Nicolelis et al. expanded on this work by designing a robotic appendage that could be controlled purely from recording direction sensitive neurons in the motor cortex, and that this control existed with and without the presence of overt movement [19]. This phenomenon, which was found to be true in humans as well, along with the field of neural spike encoding/decoding, contributed making invasive BCI techniques at the state-of-the-art for brain-controlled prosthetic development. 7

Figure 2.3: Georgopoulos showed that a single neuron in the motor cortex of the monkey have a preferred ﬁring direction [17]. In the case of the neuron shown here, the highest ﬁring rates are when the monkey moves its arm between 90 and 210 degrees.

Besides some notable exceptions, such as the work at the BrainGate project [20] where microelectrode arrays are implanted into the brain, most invasive studies in humans use subdural grids which record the electrocorticogram (ECoG). This was ﬁrst achieved in a closed-loop paradigm by Leuthardt et al. [21]. ECoG currently holds much of the potential of the BCI ﬁeld as it delivers improved spatial resolution and signal quality of underlying cortical populations compared to EEG while also being relatively safer to implement than more invasive methods. However, at this point the majority of subjects participating in ECoG studies are those for which a grid is being implanted for epilepsy localization purposes. 8

2.1.2 Recording methods

Electroencephalography

BCIs use a number of methods to record the activity of the brain. The earliest described and most commonly used still is the EEG, which is a measure of electrical potential of the brain sensed by electrodes placed on the scalp. The macroscale electrical potentials measured by EEG are generated at the cellular level by the transfer of ions in and out of neurons and glia. Specifically, synchronous dendritic current flows of millions of cells having similar columnar organization are shown to contribute to macroscale electrical phenomena [22]. Additionally, potassium-mediated current flows associated with glial depolarization may serve to amplify this effect [12]. The currents associated with these membrane depolarizations that flow through the extracellular space generate field potentials, and the time course on which they occur dictate the frequencies observed in EEG. For groups of directional neurons, for example in layer V of the cortex, the extracellular currents along the main axes of the neurons sums to the generation of a dipole layer. These dipoles are mesosource generators of electrical potentials, which undergo some low-pass transformation by a conductive medium [22]. These types of large sum- mations of synchronous potentials are the type that can be measured at a distance of an EEG scalp electrode [12]. Various factors contribute to the generation of EEG functional dipoles in the cortex. Brain tissue is inhomogeneous, with conductance varying by location. It is also anisotropic, especially white matter tracts. The skull, which possesses both of these properties, has a major effect on the volume conduction of the EEG. An inner layer of conductive cancellous bone shunts current due to the greater resistance of the outer layer of cortical bone [22]. As a result, a single scalp electrode measures the neural activity from tissue masses containing 108 109 neurons [22]. − The EEG is a measure of voltage, determined as the potential difference between an active electrode and a reference. This means that all EEG recordings are bipolar, however, EEG can be recorded using a number of choices for the reference electrode. These references can be determined by physical location, or virtually by subtraction. Linked-ear or linked-mastoid references are popular choices for physical references, as they are as electrically distant as can be achieved on the head, but suffer a critical drawback. The actual reference provided by these linked electrodes varies depending 9 on the differential resistance at their interfaces [22]. If the resistances are equal, the reference is average potential between the ears, but if the resistance at one ear is much higher, the potential will be heavily weighted towards the other ear. Additionally, this linkage can cause current flows between ears, leading to electrochemical potentials at the electrode interface that are detrimental to recordings.An ideal reference is dependent on the task being studied, and it should be electrically distant from the generators involved in the task. The test for a good reference is if the EEG dynamics of interest remain unchanged if the reference is moved. This means that the source of the dynamics of interest are sufficiently far from the reference to not be affected by its exact placement [22].

Magnetoencephalography

The magnetoencephalogram (MEG) is the electromagnetic complement to EEG, and also possesses a substantial history as a tool for neuroscience research. Superconducting quantum inference devices (SQUIDs) were developed as magnetic sensors in the 1960’s. Later that same decade MEG recording with SQUID technology was demonstrated by Cohen et al., and is currently is the most widely used application of this technology for biosensing, although magnetocardiography and low-field magnetic resonance imaging are active areas of research [23]. As part of Cohen’s characterization of MEG, he and his colleagues performed both theoretical and experimental demonstrations of the differences between magnetic and electrical recordings of the brain. [24, 25]. Using both theory and data from evoked potential studies, they showed three fundamental properties of MEG: the MEG lead field pattern is rotated 90 degrees with respect to EEG, MEG displays slightly better localization of sources due to insensitivity of volume current conduction by the skull, and there is asymmetry in the EEG field due to the radial component of the dipole from which MEG is unaffected [25]. Somewhat of a paradox has existed in MEG research concerning the usefulness of simultaneous MEG and EEG. Malmivuo describes the concept in terms of Hemholtz’s Theorem [26], which describes a general electric field as the sum of two vector fields: the flow source and the vortex source. For this type of recording, we measure these sources as the electric and magnetic fields from the brain. For each of these fields, there are three orthogonal lead field components (sensitivity to electromagnetic fields) which are independent of each other. It is this independence which has led researchers 10 to believe that the addition of magnetic recordings could provide new information not found in EEG. While each component is independent, the signal recorded by them is not since they originate from the same volume source. Therefore, the independence of sensitivity distributions of the two methods does not imply the independence of recorded signals. Even in light of this, a common finding that has been echoed with a number of researchers is that the information provided by simultaneous EEG and MEG recordings is complementary for source localization [27, 28, 29, 30]. One reason for the complementary nature of the two methods is that the sensitivities of the lead fields are different for the two recording methods [26]. This can arise from heterogeneities in the volume conductor [28]. MEG selectively identifies tangential sources, or those arising from cortical infolds [22, 26], which can be used to identify source topology of, for example, photic alpha entrainment as having significant tangential generators [31]. EEG is more sensitive to deeper sources within the brain, while MEG is better at localizing sources closer to the surface [32], and is less susceptible to volume conduction effects or skull deformities than EEG [22, 33]. There have been a few BCI studies which employ MEG. While useful as a tool to probe brain activity, MEG is mainly hindered by its large size and cost, and special considerations that accompany the used of super-cooled magnets. MEG has been used in a motor-imagery paradigm to communicate binary decisions using modulation of sensorimotor rhythms [34]. MEG was used in the same way to decode hand movement in four directions [35]. Using simultaneously recorded EEG, it was shown that MEG was superior at distinguishing between the four states. MEG has been combined with source localization techniques during a visual evoked potential-based BCI to demonstrate how cortical generator locations change with stimulation frequency [36].

Other Methods

In addition to macroscale measurements of electromagnetic brain phenomena, BCIs have also been implemented with fine-scale electromagnetic measurements, as well as metabolic correlates of neural activity. Invasive electrical measures include ECoG [37] arrays and implanted depth electrodes [38]. The former measures the local potentials of a large population of neurons, while the latter measures the multi-unit spiking activity of neurons in the proximity of the electrode. The potential for technologies which achieve 11 the recording quality of surface electrodes without the risk of invasive brain surgery are obviously needed. A potential candidate is the epidural screw electrode, which is implanted within the skull, a source of major information loss in electrical brain recordings, but without penetrating the dura mater, and thus avoiding the complications of infection and replacement [39, 40]. Metabolic BCIs allow for the indirect assessment of neural activity, usually through the measurement of blood flow. Functional magnetic resonance imaging (fMRI) [41] records a blood oxygenation level-dependent (BOLD) response which is a measure of oxygen saturation in brain tissue that can be correlated with neural activity. Near infrared spectroscopy (NIRS) [42] uses an optical signal to measure the concentration of oxygenated hemoglobin in the blood, also signaling the metabolic demands of active tissue. Although these technologies will not be covered in further detail here, their contributions to the field have been substantial in many ways. Invasive recordings have enabled the longest-running applications of continuous BCI use [38], and allowed for control at the finest levels of detail, down to individual finger movement [43]. Metabolic BCIs have allowed for communication to be established in advanced stages of neuromuscular paralysis, where electrical BCIs have so far failed [44]. These technologies certainly help augment our understanding brain phenomena, however EEG remains the most widely used modality because of its high temporal resolution, its ease of use, and its knowledge base encompassing over 90 years of theoretical and experimental work.

2.1.3 BCI paradigms and associated brain phenomena

Types of BCIs

Over the relatively short history of brain-computer interaction, a number of paradigms have been shown to be successful in extracting information of intent directly from neural activity. Here an overview of the paradigms is given, with focus on two such mechanisms which are used in this research, the P300 and motor-imagery BCI paradigms. BCI paradigms can most generally be distinguished by the type of interaction the user has with the system. Three general categories of BCI systems are those which rely on active, reactive, or passive mechanisms for control [45]. Active BCI paradigms are those which rely on the self-modulation of brain signal in a process that is completely 12 endogenous; the user is capable of generating the control signals without external cuing or stimulation by the computer. Active approaches include all forms of mental imagery, of which motor imagery, spatial navigation, mental arithmetic, and verbal imagery represent some which have been used successfully. Slow cortical potentials fall into this category as well [14], in which users are trained through feedback conditioning to generate long bouts of cortical depression to facilitate binary communication. Reactive BCIs rely on the user to respond to a stimulus from the computer. The control signals generated as part of a reactive BCI would not be possible without the stimulus provided by the computer, which can be auditory, tactile, or visual. Perhaps the most successful BCI device to date is the P300 speller, an example of a reactive system because user delivers their intent by reacting to one of many target stimuli, dif- ferentiated by the presence of an evoked brain potential. Another visual BCI in this group is the steady-state visually evoked potential (SSVEP) BCI, which presents the user with a number of stimuli ﬂickering at different frequencies [46]. The user delivers intent by reacting selectively, in this case focusing their attention, on the stimulus which corresponds to that intent. Reactive BCIs can also be facilitated with other sensory modalities such as somatosensation and audition [47, 48]. The last category is the passive BCI, which is completely absent of voluntary human control. Although there is some controversy around ‘diluting’ the deﬁnition of BCI with the inclusion of devices that serve as more or less cognitive state monitors [49], the inclusion of passive monitoring, especially as components of a ‘hybrid’ BCI system, has gained momentum [6].

Motor-imagery

Imagery is a widely used tool in BCI because it requires no overt movements, and generates a distinct pattern of cortical activation which can be discriminated as focal deviations in the EEG. Various types of mental imagery have been used in BCI control, including spatial navigation, mental rotation, mental arithmetic, word imagery, auditory imagery, and motor imagery [50, 51, 52, 53]. Although these studies point to a number of imagery techniques, motor imagery has received the most attention as a mental strategy for achieving communication of voluntary intent through a BCI. This may have been because this type of imagery provides high inter-session stability [51], or simply because of the straightforward implementation of imagery as a possible aid for motor 13 system dysfunction. The performance of motor imagery is not an intuitive task; in most individuals, it is a skill that requires substantial practice. Learning how to control these signals is similar to other physical skills. By using a combination of practice and feedback, users are able to modulate brain activity effectively. Kinesthetic, or first-person imagery of motor action, rather than visual, or third-person imagery, has been shown to be more effective at enhancing cortical excitability [54] and better for generating desynchronization over motor regions of the brain [55]. The imagination of motor action is not only useful in BCI research, but also helps to determine efficacy of rehabilitation of upper extremities. Various tests and revisions on these tests have been developed to assess the ability to perform both visual and kinesthetic imagery [56, 57]. There is a substantial literature unifying the phenomena of motor execution, motor imagery, and motor observation through anatomical, functional, and lesional studies. Jeannerod proposes a “unifying mechanism for motor cognition”, which he states imagery and action observation provide a neural simulation for motor action [58]. Bol- stered mainly by fMRI evidence, there is large overlap in the activated motor brain regions involved in all three motor states, although, understandably, there are differences the magnitude and topography of those tasks, as well as patterns of functional connectivity. [58, 59]. Those who are trained and skilled performers of motor imagery, for example, show greater activation of pre-motor areas, a locus of motor planning [60, 61]. A typical estimate is that imagery produces about 30% of the intensity of overt action as recorded by fMRI [62]. The brain circuitry involved in coordinated motor actions and imagery originate at the level of the neuron, although the large scale fronto-parietal circuitry that contributes to the generation of oscillations are more relevant to EEG-based recording. These brain oscillations that are related topographically and functionally to motor representation are termed the sensorimotor rhythms (SMR). Although they encompass multiple frequency bands , the first discovered in 1952 by Gastaut [63] was central or Rolandic mu rhythm, which oscillates at 8-13 Hz and decreases in the presence of movement. This rhythm is related in to the occipital alpha because of its similar rate (8-13 Hz) and its classical interpretation as an idling rhythm [64]. However, the mu rhythm has been shown to be anatomically [64] and functionally [65] distinct from the occipital alpha rhythm. Thalamo-cortical loops are involved in the generation of mu rhythms in cats [66, 67], 14 and at least for its occipital counterpart, the alpha rhythm, this functional circuit exists in humans as well [68]. The loop receives additional inputs from secondary motor areas, including the basal ganglia and cerebellum [69]. In the case of motor imagery, there is an inhibitory influence from the supplementary motor area in modulating mu power changes [69, 70]. Mu rhythms are popularly employed in BCI because they are of high amplitude and easy to resolve with EEG recordings. However, the activity in higher frequency bands such as beta (18-25 Hz) and gamma (30-100 Hz) have been implicated in EEG, ECoG, and intracranial recordings to involve localized processing [71, 72] and are causal for widespread changes in mu activity [73]. An example of a typical center-out motor imagery task is shown in Figure 2.4. The mechanism for control in this task involves the depression of mu and beta rhythms when motor-imagery is performed, a phenomenon termed event-related desynchronization (ERD) [74]. This term was chosen because the oscillations of EEG represent postsynaptic firing in a large group of neurons [75]. ERD is interpreted as the electrophysiological result of activated cortical areas becoming involved in the initiation of a motor behavior. Factors such as task complexity, attention, and performance efficiency contribute to the amplitude and spread of desynchronization [76]. This blocking begins to occur during the planning stages of motor action, and manifests shortly before the action is performed [64, 76]. The patterns of desynchronization are somatotopically localized to the representation areas of the imagined limb, especially in the upper frequency bands known to reflect the activations of smaller scale networks. These power changes at the scalp are in line with intracranial recordings [72]. The signals of interest in an imagery task are the topographical differences in SMR desynchronization, which are calculated commonly through a frequency-based method. The differences in SMR power over the pre-motor and motor cortices enable classification of covert motor intent into a command signal [74].

The P300

The P300 is a positive EEG evoked potential that occurs around 300 ms in response to a novel, infrequent, or unexpected stimulus, commonly called the ‘oddball’ response. The P300 is typically recorded over the medial centro-parietal cortex, peaking anywhere from 300-900 ms after the stimulus, with amplitude proportional to the rarity of the stimulus [77]. A theory for P300 generation centers on context updating [78]. According to 15

-2 ) 2 V

-4 µ Time

-6 Power(

-8

-10

-12

-14

60 ) 2 V

µ 40

20 Power( 0 0 10 20 30 40 50 60 Frequency (Hz)

Figure 2.4: (Top) A two-class center out motor imagery paradigm, in which the user controls the cursor (ball) to either a left or right target (box) on the screen. (Middle) Spectrogram of channel C4 during a motor imagery run, with the timing of left and right imagery trials marked on the side in blue and red. Each marker corresponds to a three-second period of imagery. (Bottom) The average of the spectra over all left and right time windows. Discrimination between the two tasks is evident in the mu and beta bands. this theory, a change to stimulus attributes act on attentional and memory processes to update the context of the stimulus while producing the P300 evoked potential. Without this detection of change, only exogenous sensory evoked potentials are generated as a 16 result of the stimulus. There are diverse structures implicated in the generation of the subcomponents of the P300 [77]. However, there are some commonalities found between intracranial recordings, combined fMRI/EEG, and lesion studies. Involvement of the temporo-parietal junction implicates a circuit pathway between frontal and temporal/parietal regions of the brain for generation of this response. [78]. Distinct varieties of the P300 have been demonstrated, with the component most relevant to the oddball response inherent in the P300 spelling task, being the P3b potential. This specific potential is characterized by its maximal response in parietal cortex and occurrence that has slightly greater latency than the anterior P3a evoked potential [78]. For the P3b, the actions of the posterior parietal cortex also factor in as both an integrator of visuomotor inputs as well as processing goal-directed attention [77]. The P300 speller, which was first developed in the late 1980s employs this covert marker for selective attention to establish a communication channel directly between brain and machine. The system relies on the user to react to a specific letter of intent out of a larger group of randomly flashing letters in a way that generates a P300 only for the intended letter [79]. Usually, these devices are implemented by arranging the alphabet in a grid on a computer screen, and flashing the rows and columns of the grid in a random pattern. With this type of system, a letter is decoded as the one at the intersection of the row and column which elicit the largest P300 response. Other stimulation methods include flashing the letters in randomized groups in order to limit adjacency errors (Figure 2.5), [80]. The process of P300 classification generally requires the averaging of multiple trials, and there is a trade off between device accuracy and speed. Of current BCIs available, these devices perform the best in terms of accuracy and information transfer, being able to deliver nearly 100% accuracy and 20 bits of information transferred per minute [80].

2.1.4 The BCI pipeline

The signals generated by active or reactive brain modulation comprise only half of a functional brain-computer interface. The computation performed on these signals, namely the pre-processing, feature extraction, classiﬁcation, and feedback are critical to the function of the system. Perhaps most of the effort that has gone into improving BCI technology has centered on methods of computation for brain signal translation. Of 17

0 500 Time after cue (ms)

Figure 2.5: Top: A checkerboard P300 speller, with four letters highlighted at once. The user is focusing on the letter ‘A’. Bottom: Representative EEG signals from one individual. The blue line is the average EEG following the target letters, and the red line is the average EEG following the non-target letters. The P300 is best observed in Cz (with the axis markers), although it can be seen in many central and parietal channels. course, different feature extraction methods and classiﬁcation techniques are used for each type of biosignal and each type of paradigm. Extensive reviews are given on these 18 elsewhere [81, 82, 83], but I would be remiss to fail to give at least an overview of the major trends in signal processing that have accompanied BCI research.

EEG preprocessing

Some recording methods are more sensitive to artifacts than others, but all experience at least some infiltration by signals not relevant to the task. These can include muscular, ocular, respiratory, and cardiac artifacts, as well as line noise from nearby electronic devices running on 60 Hz alternating current power. Two common examples of jaw and ocular artifact are displayed in Figure 2.6. Often the first step in feature extraction is to remove or reduce these artifacts from the raw data before proceeding. Line noise is often removed during recording through use of digital bandpass filters implemented within the amplifier. Most brain signals useful for EEG-based BCI are below the 60 Hz range of line noise, so the filtering of these signals yields minimal loss of information. For addressing muscular and ocular signals that may appear in the recording, additional electromyogram (EMG) or electrooculogram (EOG) electrodes may be integrated into the system. This way, trials having these artifacts can simply be removed from further steps, or can be used to create spatial filters which reduce artifact contamination [84, 85]. Although respiratory and cardiac signals may be picked up by certain EEG leads, these artifacts are much smaller if present at all, predictable, and can also be removed using spatial filters. As mentioned earlier in the introduction, each EEG channel is the bipolar voltage difference between the active and reference electrode. Often, the reference electrode is chosen to be in an inactive region, but the reference may be recalculated later in order to isolate signals of interest more effectively. These virtual, as opposed to physical, references are generated to create reference-free or more localized recordings of neural activity (Figure 2.7). The common average reference (CAR) theoretically provides a reference-free basis by subtracting the average of all electrode potentials. Justifica- tion for this relies on the assumption that the recording electrodes comprise a surface containing all brain currents within a volume, which, for all but the densest electrode arrays, is an assumption that is generally not met. Other references, such as the bipolar or Laplacian reference, utilize pairs of closely separated electrodes to estimate with higher resolution the local potential gradient in the direction between the electrodes [22]. The surface Laplacian, or second spatial derivative of scalp potential, estimates the current 19

lEOG rEOG cEOG Fp1 Fp2

Fz F3 F4 F7 F8 V µ 100 100

T7 C3 Cz C4 T8

P7 P8 P3 P4 Pz

O1 O2

122 123 124 125 126 127 128 129 130 Time (seconds)

Figure 2.6: An example EEG trace corrupted by two types of muscular artifacts. The jaw clench occurs between second 124 and 125, and is best isolated in temporal channels, although it can be seen in most channels. Eye blinks, on the other hand, are best viewed in frontal channels (blue) and can be isolated with proper referencing. density exiting through a portion of the skull. This band-pass filter is used to empha- size local sources and suppress others. Whichever referencing scheme is utilized, care should be taken in the interpretation of data, as the sensitivity to source location and orientation is affected by this choice [86] Additional preprocessing methods are often used in bioelectric recordings to isolate data of interest before performing feature extraction and classification. Such algorithms include spatial filters, component maps, and dimensionality reduction algorithms [82]. Such preprocessing can serve a few purposes. The Independent Component Analysis (ICA) spatial filter, for example, serves to separate multichannel EEG into statistically independent components [87]. ICA is commonly used to perform artifact rejection; one could separate out an eye blink signal, discard it, and remix the components back together to achieve eye blink-free data [88]. Another useful feature of a spatial filter is for dimensionality reduction, especially for systems intended for online operation. Princi- pal Component Analysis (PCA) is a common tool used in signal processing, as it uses the covariance structure of a multivariate data set to find a set of orthogonal components projected on a principal subspace [89]. Using PCA, one can extract a few principal components which captures majority of the variance in the data, thereby reducing the required processing with minimal loss of information. Related to PCA is the Common 20

(a) 40 µ V

2.5 6 (c) 2 (b) 4 ) V) 2 µ V

µ 1.5 2

1 0 Power ( Power

0.5 ( Amplitude -2

0 -4 0 20 40 60 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Frequency (Hz) Time after cue (seconds)

Figure 2.7: (a) An example EEG trace of channel P3 colored by the type of reference used to generate it. Black – Unipolar (ear referenced). Blue – Common average reference. Red – Laplacian reference. (b) Examples of power spectra in channel C4 using the three referencing schemes, and (c), the average visually evoked potential in P3 over one run. Visual cueing stimulus occurs at time zero.

Spatial Patterns (CSP) algorithm, which is used to ﬁnd the maximally different proportions of the combined variances of two sets of EEG data, e.g. left and right motor imageries [90].

Feature Extraction

A feature is the piece of information that is extracted from the brain signal to be interpreted by the classifier. The process can be very straightforward or highly technical, however the purpose of feature extraction is to isolate a relevant signal of interest to the task, so that a lower dimensional signal can be handed off to the classifier. The dimensionality reduction accomplished by feature selection serves two purposes: it makes the classifier more generalizable by not over fitting the highly dimensional training data, and it allows for faster online computations, especially when using complex classification schemes. The nature of features used in BCI control are highly dependent on the recording modality and paradigm; features extracted in an EEG-based motor imagery task bear little resemblance to the neuronal firing rate codes that are used for control in the same task using intracranial recordings. For EEG-based BCI implementations, where signals 21 of interest are transient changes in ongoing oscillations, both time- and frequency-based features may be extracted (Figure 2.8). Substantial reviews have been compiled that pick apart the intricacies of feature calculation and selection methods [82, 83, 91]. Many of the methods, especially those related to motor imagery, rely on spectral analysis methods. These are used when the information of interest is contained in non- phase locked oscillatory changes, which can be lost by averaging trials in the time domain. Spectral analysis requires signal transformation into the frequency domain, using methods such as the Fourier and Wavelet transforms [92]. Rapid versions of these algorithms allow for time-frequency decomposition of online data, to allow for extraction of frequency and power in real time. Features in the time domain are also useful for many BCI applications. A natural example is the extraction of evoked potential features using a system such as a P300 speller, in which case the signals of interest are timed to a specific stimulus. Simple averaging in the time domain allows for noise to be suppressed, increasing the quality of the control signal. Following averaging, features utilized have been as simple as peak amplitude or area under the curve during a period of the evoked potential. Another useful time-domain method is the extraction of band powers from motor imagery data. Band power calculations first involve filtering in the relevant frequency band and then estimating the envelope of the square of that signal to determine the approximate power of that frequency component over time [76]. This method allows for visualization of the changes in power in the sensorimotor rhythms due to movement imagination, and underlies the analysis for calculations of ERD, which is defined as deviation from baseline band power [76]. In addition to features computed directly from the time and frequency representa- tions of individual channels, other methods compute statistics about the interactions between channels, either through cross-correlations or coherences [93, 94]. Additionally, random process models have been applied to EEG data in order estimate parameters of best fit that can be used to in classification [95].

Classiﬁcation

Classiﬁcation is at the heart of the brain-computer interface. This process allows the computer to make decisions about the intent of the user based on the available data. As with feature extraction, classiﬁcation techniques are widely varying and data-dependent, 22

50 (a) V) µ

0 Amplitude ( Amplitude

-50 41 42 43 44 45 46 Time (seconds)

20 (b)

V) 10 µ

0 Amplitude ( Amplitude -10

142 144 146 148 150 152 Time (seconds)

40 50 Target Left (c) non-Target 40 Right ) 30 ) (d) 2 2 30 20 20

PSD(/muV 10 PSD(/muV 10

0 0 0 10 20 30 40 50 60 0 10 20 30 40 50 60 Frequency (Hz) Frequency (Hz)

Figure 2.8: (a) Time-based features used in a P300 spelling task. Black line is a portion of EEG from channel Cz, red and black patches are search windows for maximum EEG amplitude for target and non-target trials. Asterisks mark the maximum amplitudes in those windows. (b) Band power features for motor imagery data during left (blue patch) and right (red patch) imagery periods. The band power is calculated by filtering the EEG signal in channel C4, squaring it, and then calculating the envelope of this signal. This provides an estimate of the time-varying power in that frequency. (c&d) Frequency-based features for the P300 and motor-imagery tasks in the same set of channels. Discriminable features are identified by differential power levels or frequency bands between classes. with both simple and complex implementations [82, 83]. Broad categories of classifiers emerge from the literature: those based on thresholds, generation of linear and nonlinear decision boundaries, neural networks, clustering algorithms, and eclectic variations on Bayesian inference. Linear discrimination methods are by far the most popular and have received attention due to their success in a field-wide BCI algorithm competition [96]. These include linear discriminant analysis, and linear support vector machine algorithms. Both of these algorithms utilize the covariance structure of a set of labeled training data in order to build a decision boundary that maximizes the between-to-within group variance or 23 the maximum margin of separation between classes, respectively. Classifiers can operate continuously, as in the case of a motor-imagery system for driving limb prostheses, or they can operate discretely, making a decision every few seconds after sufficient data has accumulated, as in the case of the P300 speller. In the case of the former, the classifier has to choose from a small set of outcomes, in the simplest case to drive a prosthetic to the left or right. On the other hand, a P300 classifier is usually making a decision from thirty or more options. Classifiers can be built from a dedicated set of training data recorded on the same day or even from a previous session. Alternatively, adaptive methods allow for immediate classification as trials become available [97, 98]

Feedback

A final and necessary component of the BCI system is the generation of feedback. Feed- back allows the user to be aware of their success using the system and, if necessary, how to adapt to improve their performance. The most common form of feedback is a visual output of the decision of the classifier on the computer screen, although other forms of feedback have been used. Feedback modality is especially important for users with compromised sensation. For example, users unable to focus on a computer screen may be better served receiving cuing signals and feedback by sound [99, 100]. Alternatively, those with spinal cord injury, who have motor and sensory limitations of the limbs, may not be able to use a tactile feedback system as well as someone with a primarily motor disorder. These methods of interfacing are suited for different types of applications. P300 are more suited for spelling and environmental manipulation though some form of aug- mented reality interface, and they are more appropriate for discrete multi-class decision making. This type of device has proven its usefulness in clinical applications, for both patients with paralysis, degenerative disease, and disorders of consciousness [101, 102]. These systems are implemented communication platforms, which use the classification of the user’s brain signals to communicate words or actions to an effector, which can be a text document for composing an email [101], or a controller for a wheelchair [103]. On the other hand, the active modulation of an SMR BCI has more appropriate applications for continuous, graded control of low dimensional systems. With sufficient training, motor imagery offers a level of control that is most similar to motor action. 24

Although the spatial resolution of an EEG-BCI is not nearly as fine as invasive methods, amazing levels of control have been achieved in those who train to become profi- cient. For example, robust three-dimensional control of a cursor on a computer screen [104] and even a quadcopter to targets in a real-world environment [105] have been demonstrated using EEG-based motor imagery. Immersive feedback using virtual reality environments has been used to explore how control signals are modulated in life-life operating scenarios [106, 107, 108]. Other forms of feedback, primarily employed with intracranial BCI platforms, rely on real-time control of prosthetics. Users are able to gage the progress of their training and generate rewards using a physical effector on their environment. These often employ sophisticated control systems to achieve robust control. Both non-human primates and human volunteers have utilized motor imagery along with invasive recording of multi- unit neural responses to control prosthesis to achieve remarkably natural movement [20, 109]. Invasive recordings enable recording from multiple neurons with preferred firing tuned to a specific motor modality and direction of action, and thus allow for control of multiple degrees of freedom. Motor imagery has also been applied to systems using ECoG [43]. With a grid implanted on the cortex, the imagination of individual finger action was able to be decoded using this type of system.

2.2 Amyotrophic Lateral Sclerosis

ALS is a progressive neuromuscular disease with roughly 6,000 new diagnoses in the US each year [110], with a mean onset age of 55. ALS results in progressive muscle weakening, leading to eventual loss of voluntary limb movement, inability to speak or eat, and respiratory failure. The mean duration of survival after diagnosis is three to five years [111]. ALS was first described in 1869, and more than a century later there exists a very basic and incomplete understanding of its pathologic mechanisms [112]. 5-10% of presenting ALS cases appear to have a genetic component [112], although this number may be underestimated [113]. For the remaining patients who experience sporadic ALS, the etiology is unknown. The resulting motor neuron loss appears to be a result of oxidative damage, mitochondrial dysfunction, defects in axonal transport, growth factor deficiency, and glutamate excitotoxicity. Similar changes in neighboring astrocytes may also disrupt glutamate transmission, exacerbating excitotoxicity and hastening cell death 25

[112]. Even though the exact disease pathway is an area of active study, the overt manifestations encompass gross motor changes that warrant BCI intervention. However, other manifestations of the disease could also be relevant to BCI design: non-motor changes and genotype-phenotype associations.

2.2.1 Signs and Symptoms of ALS

The formal El Escorial criteria used to diagnose ALS is based on presence of progressive upper motor neuron and lower motor neuron abnormalities in multiple regions, while also ruling out other motor disorders [114]. Beyond these common criteria, the variations in disease onset, aggressiveness, and extra-motor effects are substantial. ALS onset can be bulbar, spinal, or mixed in nature, with poorer prognosis for those with bulbar onset [115]. Some patients experience primarily upper motor neuron symptoms of spasticity and brisk reflexes, while some experience primarily lower motor neuron symptoms of muscular atrophy and paralysis [116] (Figure 2.9). These variations are assessed in the clinic through the use of a standardized scale, the ALS Functional Rat- ing Scale - Revised (ALSFRS-R) [117], neurologist evaluation, and electrophysiological recordings. The ALSFRS-R is a widely-used, 12-item, ALS-specific questionnaire assessing physical function in the bulbar, upper limb, lower limb, and respiratory domains. Each item is scored from 0 (poorest function) to 4 (normal function), and the scores are added to produce a total score from 0 to 48. In some patients, ALS results in degeneration outside the traditional motor regions, leading to cognitive and behavioral abnormalities. Although not a defining component of the disease, behavioral and cognitive symptoms can appear together or separately in disease manifestation [118]. In rare cases, patients can meet the criteria for frontotemporal lobe degeneration (FTLD), which can result in frontotemporal dementia (FTD). The evidence for a continuum of disease that links ALS and FTD is backed by clinical, pathological, and genetic evidence [119]. Physical symptoms are a diagnostic criteria for the disease, but some behavioral or cognitive symptoms occur in 10-75% of patients, with dementia occurring in 15-41% [114]. Behavioral symptoms are coincident with cognitive symptoms in about half of patients, and with depression in a smaller portion of patients [118]. The connection between ALS and the behavioral variant of FTD (bvFTD) is evident given similarities in 26

Figure 2.9: (Left) Upper and lower motor neurons are degraded in varying proportions, leading to symptoms ranging from spasticity to speech impairment to paralysis [120]. (Top Right) Effect sizes and 95% confidence intervals in various cognitive domains. A positive effect size indicates worse performance in the ALS group compared to controls [121]. (Bottom Right) The spectrum of genetic mutations known to occur on the ALS- FTD continuum. [122] grey matter changes of the anterior cingulate, temporal pole, and prefrontal cortex, as well as the underlying white matter in these regions [119]. Mental rigidity is the most common behavioral change associated with ALS. Patients can display decline in social interpersonal conduct, self-centeredness, impairment in regulation of personal conduct, apathy, irritability, emotional blunting and loss of insight [123, 114]. Many of the cognitive functions that have been shown to be compromised in ALS are associated with frontal regions that are degraded in FTD (Figure 2.9). The most common cognitive deficit is in executive functioning, relating to the ability to organize information, shift attention, and inhibit behavior [114]. A common test of executive functioning and working memory is verbal fluency, measured by intrinsic response generation and widely agreed to be affected [114, 121, 124, 125, 126, 127, 128, 129]. Other studies have also pointed to language deficits [121, 128, 130], highlighting possible lim- 27 itations in linguistic or semantic knowledge. This finding is controversial, as other studies have pointed to normal abilities on semantic knowledge, measured by naming and grammar tests [127, 131, 132]. Recently, another trend in research has appeared, with scientists classifying ALS patients with cognitive limitations into two main categories, those with executive dysfunction, and those with language dysfunction [128, 130]. In their analyses, some patients show reduction in performance on tasks in one or both of these realms of cognition. Deficits in language, executive functioning, and physical ability lead to changes in discourse, the content and productivity of which is altered in ALS patients. Other commonly observed cognitive deficits are attention/mental control [114, 125, 126, 127], visual and verbal memory impairments [121, 127, 129], and free recall [114, 125, 133]. Genomic risk factors have been found in both the familial and sporadic types of the disease [113, 134, 135], and provide further evidence in support of an ALS-FTD spectrum [136] (Figure 2.9). Such findings include the numerous mutations of SOD1, FUS, TARDBP, of which there are evidence for greatly different phenotypes [113]. A recently discovered genetic marker is the repeat of hexanucleotide GGGGCC in the C9ORF72 gene, which in healthy people is rarely repeated more than five times, but in 40% of familial ALS patients and 7% of sporadic cases can be repeated hundreds and even thousands of times [134]. The expansion of this gene results in approximately equal proportions of individuals with ALS and FTD [135]. The forming consensus is that a multi-genetic contribution underlies diverse pathological mechanisms which converge on the ALS phenotype.

2.2.2 Brain Imaging in ALS

Today, diagnosis of ALS is done independent of brain imaging methods such as EEG and MRI. However, structural magnetic resonance imaging can be used as an exclusion criteria for other neurodegenerative diseases, lesions, or myelopathies [137]. MRI has been used with much success in ALS research in realms other than diagnostic testing. There are many results which are specific to particular regions of the brain. I will not attempt to give all of the findings here, but rather an idea of the heterogeneity of structural and functional changes that occur within ALS. Brain imaging in ALS confirms atrophy in primary motor regions, although MRI 28

Figure 2.10: (Left) The brain of a patient with sporadic ALS, showing atrophy of the frontal lobe [138]. (Right) Graphical representation of the changes in grey matter (Mo- tor Cortex, MPFC = medial prefrontal cortex, Temporal pole) and their underlying white matter tracts (CST = corticospinal tract, CC = corpus callosum, ILF = inferior longitudinal fasciculus) in the ALS-FTD continuum [119]. studies produce inconsistent results [137]. An imaging finding that is consistent across MRI studies is degeneration in the corticospinal tract, measured using diffusion ten- sor imaging and reflecting disease severity [137, 119]. A recent study [10] confirmed pathological and physiological findings that the presentation of bulbar or limb symptoms is associated with atrophy in the corresponding brain regions, and that the level of degeneration is linear with the functional score in these areas. There are non-motor anomalies in brain structure as well. The frontotemporal and parietal regions are additionally affected, as both grey matter and underlying white matter in frontotemporal and parietal regions display signs of atrophy [119, 137, 139]. Fur- ther evidence for an ALS-FTD continuum comes from imaging studies which show a distinct trend in degeneration in motor, frontal, and temporal areas (Figure 2.10). Those with bvFTD showed degeneration in the anterior cingulate, motor and premotor cortices, similar to the pattern observed in ALS patients. Damage to the anterior cingulate has been shown to produce the hallmark increase in apathy common to both groups [119]. Those with FTD produced greater overall atrophy in grey matter regions, including the prefrontal and temporal cortex, as well as in the striatum. ALS patients overall showed more white matter changes, especially in the corticospinal tract. A distinguishing feature in the continuum of structural changes was the spread of pathology into the anterior temporal lobe that exists in ALS patients co-diagnosed with FTD, but does not exist in patients with classic ALS [119]. 29

The use of EEG for describing electrophysiological changes in ALS have been in- conclusive, leading one to believe that gross brain activity is essentially unaffected by the disease [140]. Currently, the use of EEG for diagnostic purposes does not meet the sensitivity and specificity requirements needed to justify it as a clinical tool, and is not one that is regularly used for that purpose. While resting state changes in gross EEG do not provide a reliable marker for disease progression, event-related potentials (ERPs) measured by EEG under certain conditions has been shown to be sensitive to the cognitive status of ALS patients. Both the N200, a negative wave associated with target identification [12], and the P300 potentials arising from an oddball task of auditory stimulation have been shown to be of significantly longer latency in ALS patients [141]. These findings highlight the changes that occur in the ALS brain which affect the endogenous components of the ERP, specifically the N200 and the P300, while leaving the exogenous early components unaffected.

2.2.3 Assistive and Augmentative Communication in ALS

A product of the disease is the eventual loss of muscular coordination responsible for speech production. At some point, 80-95% of ALS patients are unable to use their own speech to communicate [142]. As the disease progresses, many people with ALS develop locked-in syndrome (LIS), which describes individuals who are awake and conscious but in a physical state of almost complete immobility and loss of verbal communication [143]. The last muscular control available to an ALS patient with LIS are the muscles of the anal sphincter and the eye [144] both of which are lost as the patient transitions to completely locked-in syndrome (CLIS). In CLIS, even eye movement has been compromised, eventually leaving a portion of the 2% of ALS patients who receive invasive ventilation support, the majority of which are unplanned, without any means of communication [145]. For this reason, patients with ALS are one of the most cited clinical target populations for BCI use [146], as these systems would remain the only plausible alternative for interpretation of intent while the patient is in a state of complete voluntary muscular insufficiency. BCI has been researched for use as an augmentative and assistive communication (AAC) technology for those living with advanced ALS. Common low-technology AAC devices include communication boards and notepads. High-technology systems may 30 include computer-based speech synthesizers, as well as eye- and head-tracking systems. AAC acceptance overall in the ALS population has increased from 73% prior to 1996 to 96% by 2004 [147]. Motor-disabled patients consistently rate mobility, communication, activities of daily living, and employment as the main reasons to improve independence through the use of AAC technology [148, 149]. Those who use the devices for the longest are those who opt for mechanical ventilation [150] and continue to use them while receiving breathing support. The majority of BCI systems have been tested in healthy individuals. However, a few studies have explored the applicability of BCI-AAC devices in patients afflicted with ALS. The first were SCP-based BCI systems explored by Birbaumer et al [11], who achieved communication with ALS patients and continued to do so after they developed LIS. Using language-based P300 systems, ALS patients have shown accuracy rates similar to or slightly below the accuracy of healthy control subjects [151, 152]. Another system which has achieved success in the ALS community is the imagery-based BCI. [153, 154]. Recent results indicate that the type of successful imagery and location of disease onset are not correlated [153, 155]. However, the presence of significant bulbar involvement was found to be negatively correlated with the ability to modulate motor rhythms [155]. This could signal that behavioral alteration, which is more prevalent in the bulbar subtype [156], is partially responsible for the decrease in performance. A caveat remains, however, as ALS patients having CLIS at the time of implementation were unable to gain control of the device [157], similar to the results found in SCP studies [11]. There is evidence that P300 speller systems may be the most reliable and rapid BCI-AAC communication devices for these patients [158], although the usefulness of this paradigm may be limited primarily to the auditory modality in CLIS [144].

2.3 Unsolved issues with BCIs

2.3.1 Asynchronous Use

One of the biggest challenges facing deployment of BCIs for communication purposes are the cue-based paradigms by which the systems operate. The tasks the user performs, cued by the computer, are synchronized to the system that performs the decoding. This 31 allows for little flexibility regarding the pacing of the system, including when to send user intent to the computer. The ideal BCI would be one that is self-paced, the user controls when and how often to communicate their intent. A system that translates the signals of unknown timing into commands is called asynchronous. In practice, this means that a user has the ability to initiate a “no-control” option, during which the computer does not perform classification or generate feedback from the brain signals. Classifying a state of unknown timing is what makes asynchronous BCI such a difficult prospect. Different approaches have been taken which dictate the timing of the system. Signals that exceed threshold [159, 160], have been used to indicate an action state, while others require the control signal be sustained, accumulating a sufficient amount of information, [161, 162]. Additionally, refractory periods on sequential classification have been used to give structure to self-paced decision making [161]. Other mediators of asynchronous control include low frequency oscillations in the 1-4 Hz range [163], motor rhythm idling states [164], as well as steady state evoked potentials [160]. Of different types of mental imageries, motor-imagery has been shown to be one of the most effective tasks at discriminating between active and idle states, while auditory imagery proved to be least separable [52]. Beta rebound found after the imagination of foot movement has been used to initiate control over a BCI [159], as well as indicate the system for intent to begin a second phase of classification in a hybrid-type BCI [161]. Perhaps the sub-field of BCI most interested in asynchronous use are the BCI- controlled wheelchairs. Often these devices are paired with a semi-autonomous guidance systems that generally integrate information about the environment to weight decisions from brain signals accordingly [103, 162, 165]. In this way, a user is able to make self-paced decisions about the direction of the wheelchair without having to worry about the avoiding obstacles or minor trajectory changes. Work in Chapter 3 addresses a novel solution for asynchronous BCI use by utilizing supplementary brain information in the assessment of mental state. By taking advantage of previously-unused EEG signatures of attentiveness and cortical excitability, we describe an additional potentially useful mechanism for creating self-paced BCI control. 32

2.3.2 BCI Inefﬁciency and Predictors of Success

For some BCI tasks, particularly the SCP and motor-imagery paradigms, a substantial percentage of individuals are unable to gain control of the BCI even after a typical amount of training with the device. These particular paradigms are the most user intensive in terms of training, and require sustained attention to complete. Despite training, it is estimated that roughly 20% of healthy individuals fail to achieve an effective control strategy with a motor-imagery paradigm [166, 167], although estimates have also been higher [168]. The process of training on a BCI can be very time consuming and may result in frustration if the user is unable to establish control over the system. For this reason, different electrophysiological markers have been assessed for predicting performance on a BCI system over a short training period, before much time is spent training on the system. With such predictors, we hopefully are able to begin the process of personalized engineering an effective BCI system. Different variables have been found to be predictive of BCI success, ranging from electrophysiological tests to psychological evaluations. Here we focus on predictors of performance for the SMR-BCI in particular. Decent reviews of the ﬁeld are given in [169, 170]. In the latter, the authors discuss the identiﬁcation of useful performance predictors, as well as the time scale of the predictor, or whether long or short term performance can be associated with the factor. Blankertz et al. found that a combination of resting sensorimotor rhythm amplitudes were predictive of subsequent trainability on the motor-imagery task, and were able to establish a fairly strong correlation between online accuracy and their performance predictor [166]. Halder et al. found that there were initially substantial differences in fMRI activation of the supplementary motor area in high vs. low performers [171]. Structural MRI has also been able to distinguish high performing and low performing SMR-BCI users [168]. Central white matter changes, rather than gray matter changes were the relevant features for making this distinction. Other research has focused on how performance varies within subjects over time on a trial-by-trial basis. High-gamma power in the pretrial period has been shown to correlate with SMR-BCI performance [172]; the fronto-parietal network that generates these rhythms is hypothesized to be associated with attentional processes. Training strategies for maintaining the user in a state receptive to BCI control have been proposed using classic meditation practices [173] as well as neurofeedback [170]. 33

Earlier lines of research focused on psychological factors. “Locus of control of reinforcement”, a measure of whether perception of the result of a person’s attempt at BCI control is a result of their own performance/personality, or a result of luck, coin- cidence, or destiny was used to study BCI ability [174]. The authors’ conclusion was that those comfortable with technology and their power to control it generated higher accuracy with the motor-imagery task. Motivation has also been associated with device utility, both positively (feeling of challenge and confidence in mastering the device) as well as negatively (fear of failure) [158]. Both ability to concentrate on a task, and measures of visuo-motor coordination have been associated with high SMR-BCI performance [167]. Furthermore, self-rating of kinesthetic imagery quality via imagery questionnaire has correlated with performance on the SMR BCI task [175]. Interest- ingly, users who are experienced in meditation are also able to modulate brain rhythms more effectively [176]. Work in both Chapters 3 & 4 directly address the issues associated with BCI performance predictors and device literacy. In Chapter 3, alternate EEG predictors are assessed as a potential gating mechanisms for improving BCI performance by allowing classification to occur only during times which are most favorable. In Chapter 4, we elaborate on a possible source of device inefficiency in ALS patients, the high prevalence of mild to moderate cognitive dysfunction found in this population.

2.3.3 Limitations of the locked-in

A major caveat concerns the goal of personalizing a communication device for the severely disabled. Currently, the state-of the art of assistive communication in ALS is the eye-tracking device. Arguably, if eye function is intact, this type of system is a superior tool for intuitive, rapid communication. However, as oculomotor function declines, or gaze holding becomes too fatiguing, the efﬁcacy and desire to use such a device among patents deteriorates [177]. The unique utility of a BCI-AAC in ALS is for those who have lost residual eye movement, which gradually occurs during the transition to CLIS. Although the capacity to control a BCI is relatively conserved in ALS patients with LIS, no attempts at establishing communication using a non-invasive electrophysiological recording have been successful [178]. This is true even when intensive conditioning 34 was performed in patients who possess positive responses to passive tests of cognition [179]. Furthermore, there is no indication that invasive recording techniques produce imagery signals with greater discriminability [144], although for technological and ethical reasons, this a relatively untested BCI paradigm. Recently, marginal communication has been established in an ALS patient after having been completely locked in for two years [44]. This BCI system, based on NIRS technology, measured metabolic brain activity on a 25-second time scale to infer ‘yes/no’ responses to aurally-delivered question prompts. These results, especially the 100% accuracy achieved during certain sessions, are highly encouraging. Lack of success with other CLIS paradigms have been attributed to technology incompatibilities [44], limitations of sensory input due to loss of ocular control [99], cycles of vigilance [178] as well as the extinction of goal-directed behavior [180]. The latter phenomenon, which is hypothesized to occur as a result of the complete abolition of motor control and feedback, represents a signiﬁcant cognitive change that is hypothesized to occur in CLIS. In the latter half of Chapter 4 we detail certain clinical factors which correlate with a decline in BCI function. In opposition to physical health, which may decline while BCI performance is maintained, psychological health impairment correlates with decreases in performance in two types of BCI systems. Although no completely locked-in patients were evaluated in our study, we elaborate on the cognitive factors that may contribute to low performance, and in Chapter 5 describe potential strategies to increase performance through personalization. CHAPTER 3

GATING MECHANISMS FOR BCI

One of the biggest challenges for sustained BCI use is the issue of asynchronous control. BCI users are often presented with systems governed by rigid timing; the cue is presented during a fixed period and the user is given a set time to complete the task, after which classification occurs and feedback is presented. What happens when the user is focused elsewhere, unable to complete the task, or ready to stop? The need for asynchronous control is apparent, especially in cases where the user may not be able to manually input a command or turn off the device. The following two studies were performed during the first two years of my thesis work. Although they hardly present a final solution to the issue of asynchronous timing within the BCI environment, they offer a possible mechanism for offloading more control to the user. These studies were performed in healthy, college-aged participants, and consider alternate features found in the recording, which, although ignored in the primary classification task, could serve an entirely different purpose as a gating feature. As a gating mechanism, secondary neural signals could serve as a switch to activate or inactivate the process of primary task classification, effectively setting the start and endpoints of system use, as well as the timing of trials within the operational period. In these studies we looked for gating signals reflecting states of vigilance and task readiness that are able to be extracted from the EEG as information supplementary to the primary task. The work done in processing and validating these measures in offline analysis are described in the remainder of the chapter. 36

3.1 Evoked Responses to Asymmetric Cueing [1]

In the case of a motor-imagery paradigm, the evoked responses to synchronization cues are often overlooked or treated as artifact. Typical stimuli used in motor-imagery paradigms are unilateral in nature [87, 95, 181]; they take the form of arrows or boxes presented to the user in a single visual hemifield, which can lead to asymmetric visually evoked potentials (VEPs) in the recorded EEG. Substantial literature on VEPs describes the hemispheric asymmetries in evoked potentials recorded from the scalp that occur as a result of imbalanced visual stimuli. Subjects exposed to a unilateral stimulus exhibit shorter latencies for the characteristic positive and negative evoked potentials over the visual cortex contralateral to the stimulus [182]. These potentials, denoted as P1-P3 and N1-N3, occur in the 30-300 ms following flash stimulus presentation to the foveal retina. The increase in transmission time is explained anatomically by signal delay across the corpus callosum [183]. The implications of this phenomenon have yet to be utilized in a BCI paradigm. In this study, we assessed whether the VEPs due to cue asymmetry be used as part of a hybrid BCI as a priming system for attention detection in an imagery task. Data were recorded from 5 subjects (ages 18-28, two female) in this study. Nineteen EEG channels were arranged on the scalp according to the standard 10-20 system, referenced to linked earlobes. Additionally, signals were recorded from four EOG channels for the purpose of artifact correction. Signals recorded from the scalp were amplified and bandpass filtered between 0.1 and 30 Hz using a commercial EEG recording system (Guger technologies, www.gtec.at). Signals were sampled at 256 Hz. The study was conducted in accordance with guidelines approved by the Institutional Review Board of Penn State University. In order to assess the effect of unbalanced vs. balanced stimuli with minimal additional factors, we used three different types of arrow cues. The unbalanced cue extended from the center of a fixation cross to the periphery. In contrast, the balanced cues were centered in the middle of the cross, using only the directionality of the arrowheads to indicate cue type. For these long and short balanced arrows, their ends continued to the periphery or halfway to the periphery of the screen. These visual stimuli were the cues used to direct subject intention. Each subject performed four motor imagery sessions lasting approximately one hour each. Each session comprised four to five runs. Each 37 run consisted of 50 trials, and the presentation of the cues in each run was randomized. In each trial, the subject performed five seconds of imagery of hand movement on the side indicated by the cue. Raw data were first run through a regressive EOG correction algorithm [184] to remove artifact associated with the activity recorded from the four EOG channels. Follow- ing this, trails were rejected if the amplitude of the EEG in any of the channels exceeded 60 mV. Artifact control resulted in the removal of 0%, 4.8% 25.5%, 13.8%, and 0% of total trials for the five subjects. Data corrected for artifact were used in subsequent analysis. Hemispheric evoked potential asymmetry can be measured with the event-related lateralization (ERL) statistic, as in (3.1) [185].

VEPO2(L) VEPO1(L) +VEPO1(R) VEPO2(R) ERL(O1,O2) = − − (3.1) 2

This equation gives the event-related lateralization across channels O1 and O2 in the visual cortex in response to the mean VEP evoked by left (L) and right (R) cued trials. Significant lateralization was found in four of the five subjects who participated in the study. The period of lateralization was highly stereotyped, occurring with a negative peak at 200 ms and a positive peak around 300 ms (Figure 3.1). The interval of significant lateralization due to cue type was found to be 203-305 ms after cue presentation, as determined by an ANOVA (p < 0.01). Post hoc paired significance tests confirmed that the ERL evoked by the unbalanced cue was found to be significantly different from both short and long balanced cues. The common finding across subjects was that ERL occurs due to latency discrepancies in two major evoked potential peaks. Because the contralateral signals consistently lead the ipsilateral ones, the ERL during the negative portion of the evoked potential has a negative sign, and during the positive component yields a positive lateralization value. The expected consequence of this hemispheric lateralization is an increase in the accuracy of the classifier before the sensorimotor modulations produced by motor imagery are expected to occur. Indeed, this increase was observed in all subjects who demonstrated cortical lateralization, and was found roughly 200-700 ms following cue presentation (Figure 3.1). This classification increase occurs uniquely for the unbal- 38 anced cue because the evoked potentials due to each of these cues contain hemispheric timing differences which are not present in the case of balanced cues. This finding mo- tivates a few things for future motor-imagery BCI work. In order to classify intent of the motor actions of the user and not an evoked artifact of the stimulus, a balanced cue should be used. Alternatively, the data from the initial period following cue presentation should be discarded in classification of the imagery-task. This alone is not a new result for the VEP or BCI communities. However, this result served as an impetus to rethink the purpose of the cuing mechanism. The arrow stimulus in a BCI training paradigm serves to direct the intent of the user, not create spurious classification features independent of intent. In a self-generated feedback task with no cue, the user is in control of the direction of the imagery. In this case of free form control, the stimulus could be re-purposed as a probe for assessing user attention. Given the user is attentive to the task, the positive classi- Figure 3.1: (Top) Aggregate ERL curves for all five subjects due to unbalanced as well fication of an evoked potential may serve as short and long balanced cues that were to gate the continuation of the remainder presented at time 0. (Bottom) Classification of the trial. Otherwise, the trial may be accuracies for detecting left vs. right trials using evoked potential traces for each type aborted or the system inactivated. With of cue. this goal in mind for the cue, the stimulus must be chosen so that it either evokes a strong and consistent VEP for an individual trial, or possibly even a strong lateralization of the VEP, so that real time assessment of task vigilance may be made. Although this work did not produce any groundbreaking results in asynchronous BCI, it focused our goal of using visual cues for purposes other than just directing the user. In the following study, we consider additional gating variables for their predictive value for success in an imagery task, so that we may use these features to enable user- regulated timing of a motor imagery interface as part of a two-stage BCI system. 39

3.2 Use of single trial gating signals to optimize motor- imagery BCI [2]

3.2.1 Introduction

As was described in Section 2.1, the imagination of motor action is a common mental task used for delivering intent without overt action, and is reflected as changes in the SMRs over motor and premotor cortical areas. Success in controlling an SMR-based BCI system depends on the user’s ability modulate these motor rhythms from a baseline state. Little is known about why some individuals are able to do this well and why others are not, but one such predictor, the level of baseline SMR amplitude, has been shown to be positively correlated with the success of the individual in completing the task [166]. BCI performance has also been demonstrated to be associated with pre-trial gamma band (70-80 Hz) amplitude [172]. EEG oscillations such as the SMRs have been given roles as modulatory gating mechanisms for information transfer between the cortex and subcortical structures such as the thalamus [12], and may explain the dependence of imagery performance on these rhythms. Using transcranial magnetic stimulation (TMS), low amplitude oscillations were shown to be representative of an excitable state of the motor cortex [186]. These findings indicate that the regulation of SMRs, specifically the mu rhythm, is likely to condition the cortex for forthcoming perception or action [187]. A BCI that relies on changes in mu rhythm may be affected by pre-trial mu amplitude simply because high mu before the trial offers opportunity for larger decreases from baseline level. However, the ability to form motor imagery may also be modulated by oscillations representing motor network excitability. Early work by Bishop demonstrated that the phase of ongoing cortical oscillations partially predicted the seemingly random variations in evoked potentials due to visual stimulation [188]. This demonstration led to the hypothesis that these oscillations represented cyclical cortical excitability. Later studies showed how these periods of excitability could be demonstrated by consistent changes in perceptual or motor thresholds. Phase and amplitude of the occipital alpha at the time of visual stimulus presentation has been shown to be predictive of stimulus detection [12, 187]. Somatosensory cortical evoked responses to painful stimuli are also facilitated by the amplitude of 8- 40

13 Hz mu motor rhythms [189]. Further evidence comes from the work of Kruglikov et al. [190], who showed that auditory evoked potential morphology is a function of broadband EEG phase. In addition changes in excitability resulting from oscillatory activity, evoked responses to varying stimuli have been shown to be modulated by region of fixation [191, 192], and serve as a marker of visual attention. Visual attention is controlled by a distributed network of cortical and subcortical areas which act to provide “bias signals” that enhance or suppress the responses to visual stimuli [192]. The distinction between fixation and attention in the context of a BCI is an important one. In a cued BCI paradigm, attention is required for the subject to understand the cue and react to it, but fixation on that cue is not absolutely necessary [191]. Of the components of the VEP, the N1, N2c, and P3b peaks have been shown to be influenced by visual attention [12, 191, 193]. Increases in amplitude of the first two components and a decrease in the latency of the P3b have been shown to be correlated with visual attention. The aim of this study was to identify intra-individual predictors of single trial BCI success. By finding brain signatures which correlate with improved performance, we can effectively increase the accuracy and bit rate achieved with the interface. One possible way to do this is by gating trials with low predicted performance. We evaluated the utility of three of these brain signatures as gating variables: the amplitude and phase of ongoing motor rhythms, and the visually evoked potential produced in response to the cuing mechanism. These were chosen because of the evidence for these features to associate with motor-readiness and task vigilance, respectively. We show that these peri-stimulus features of the EEG, which are generally discarded during motor-imagery feature selection, are correlated with performance in a subset of subjects and can be used to boost the information transfer rate of a BCI communication system.

3.2.2 Methods

Experimental Setup

A commercial EEG recording system (Guger technologies, www.gtec.at) was used to acquire data from subjects. Data were sampled at 256 Hz and band pass ﬁltered at 0.1- 30 Hz. This bandpass range was chosen to preserve VEP components and oscillations in the mu band. Data were recorded within the Simulink environment in MATLAB and 41 stored on a notebook computer (Dell Latitude E6400) running Windows XP. Subjects were seated in a chair facing an LCD monitor which displayed cuing and feedback information. The experimental protocol was approved by the Institutional Review Board of Penn State University.

BCI Paradigm

Nine right-handed volunteers, all male with ages ranging from 18-37, participated in a cue-paced, one-dimensional center-out motor imagery task. Channels FC3, FC4, C5, C3, C1, C2, C4, C6, CP3, CP4, P5, P3, P1, P2, P4, P6, PO3, and PO4 were recorded, in addition to three EOG electrodes placed on the lateral canthi as well as just above the nasion. All channels were referenced to linked earlobes, and ground was placed at Fpz. Each subject performed four sessions over a two week period. Each session lasted approximately 1.5 hours. During each session, the subject performed five runs of 60 trials each, divided equally between left, right, and no-target cues that were presented in a randomized sequence. The first run of each session, the training run, was used to train the classifier so the remaining four testing runs could be used to give feedback to the subject as they performed the task. In each trial, the subject was cued by an arrow pointing in the left or right direction. Arrows were displayed symmetrically in the visual field to minimize asymmetry in the evoked potential due to cue type, which in Section 3.1 was shown to falsely contribute to BCI classification accuracy independent of user-driven modulation [1, 194]. Subjects were instructed to perform imagery of an object-oriented grasping action for the hand corresponding to the direction of the arrow being displayed. If no arrow appeared, the subjects were informed to relax and were given no visual feedback. In each trial of the training run, a fixation cross would first appear at which time the subject would relax. If this were followed by an arrow cue one second later, the subjects were instructed to perform imagery while the arrow remained on screen for two seconds. Each trial was followed by a random inter-trial period of 1-2 seconds. During the testing runs, a target appeared on the far side of the screen in the direction of the arrow, and during the two seconds in which they performed imagery, a cursor moved on the screen to provide the subject with feedback. Feedback served as an indicator to the subject how closely their EEG signals of intent matched templates for left and right cues developed in the training run. This 42 was done by summing the squared distance D of the band power in trial n from the m (left, right) template band power over all time points t and features f (3.2). Each ∈ template µ consisted of mu and beta band powers from channels C3, C4, P3, and P4 during the period of motor imagery, making up f = 8 total band power features.

8 2 Dn,m(t) = ∑ (xn, f (t) µm, f (t)) . (3.2) f =1 −

Here, xn, f represents a band power feature from a single trial. The feedback at each time point was calculated as the log ratio of the left distance over the right distance (3.3).

Dn,le ft(t) Feedbackn(t) = Feedbackn(t 1) + log . (3.3) − Dn,right(t)

This control algorithm drove the cursor to the left when this ratio was less than one, and to the right when the ratio was greater than one. Feedback was accumulated over the length of the trial and then reset to the center of the screen at the beginning of the next trial. Zn, the success of the subject at performing imagery during trial n, was deﬁned according to (3.4),

 Feedbackn(T), if Cue = right Zn = (3.4)  Feedbackn(T), if Cue = le ft, − where the time index T marks the end of the trial. Trials were classiﬁed as successful if at the end of the imagery period the cursor was on the same side of the screen as the target. Over the four sessions, each subject completed 16 test runs consisting of 960 trials, with 320 trials each for left, right, and no-target cues. The data belonging to the 320 trials of left and right cues were analyzed ofﬂine using the methods described below.

Preprocessing

Artifact correction

The ﬁrst step in ofﬂine data preprocessing was removal of eye-related artifacts that resulted from blinking or eye movement. This was a two-step process including artifact reduction and trial rejection. Artifact reduction was accomplished by linear regression 43

[84]. This least-squares method assumes the linear superposition of neural and artifact sources to produce the measured signal. Assuming the independence of the artifact sources and the neural sources, data can be used to ﬁnd a weight matrix, which represents the projection of noise sources onto neural sources. We used the 18 EEG channels as our recorded signal Y, and the three EOG channels as the noise sources U, to ﬁnd the weight matrix W, and solve for decontaminated neural sources S (3.5).

Y18 T = S18 T + W18 3 U3 T . (3.5) × × × · ×

Here, T is the length of the data segment from which the weight matrix is calculated. In practice, this method may be suboptimal if there is signiﬁcant leakage of the task- relevant EEG data into the designated noise channels, rendering the assumption of independence void. To minimize this effect, we solved for W using data which was sampled from when the EOG channel exceeded 75 µV, as in the case during a blinking event. Following artifact reduction, trials

20 were rejected if channel FC3 displayed V) µ absolute amplitude of greater than 50 0 (a)

µV. To control for artifact in unin- Amplitude( -20 10 V) tentional movements, bipolar electrodes µ were placed on the forearms of a subset of 0 (b) the subjects to record EMG muscle activ- Amplitude( -10 ity during the recording session. EOG and 5 (c) EMG were both analyzed ofﬂine to rule 0

Phase(radians) -5 out possible contamination from overt eye

V) 10 and arm movements. µ (d) 0

-10 Amplitude( -1.5 -1 -0.5 0 0.5 Extraction of gating variables Time after cue (s)

Amplitude and phase of peri-stimulus mu Figure 3.2: Phase and amplitude extraction Data in channels C3 and C4 were fil- of a single EEG trial. (a) Trial n of EEG data. (b) Signal was filtered in the subject- tered with a Laplacian spatial filter to lo- specific mu band. (c) The phase of the band- calize the mu rhythms specific to the mo- limited signal at τn, the time of cue presen- tor cortex [195]. Peak mu frequency in tation, is marked by the vertical dashed line. these channels was found for each subject (d) Amplitude of the band-limited signal. 44 using the multitaper spectral analysis method [196]. The EEG data in these channels were filtered using a zero-delay filter in a 2 Hz range around the peak mu frequency. A Hilbert transform was applied to this band-limited signal to generate the analytic signal

Sa(t), comprising a real part S(t) made up of the original data, and an imaginary part H(t), its Hilbert transform (3.6),

Sa(t) = S(t) + iH(t). (3.6)

The amplitude of the signal A(t) was calculated as the Euclidean norm of the real and imaginary parts of the analytic signal (3.7). q A(t) = S2(t) + H2(t). (3.7)

The instantaneous phase φ(t) was calculated as the four quadrant inverse tangent of the ratio of the imaginary part of the analytic signal to the real part (3.8). Phase ranged from -π to π. H(t) φ(t) = arctan . (3.8) S(t) For each trial n, the amplitude and phase of mu in C3 and C4 at the time of cue presentation were extracted (Figure 3.2). Mu amplitude was the ﬁrst gating variable. Linear regression between trial success and amplitude was used to determine association. We assumed that the success at the end of the trial was a linear function of the natural-log- transformed peri-stimulus mu amplitude (3.9). Log transformation was performed on the spectral features to enforce the normality of the data as well as reduce outlier effects [197].

Zn = α ln(A(τn)) + β. (3.9) ×

In this equation, α and β are the parameters of the regression, and A(τn) is the magnitude of the mu rhythm at the time of the cue of trial n. We used the slope of the regression α, as the measure of correlation between mu amplitude and trial success. Mu phase was the second gating variable. We assumed trial success was a cosine function of the phase of the mu rhythm.

Zn = γ cos(φ(τn) δ) + ε. (3.10) × − 45

50 50 (a) (b)

0 0

-50 -50 V) µ

50 50 (c) (d)

0 0

-50 -50 PO4-FC4 Amplitude ( PO4-FC4Amplitude 20 20 (e) (f) 10 10

0 0

-10 -10

-20 -20 0 0.1 0.2 0.3 0.4 0.5 0 0.1 0.2 0.3 0.4 0.5 Time after cue (seconds) Time after cue (seconds)

Figure 3.3: Construction of the VEP template from phase-corrected left cue trials, channel PO4-FC4, subject 7. (a) A single trial of VEP data. (b) Phase-matched correction by subtraction of a non-cued trial with similar phase (black line) results in the phase- corrected trial (red line). (c) Uncorrected trials of the same phase group and their average. (d) Corrected trials of the same phase group and their average. (e) All eight phase groups and the average resulting VEP (black trace). Regions with thick horizontal lines indicate times during which the data in the phase bins were signiﬁcantly different as determined by ANOVA between the 8 phase groups (p<.05, Bonferroni corrected). (f) The ﬁnal template for this subject is the average of the phase corrected trial groups (black line).

Here, γ, δ, and ε are regression parameters specifying the amplitude, phase, and offset of the fitted cosine, and φ(τn) represents the phase of the mu rhythm at the time of the cue of trial n. We specified the frequency of the cosine to be one cycle per 360°of phase. To find the parameters of this cosine model, we performed a non-linear regression using the MATLAB function nlinfit.m. The correlation measure between mu phase and trial success was the amplitude of this fitted curve, γ. Match to VEP template The final gating variable was related to the quality of the VEP. In order to define 46

20 20 S1 S1 0 0

-20 -20 0 0.2 0.4 0 0.2 0.4 20 20 S2 S2 0 0

-20 -20 0 0.2 0.4 0 0.2 0.4 20 20 S3 S3 0 0

-20 -20 0 0.2 0.4 0 0.2 0.4 20 20 V)

µ S4 S4 0 0

-20 -20 0 0.2 0.4 0 0.2 0.4 20 20 S5 S5 0 0 PO4-FC4 Amplitude ( -20 -20 0 0.2 0.4 0 0.2 0.4 20 20 S6 S6 0 0

-20 -20 0 0.2 0.4 0 0.2 0.4 20 20 S7 S7 0 0

-20 -20 0 0.2 0.4 0 0.2 0.4 20 20 S8 S8 0 0

-20 -20 0 0.2 0.4 0 0.2 0.4 Time after cue (seconds) Time after cue (seconds)

Figure 3.4: Uncorrected (left) and phase match corrected (right) VEPs for all subjects. This represents for all subjects what are panels (e) and (f) for subject 7 in Figure 3.3. Colored lines are the eight phase groups, and the black trace is the resulting average VEP. Regions with thick horizontal lines indicate times during which the data in the phase bins were significantly different as determined by ANOVA between the 8 phase groups (p<.05, Bonferroni corrected). 47 whether the subject produced an evoked response during a single trial that represented good fixation, we needed to define a template that represented a typical VEP waveform for that subject. Following 0.1-30 Hz bandpass filtering, we first performed phase-matched control trial correction, following the methodology defined by Kruglikov [190]. This was done because the phase of EEG at the time of the cue presentation introduces an averaging bias. Individual trials were corrected using phase-matched control trials, or trials in which a similar alpha phase was evident but in which there was no stimulus presented. By subtracting out a signal with similar phase, we remove the predominant alpha rhythm and are left with a trace that better reflects the underlying neural correlate of the VEP (Figure 3.3). Such a procedure removes peri-stimulus bias by phase of the evoked potential in all subjects (Figure 3.4). Once corrected for phase, templates for left and right cues were defined by averaging EEG data over trials corresponding to each cue type for bipolar channels PO3-FC3 and PO4-FC4. This referencing scheme was the closest approximation to the standard Oz- Fz [198] we could accomplish with our electrode montage. The timing of the template ranged from 100 to 400 ms after cue presentation. A metric describing the match of individual trials to the template was created using a matched filter approach. This technique comes from radar communication as a method for maximizing the probability of detecting a target waveform in the presence of Gaus- sian noise [199]. Although a derivation can be found elsewhere [200], the optimal filter for maximizing the signal to noise ratio of a signal generated by a linear, time invariant system with added Gaussian noise is the time-reversed version of the transmitted signal. This time-reversed template is called the matched filter. We applied the template as the transmitted signal s(t), and the phase-corrected single trial data as rn(t). The output of the matched filter yn(t) was the convolution of the trial data with the time-reversed template (3.11). The output of the matched filter that we used was the central value of this convolution, normalized to the magnitude of the template (3.12). This value, which we called the matched filter value (MFV), was related to the signal to noise ratio of the VEP, and served as the third gating variable. Trials with large MFV were interpreted as being trials which the subject produced a robust VEP.

yn(t) = rn(t) s( t). (3.11) ∗ − 48

y (0) MFV = n . (3.12) n s k k

Here, represents the convolution function, and yn(0) is the central value of the output ∗ of the convolution. The magnitude of the template s, in this case the Euclidean norm, is denoted by vertical brackets. Similarly to the previous gating variables, we determined the relationship between the MFV and success on the BCI task. Again, we assumed that the trial success was a linear function of the MFV (3.13).

Zn = ζ(MFVn) + η. (3.13)

The slope of this regression, ζ, reﬂects the correlation between of the outcome of the trial and the match to a VEP template of good ﬁxation.

Permutation testing for signiﬁcance

The three gating variables were tested for significant correlation with trial success in each subject. Because the dependent variable, the trial success, was not normally distributed, we chose to perform a non-parametric permutation test to determine the statistical significance of the regression parameters. We first computed the test statistic Qobs, the slope of the linear regression (α, ζ) or the amplitude of the fitted cosine (γ). Then we shuffled the Z success outcomes between the trials and used the same fitting procedures to calculate the permuted statistics, Q(k) for each permutation k = 1...K, where K = 1000. This permutation was performed with each gating variable from EEG features in two channels. Correction for multiple comparisons was done for each hypothesis test in the following manner, as described in more detail in [201]. For each permutation k, both the maximum Qmax(k) and minimum Qmin(k) values were chosen from the two Q(k) statistics evaluated for both channels. This resulted in K values making up each of the maximum and minimum empirical distributions of the randomized data set. The calculation of p-statistics, the probability of Qobs belonging to the null distribution of the permuted set, is given in (3.14).

1000 ∑ (H(Qmax(k) Qobs)) + 1 p = k=1 − . (3.14) high 1000 + 1 49

(1) Rank trials (2) Regression test statistic MFVn φ(τn) A(τn) n Z n , k Z Success, Trial Success, Permuted Trial Permute

A(τn) (K=1000 times) A(τn) Trials Zn = Zk,n = Ranked Trials α A(τn) + β α A(τn) + β × Qobs × Q(k)

(3) Inference

Calculate Signiﬁcance Generate Null Distribution Time Time Feedback

Figure 3.5: This flowchart shows a generic example of the permutation test for determining significant relationships between gating variables and trial success. (1) Test data from left and right trials are sorted using the rankings determined by each gating method. Shown here, the trials are ranked according to mu amplitude at the time of the cue, A(τn). Trials with high A(τn) are shown on the bottom of the block. The feedback at the end of these trials is also shown, with left trials in blue and right trials in red. Feed- back was subsequently simplified to trial success, Z. (2) The observed test statistic, Qobs is the slope of the regression that relates mu amplitude to trial success. The permuted test statistic Q(k) was computed K=1000 times after permuting the success scores of the th trials to Zk,n. (3) The k value for the maximum and minimum null distributions Qmax and Qmin were found from Q(k) across both channels tested. Finally the p-value of the test statistic distribution is computed as the percentage of values in this null distribution exceeding that of the observed test statistic.

Here, H is the Heaviside function. Ones are added to the numerator and denominator to include the Qobs in our null distribution. plow was also calculated similarly using Qmin to test for significance from both tails of the null distribution. Findings of either phigh or plow less than 0.025 are significant deviations from chance. The generic process of trial ranking and permutation testing for significance is shown Figure 3.5.

Trial gating simulation

To simulate the effect of utilizing the potential gating variables online, we computed how the accuracy and bit rates would change for each subject if a portion of the trials having 50 the least predictive value for task completion were removed. This predictive value was determined from the output of a linear discriminant analysis (LDA) classifier, which had been trained with the gating variables as the input and trial success as the outcome variable. Trials in the test set which were classified as having the lowest predictive value were gated, meaning they were stopped before the imagery period began. The simulation was repeated at different thresholds, from gating no trials up to gating 70% of the total trials. Gated trials were skipped 500 ms after stimulus presentation and were interpreted as no-decision while taking 1.5 seconds (1 second before cue and 0.5 seconds after) to complete. Allowed trials resulted in a decision while taking four seconds to complete. Accuracy was defined as the number of allowed trials performed correctly over the total number of allowed trials. 5-fold cross validation was used to find an average of the accuracy of the gating procedure at each threshold. The complete cross validation procedure was completed 50 times for each gating scenario and the accuracy was taken as the average. The bits per trial, B was calculated for an N = 2 choice task having accuracy P following the form of Wolpaw et al. (3.15) [8].

(1 P) B = log N + Plog P + (1 P)log − . (3.15) 2 2 − 2 (N 1) − The bit rate BR in bits/sec was calculated using (3.16).

BR = B/Tµ , (3.16) where the average trial time, Tµ , was increased to represent a penalty for disposing trials.

Tµ was calculated from the number of allowed trials, Na, and the gated trials, Ng, using (3.17).

Tµ = (4Na + 1.5Ng)/Na. (3.17)

3.2.3 Results

BCI performance

The operation of the BCI was successful for eight of the nine subjects, who achieved classiﬁcation accuracies for a left vs. right imagery task of 64.1-97.8%. Subject 2, with an accuracy of 49.5%, indicative of random BCI control using this method, was not 51

Low Amplitude High Amplitude -π Phase π Phase Low MFV High MFV 2 2 10

(a) (d) V) (g)

V) 1 V) 1 5

0 0

0 -1 -1 Amplitude C4 ( C4 Amplitude ( C4 Amplitude Amplitude PO4-FC4 ( Amplitude -2 -2 -5 0 0.2 0.4 0.6 0 0.2 0.4 0.6 0 0.2 0.4 0.6 Time after stimulus (sec)

0.5 0.5 (e) 0.5 (h) ) (b) 2

V 0 0 0

-0.5 -0.5 -0.5

-1 -1 -1 Band desynch. Band

( baseline from -1.5 -1.5 -1.5

-2 -2 -2 0 0.5 1 1.5 2 2.5 0 0.5 1 1.5 2 2.5 0 0.5 1 1.5 2 2.5 Time after stimulus (sec)

0.22 0.22 0.22 (c) (f) (i) 0.2 r2 = 0.86, p = 0.001 0.2 r2 = 0.009 0.2 r2 = 0.25

0.18 0.18 0.18

0.16 0.16 0.16 Trial Success Trial

0.14 0.14 0.14

1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 Group

Figure 3.6: Three EEG features were considered as potential gating variables (shown here are the right hemisphere features): mu amplitude in C4 at stimulus presentation (left column), mu phase in C4 at stimulus presentation (middle column), and match to PO4-FC4 VEP template (right column). For each feature, all recorded trials were divided into eight groups, from low feature value to high, represented by the eight colored lines/points. The first row is the average of the EEG in these groups. The second row is the average mu suppression from baseline in C4 in these groups. The last row is the mean and standard error of the success across all trials by group. considered successful at operating the BCI system and was omitted from analysis in this study. LDA with 10-fold cross validation was performed for features derived from EOG and EMG channels. Mean classification accuracy for all subjects using EOG features was 50.6 3.8% and mean classification accuracy for the four out of 8 subjects with ± recorded EMG features was 48.0 2.1%. Bounds are standard deviations. ± 52

Table 3.1: Results of the permutation test with the three gating variables

Values in the six columns represent value of Qobs, or the slope/amplitude parameters of the three regressions in two channels. Bolded values indicate a signiﬁcant Qobs at the 0.025 level. Feature Amplitude (α) Phase (γ) MFV (ζ) PO4- Channel C3 C4 C3 C4 PO3-FC3 FC4 1 0.012 0.013 0.010 0.011 -0.0001 0.0000 2 0.003 0.008 -0.016 -0.011 0.0004 0.0005 3 0.016 0.044 0.006 -0.009 -0.0002 -0.0003 4 -0.017 0.003 0.013 -0.006 0.0000 0.0002 Subject 5 0.029 0.055 -0.012 0.019 0.0001 -0.0003 6 -0.001 0.011 -0.007 -0.011 0.0003 0.0002 7 0.013 0.027 0.021 0.013 0.0003 0.0004 8 0.018 0.012 0.015 -0.006 0.0011 0.0007

Mu amplitude is predictive of trial success

Trials were ranked according to the three gating variables of mu amplitude, mu phase, and MFV. Figure 3.6 shows the grand average results for the study, in which trials are subdivided into eight groups based on ranking by each gating variable. Average EEG traces for each group are shown in subplots (a), (d), and (g), while mu suppression from baseline is given in the second row in subplots (b), (e), and (h). For groups ranked by baseline mu amplitude, there is a significant difference in the level of suppression during imagery (Figure 3.6b, black bar indicates ANOVA with p < .05, Bonferroni corrected for 768 time points). MFV shows positive association between gating variable group and trial success, mu phase shows no association , and the positive correlation between trial success and mu amplitude is significant (Figure 3.6c, r2 = .86, p < .001). Positive correlations between trial success and both mu amplitude and MFV are also evident in the left hemisphere channel C3. On an individual basis, the permutation test procedure determined that subjects 3, 5, 7, and 8 produced in at least one channel a slope of regression, α, which was significantly greater than that produced by the random null distribution, indicating a significant positive correlation between mu amplitude and motor-imagery task success (Table 3.1). Non-linear regression produced no significant cosine fits between mu phase and trial performance. Finally, only subject 8 displayed a significant correlation between the 53

Table 3.2: Simulated gating results

Simulated gating using the three gating variables. Columns Acco and BRo give the accuracy and bit rate for the online operation of the device. BRmax gives the 95% conﬁdence interval of the highest bit rate achieved when a percentage of trials up to 70% were gated 500 ms after cue presentation over the 50 iterations of the simulation. Bolded ranges mark where a signiﬁcant improvement in bit rate was achieved by the simulation. SMR amplitude SMR phase MFV Acc BR BR BR BR Subject o o max max max (%) (bits/min) (bits/min) (bits/min) (bits/min) 1 80.9 4.45 (4.49-4.57) (4.41-4.43) (4.45-4.47) 2 76.1 3.10 (3.06-3.07) (3.10-3.19) (3.06-3.12) 3 86.7 6.51 (6.52-6.59) (6.45-6.47) (6.48-6.51) 4 69.4 1.67 (1.65-1.67) (1.65-1.67) (1.64-1.68) 5 64.1 0.87 (1.40-1.46)(0.89-0.92) (0.86-0.87) 6 67.5 1.36 (1.40-1.43) (1.34-1.36) (1.33-1.34) 7 76.9 3.31 (3.24-3.30) (3.30-3.34) (3.27-3.29) 8 97.8 12.70 (12.77-12.83) (12.61-12.63) (13.03-13.09)

MFV gating variable, with high values of the gating variable in both channels PO3-FC3 and PO4-FC4 corresponding to greater trial success.

Trial Gating Simulation

Table 3.2 gives the 95% confidence interval of the bit rates achieved by each subject in the offline simulation of gating with each of the three potential gating features. In most instances where the permutation test indicated a subject would benefit from gating trials of low value, the simulation of offline gating yielded an increased bit rate. Bolded values show where the mean bit rate achieved by the 50 simulation repetitions was significantly larger than the original bit rate (t-test, p < .05). Again, amplitude of the SMRs appears to be the variable most suited for gating the motor-imagery system. Figure 3.7 gives an example of how the accuracy and bit rate change for each subject as a function of the number of trials that are gated based on mu amplitude. On an individual basis, even in this small subject population, there was one subject for which phase gating could have improved the communication rate, and one in which MFV gating was beneficial. 54

S1 S2 S3 S4 S5 S6(d) S7 S8 100 0.25 100 0.25

95 (a) (b) 95 (c) (d) 0.2 0.2 90 90

85 85 0.15 0.15

80 80

0.1 0.1 75 75 Accuracy (%) Accuracy Accuracy (%) Accuracy Bitrate (bits/sec) Bitrate Bitrate (bits/sec) Bitrate 70 70 0.05 0.05 65 65

60 0 60 0 0 20 40 60 0 20 40 60 0 20 40 60 0 20 40 60 Percentage of trials removed (%) Percentage of trials removed (%)

Figure 3.7: Average accuracy and bit rates from 50 repetitions of the gating simulation. a) Average accuracy achieved for each subject after random gating of trials up to 70%. b) Resulting bit rate. c) Average accuracy achieved for each subject after gating of trials up to 70% based on mu rhythm amplitude at the time of the cue. d) Resulting bit rate.

3.2.4 Discussion

Prospects for gating features

In BCI, single trial identification of user intent is a difficult task; nevertheless, we have shown that ongoing oscillatory activity of the motor cortex contains additional information that can be used to produce a modest but significant increase in motor-imagery task performance. Of the three features explored, the technique of selecting for trials with high mu amplitude was the most consistently useful to limit decision making to times when users are most likely to perform higher quality imagery. The relationship between oscillatory behavior influencing motor excitability is controversial [202], but the evidence is stronger for cortical excitability changes to be correlated with SMR amplitude than with phase. The lack of correlation between the phase of the mu rhythm and subsequent suppression efficacy is in agreement with other studies which found that phase had little effect on mu suppression [202]. Whereas the phase at the time of cue presentation did have an effect on early evoked potential morphology (Figure 3.3e), it is not surprising that the effects do not propagate 1-3 seconds after the cue, during which mu suppression is evident. On the other hand, all subjects displaying significant correlation between mu amplitude and BCI task success performed better when the amplitude of mu was high over channels C3 and C4 at the beginning of the 55 trial. This could be due to two factors. Our classifier was based on reduction of mu from a baseline level to an attenuated level, and as a result, trials with larger baseline mu amplitude have a greater potential for mu power reduction. In addition, the amplitude of the mu rhythm reflects the state of excitability of its generating network [202]. While our finding is in disagreement with studies that point to a desynchronized cortex as one primed for motor output [186, 202], Blankertz et al. showed that a successful predictor of imagery performance across subjects was an increased resting state mu rhythm [166]. In a follow up study, they showed that this performance increase was associated with larger recruitment of motor and premotor regions. They concluded that recruiting more synchronized neurons for motor imagery led to higher resting mu amplitude as well as higher performance [203]. In our study we show that not only is this relationship between resting mu amplitude and imagery performance valid between subjects, but also on a single-trial basis within subjects. The third tested gating variable did not produce enough evidence to warrant its use as a potential metric for predicting motor-imagery success. Risner et al. showed that, following phase-matched correction of VEP data using unstimulated controls, average VEPs in different phase groups display the same VEP morphology [204]. We also found this to be the case for early evoked potentials, although after phase-matched control trial correction, there was some phase dependence in later evoked potentials in certain individuals (Figure 3.4, subjects 3 & 7). Because the MFV was calculated from phase- corrected VEP data, the value of this metric can be interpreted to be free of oscillatory alpha bias and instead can be associated with attention to the visual task. However, using a subject-specific VEP template to define the MFV, we were unable to find a consistent correlation between this marker of attention and the subsequent modulation of motor rhythms. Although subject 8 did display a relationship between high MFV and good performance, overall there was no group-wide trend to support this result. This has two important implications for BCI research. The first is that this type gating may provide performance improvements on an individual level. Notably, this gating mechanism significantly improved the already high bit rate of the highest performing BCI user in this study, possibly by gating the rare trials in which the user was not attentive to the task. The second is that fixation to the visual cue is not critical for task success in most users. For these users, control over the BCI communication device should not be limited by 56 their ability to directly fixate with cues on the screen, a critical allowance in the case of severe oculomotor impairment.

Trial Gating Simulation

The results of the gating simulation closely matched the results of the permutation test. Those subjects who performed better during high mu amplitude trials also achieved the greatest benefit from gating trials with low mu amplitude, with the exception of subject 7. However, more subjects benefited from gating than indicated by the permutation test. This may be due to the fact that a multivariate classifier was used to rank the trials as opposed to the permutation test run on each individual channel. The strong improvements seen in subjects 5 and 6 are also partially due to the low initial accuracy and bit rate of online BCI operation. This type of gating holds the biggest potential for improvement for subjects who do not regularly perform at a high level. For high performing users, the increase in accuracy needs to be substantial for this method to be useful.

Study Limitations

Although the oscillation of cortical rhythms is associated with the depolarization and hyperpolarization of large groups of neurons, at the level of EEG the linkage with motor cortical excitability may be tenuous; at this level of measurement from scalp, both ex- citatory and inhibitory neurons contribute to oscillatory behavior [205]. Consequently, broadly recorded signatures of rhythmic activity may be unable to describe excitability in a local group of motor neurons responsible for hand imagery. This is especially true when we attempt to use the excitability of the motor region as a predictor of SMR modulation that occurs on the order of 500-1000 ms later. The lack of findings for the MFV gating variable could be due to poor characterization of individuals’ VEPs. The recording parameters included a 0.1-30 Hz bandpass filter, which is narrower than the clinical recommendation of 1-100 Hz for identifying individual VEP peaks [198]. As a result, these peaks may have undergone some attenu- ation and blurring, and may not have been the ideal template for assessing attention with small variations in peak amplitude. A substantial limitation comes from the use of feedback in our study. Because users 57 were experiencing feedback, we cannot assure that good and bad trials were unlinked. Bad feedback on a trial may have led to subsequent good performance on the following trial. This is a limitation of the retrospective offline analysis performed in this study and these findings need to be replicated in a real-time adaptive scenario in a future study. Lastly, we seek personalized algorithms to improve performance in individual participants. Although across a population of individuals mu suppression was the most consistently helpful feature to base gating on, we did observe that phase as well as MFV were both capable boosting the information transfer rate of the system for certain users. For particular individuals, creating multivariate gating system based upon features that contribute to that person’s physiology may be more accurate than using single features. Such an exploration of combining features may be useful to explore in larger populations. CHAPTER 4

PERSONALIZATION - USER TO SYSTEM

The work in this chapter stemmed from a clinical rotation course I took within the ALS clinic at the Hershey Medical Center in the spring of 2012. My work up to that point had focused on the implementation of BCI systems in college-aged individuals, but I had a goal of designing these communications systems for patients in need of them. The knowledge I gained by this rotation, by interactions with Dr. Simmons, the nurses, staff, and most importantly the patients, led me to formulate a goal of study in this clinic, one of patient-centered BCI device personalization. The necessity for individualized BCI was apparent to upon observing the heterogeneity of symptoms experienced by patients. Most surprising was the presence of cognitive deficits that were identifiable even to my untrained eye, which, as mentioned in Chapter 2, affect a significant proportion of ALS patients. The lack of research studying the effects of cognitive decline on BCI use was the motivation for pursuing this line of study. However, as the work in the clinic progressed, additional clinical variables were studied for their influence in BCI utility. This chapter is split into three parts which focus on the larger goal of BCI personalization in the face of disease heterogeneity. In it, physical, psychological, and even genetic factors are assessed for their influence on a patient’s willingness to accept a BCI-AAC, and the utility of such a system to establish an alternative channel of communication. 59

4.1 Desire for BCI use among ALS participants is affected by behavioral health [3]

4.1.1 Introduction

Successful use of BCI devices has been documented in individuals with ALS [153, 206], but adoption of this technology as an AAC tool has been largely absent outside the research setting. This may be due to the fact that relatively few researchers assess the suitability of the technology in this population, particularly with respect to consideration of the extra-motor limitations of ALS patients. Widespread dissemination of this technology will require an understanding of, and accommodation for, the heterogeneity of ALS, which extends beyond the motor deficits hallmark to the disease. As described in Section 2.2, cognitive deficits accompanying ALS can affect verbal fluency, attention, language, visual and verbal memory and learning. A substantial portion of patients exhibit signs of the behavioral variant of FTD, which is characterized by altered regulation of interpersonal conduct and emotional blunting, which can be described by caregivers as uncharacteristic irritability, selfishness, or disinterest [114]. Certain behavioral abnormalities, such as inflexibility of thought and resistance to new ideas, which can occur prior to motor manifestations [207], have been observed to be associated with AAC device rejection [149]. Huggins et al., [208], showed that BCI technology was well received, but that the devices fell short of meeting patients’ requirements for acceptable levels of accuracy and speed. However, determination of cognitive and behavioral function were not performed in those patients, nor were they given a chance to use a BCI systems before giving their assessments. We conducted an initial survey on BCI acceptance, a BCI training protocol, and a follow-up survey to identify the factors that contribute to device acceptance. We hypothesized that patients exhibiting behavioral impairment would be less likely to have a favorable opinion of these technologies on the initial survey, as a consequence of increases in apathy and mental rigidity [114, 156]. We also hypothesized that the success a user experienced while using a BCI would influence their opinion on the follow-up survey of the utility and practicality of a BCI-AAC for long-term use. 60

4.1.2 Methods

Study Procedure

All patients attending a single multidisciplinary ALS clinic were informed of the study, and those who met inclusion and exclusion criteria were offered the opportunity to participate. Inclusion criteria were: 1) Age 18 years or older; 2) Diagnosis of definite, probable, probable laboratory-supported, or possible ALS by revised El Escorial research criteria [209]. Those with clinically significant dementia, as determined by the ALS clinic neurologist, were excluded. The caregivers were one of the individuals who accompanied the patient to their appointment at the clinic, often a spouse, relative, or full-time caregiver. Patients and their caregivers both consented to the study which was approved by the Institutional Review Board of the Penn State Hershey College of Medicine. The initial visit was followed by a three-month interim period, during which a subset of patients volunteered for a pilot BCI protocol. After completing this protocol, or at the time of their next clinic appointment three months later, participant pairs attended the follow-up session. During each session, patients were seen by the attending neurologist, and their ALS Functional Rating Scale - Revised (ALSFRS-R) was recorded. Demographic data, including age, education level, gender, time since symptom onset, phone use, computer use, and travel distance to the clinic were also recorded. The ALS Cognitive Behavioral Screen (ALS-CBS) was administered to patients and caregivers. This five-minute screen has two components which assess behavior and cognition in ALS. The cognitive portion of the exam is quartered into sections assessing attention, concentration, tracking, and initiation, while the behavioral questionnaire asks caregivers about alterations in personality and behavior in the affected individual. These items are particularly important indicators of frontotemporal dysfunction in ALS [114, 121]. This instrument has been validated against a full neuropsychological battery to detect cognitive or behavioral impairments in ALS patients [210]. Patients were categorized as cognitively impaired if they scored below 17 (cognitive screen, range 0-20), and behaviorally impaired with a score below 37 (behavioral questionnaire, range 0-45). FTD was defined by a score of less than 11 on the cognitive portion and/or less than 33 on the behavioral portion. Scores for each portion of the ALS-CBS were averaged across the initial and follow-up sessions. 61

Following the screening procedure, patients and caregivers were introduced to BCI technology through a ten-minute demonstration by the investigator. This approach was used to produce more informed responses on the subsequent survey. The introduction was standardized through use of a prepared presentation given by the same investigator. The screening investigator answered participants’ questions to the best of his knowledge, and sought to standardize responses across participants. Topics covered were: mechanism of control, potential applications, current communication efficacy, instruc- tions for daily use, and successes in ALS, with focus on P300 and motor-imagery-based spelling systems. Patient and caregiver participants then completed a survey concerning their opinion of BCI-AAC technology (patient survey in Appendix A). Patients rated how often they had used a phone and computer in the month before their diagnosis. They reported if they had ever used assistive devices for verbal or written communication, mobility, feeding, or respiration, and also reported the improvement in their quality of life with use of these devices. They were asked to rate their interest in using a P300- or motor- imagery-based BCI system for communication use on a scale from 1 to 5. On the same scale, they rated their interest in using a BCI for ten potential functions, which ranged from television control to robotic arm use, as well as the importance of the features of the BCI system, such as appearance, accuracy, and invasiveness of electrode. Finally, patients were asked about their requirements of the system, such as level of accuracy and speed. Questions on the survey concerning system features, desired functions, and system requirements were modeled from a previous survey [208]. Caregivers completed a shorter survey with similar questions, covering their level of technology use as well as their interest in having their loved one use a BCI as an assistive tool. The patients who took part in the pilot BCI study were seen for four sessions over the course of 1-2 months, and then the survey was re-administered. During the sessions of the pilot BCI protocol, patients underwent application of an EEG cap, and completed two 30-minute tasks: a P300 speller, and two-class motor-imagery cursor control. For the P300 speller, participants were instructed to mentally count when a target letter was flashed. For the motor-imagery task, participants were instructed to perform kinesthetic imagery of their left or right hand when cued. Both protocols involved a training run, in which a classifier was built using stepwise linear discriminant analysis based on the features extracted from the EEG data [206]. These features were the time averages of target 62

Initial Session (n = 42 pairs)

ALS-CBS ad- No follow up mininstered

n=10 BCI Introduction

Survey 3 month interval

n=10 n=22

ALS-CBS ad- BCI pilot study mininstered

Survey

Figure 4.1: Forty-two patient and caregiver pairs enrolled in the study. During the initial session, the researcher administered the ALS-CBS screen, gave an introduction to BCI technology, and had the patient and caregiver complete surveys. In the three month period between initial and follow-up sessions, 22 patients participated in a BCI pilot study, 10 opted not to participate in the BCI study, and 10 dropped out of the study. During the follow-up, the screen and survey were re-administered. and non-target trials for the P300 task, while for the motor-imagery task, the frequency- band powers of the SMR for left and right trials were used. In the remaining testing runs, participants received feedback as output from the classiﬁer; they were shown the selected letter in the case of the P300 task, and observed the movement of the cursor on the screen for the motor-imagery task. For the purposes of this study, the main outcome of the pilot BCI protocol was the accuracy generated online by each participant for each task over the testing runs. A diagram of the study procedure is given in Figure 4.1.

Survey Analysis

We employed the Wilcoxon rank sum test to determine whether the occurrence of cognitive or behavioral dysfunction responses altered patients initial opinions of the desired 63 functions, features, and requirements of a BCI. This was done for all patients who were deemed either cognitively normal or impaired, and behaviorally normal or impaired based on the results of the ALS-CBS. No distinction was made for patients who scored low enough to be indicated as having possible FTD. Tests were corrected for multiple comparisons by a conservative Bonferroni adjustment. Secondly, a logistic regression model was used to determine which independent variables contributed to the main outcome of acceptance, the decision to participate in the BCI pilot study. Data from all patient participants was used in the logistic regression model. Confounding variables which were highly correlated were eliminated before applying the data to the model. The statistical package R was used to compute the logistic regression, and the log-odds ratios are used to describe the relative importance of each of the factors for influencing the acceptance outcome. Forward and backward selection techniques were used to evaluate model fit based on the Akaike Information Criterion (AIC). For clarity, continuous variables were normalized before performing the regression. Finally, we tested whether user opinions changed based on their perceived success at using the device. Participants of each BCI task were deemed “high performers” if the accuracy they produced online was above the median level. The Wilcoxon rank sum test was used to test for whether changes in participant’s interest level on the follow-up survey was significantly different between high and low performers. Only data from participants who completed the BCI pilot study were included in this analysis.

4.1.3 Results

Patient Characteristics

Forty-two patients participated in the initial session of the study, all but one with a caregiver. The demographics of the patient sample are given in Table 4.1. The fraction of patients presenting with bulbar, limb, and respiratory symptoms was 23.8%, 71.4%, and 2.4%, respectively. Twenty-six (61.9%) patients exhibited cognitive impairment, while 15/41 (36.6%) exhibited behavioral impairment. Ten patients (23.8%) were classiﬁed as having both cognitive and behavioral impairment, and seven patients (16.7%) were identiﬁed as having possible FTD. Abnormalities in cognition and behavior were not correlated with one 64

Table 4.1: Patient characteristics. Median and range for 42 patient participants.

Patient age, years 59 (42-81) Time since symptom onset, mo. 30 (9.5-143) ALSFRS-R 30.3 (7-46) Gender, % male 59.5% Education, years 14 (11.5-24) Distance to clinic, miles 29.2 (0 -147) ALS-CBS cognitive score 16 (4-19) ALS-CBS behavioral score 38 (18-45) another, the average scores on each test component did not change signiﬁcantly between testing sessions, and there was no correlation between the results of the ALS-CBS and education or age.

Survey Results

The majority of patients (78.6%) used some form of assistive technology in their daily living, with mobility devices being the most prevalent (69.0%) and highly rated for improving quality of life (2.7/3 improvement rating). Fewer people used assistive technologies for verbal communication (26.2%), and ratings of quality of life improvement were lower for this type of technology (2.3/3 improvement rating). Patients’ rankings of important system features identified accuracy, variety of functions, and standby reliability (robustness against false-positive decision making) as items with the highest median response (Figure 4.2, left). Patients rated computer and wheelchair control as desirable functions, but rated robotic arm use and temperature control as the functions of least interest (Figure 4.2, right). Caregiver-reported ratings of BCI functions mirrored patient’s responses. Most participants (59.5%) required that the BCI system be at least 80% accurate, with 90.5% of respondents satisfied with a system that could produce 90% accuracy. Nearly three-fourths of respondents would be satisfied with a speed between 15 and 19 letters per minute. Fifty percent were willing to undergo 21 minutes or more of setup time. Many responders (38.1%) opted for an ideal training time that was short (between 2 and 5 sessions), while few (16.7%) were willing to invest more than 20 sessions to become trained. Fifty percent of respondents indicated they would tolerate the system incorrectly leaving standby mode (standby un-reliability) once every five hours or more (Figure 4.3). 65

BCI system features BCI system functions Accuracy Bed Control Appearance Computer Use BCI Functions Lift Setup Simplicity Light Setup Time Wheelchair Speed Recline Standby Reliability Robotic Arm Training Location Temperature Training Time Speaker Phone Type of Electrodes TV Control 1 2 3 4 5 1 2 3 4 5 Importance Interest

Figure 4.2: Patient survey responses for the initial session. Boxplots are shown with a diamond indicating the mean, the vertical line at the center of the box as the median, with the box extending to the 25th and 75th quartiles. Outliers are shown as + signs. (Left) Patient rating of the importance of features of a BCI system. Ratings ranged from 1 being “Not important to me” to 5 being “Very important”. (Right) Patient rating of their interest in the potential functions for a BCI assistive technology. Ratings ranged from 1 being “Not interesting to me” to 5 being “Very interesting”

Impact of cognitive and behavioral impairments

On an individual item basis, ratings of system features, functions, and requirements were not significantly altered by the presence of either cognitive or behavioral deficits in patients. However, aggregate ratings of BCI functions, or the sum of reported interest in all functions, were affected (Figure 4.4). Of all patients surveyed, those with behavioral abnormalities were likely to have lower interest in the BCI for its suggested functions (χ2 = 24.56, p < .05). Cognitive impairment significantly affected these ratings in the opposite direction (χ2 = 14.87, p < .05), with impaired individuals rating their interest in BCI functions as higher than those with normal cognitive function.

Factors predicting BCI acceptance

Examination of the data led us to combine the 12 sub-scores of the ALSFRS-R into three aggregate scores covering bulbar, motor, and breathing function. Phone use was eliminated as a factor due to its low variance among responders. Of the remaining factors tested, Figure 4.5 shows those that were retained after model selection. High computer use and bulbar function positively contributed to participation in the second study, while 66

20 (a) 10 0 <60% 60% 70% 80% 90% 95% 100% Minimum Accuracy

20 (b) 10 0 <5 5-9 10-14 15-19 20-24 25+ Minimum Speed (letters per minute)

20 (c) 10 0 <10 10-20 21-30 31-45 46-60 60+ Maximum Setup Time (minutes)

20 (d) 10 0 1 2-5 6-10 11-15 16-20 20+ Maximum training sessions

20 (e) 10 0 <15 min 30 min 1 hr 2-5 hrs 5+ hrs Leave Standby (once per given time)

Figure 4.3: Histograms of requirements from BCI devices, as rated by patients from the initial session.

1 1 Behavior Normal Cognition Normal Behavior Impaired * Cognition Impaired * 0.5 0.5 responses Percent of total 0 0 1 2 3 4 5 1 2 3 4 5 Function Interest Function Interest

Figure 4.4: Aggregate responses for interest in all the potential BCI functions as a percentage of total responses. Distributions of responses are separated by those who were determined to be normal or impaired in the behavioral (left plot) and cognitive (right plot) domains. Patient ratings of desired functions were signiﬁcantly affected by abnormal cognition and behavior, with impairment in these domains producing higher and lower reports of interest, respectively. 67

Computer Use

FRS bulbar

Male

Behavior

Travel Distance

Education

-20 0 20 40 Log-odds ratio

Figure 4.5: Logistic regression model for predicting participation in the BCI pilot study. Log-odds ratios and their confidence intervals for individual factors are shown in rows. Factors with significant effect on pilot study participation are darkened. maleness, behavioral score, driving distance, and education all negatively contributed. Of these, the factors which significantly contributed to pilot study participation were bulbar sub-score (p = 0.013), driving distance (p = 0.015), and education (p = 0.005).

BCI use affects opinion

For those who did not participate in the pilot BCI study, there was no change at follow- up in interest in the P300 or motor-imagery BCI systems for home use, the potential functions of the device, or the features of the system. BCI study participants were likely to change their rating of interest based on their perceived performance (Figure 4.6). This is clearest for the motor imagery task, where poor performers decreased their stated interest in the system more than high performers at a level near signiﬁcance (p = 0.063 after correction for two comparisons). Although this trend was not signiﬁcant for the P300 task, high performers increased their rating overall in the follow-up session, while low performers decreased their average rating. For those who did not participate in the pilot BCI study, there was no change at follow- up in interest in the P300 or motor-imagery BCI systems for home use, the potential functions of the device, or the features of the system. 68

P300 Motor-Imagery

10 10

8 8

6 6

4 High 4 High 2 Interest 2 Interest

0 0 st) st) po ) Low po ) Low . ( pre . ( pre erf . ( st) erf . ( st) P rf po ) Interest P rf po ) Interest w Pe . ( re w Pe . ( re Lo w rf . (p Lo w rf (p Lo Pe rf Lo Pe rf. gh Pe gh Pe Hi gh Hi gh Hi Hi

Figure 4.6: Patient-reported change in interest in the two BCI systems after use in a pilot protocol. Response groups were separated by performance as well as pre- and post-BCI pilot study. Changes in opinion from pre- to post-session reached signiﬁcance between the high and low performers on the motor-imagery task, with low performers reporting signiﬁcantly lower interest than high performers at follow-up.

4.1.4 Discussion

Despite the relatively small size of this study, our patient population appears to be representative of the ALS patient population as a whole. The median age, the proportion of men, the fraction presenting with bulbar symptoms, and the proportion demonstrating cognitive and behavioral deficits were consistent with published data [114, 211, 212]. Our main findings demonstrate that factors of cognitive and behavioral health contribute to a patient’s interest in pursuing BCI technology for assistive communication. After using a trial BCI device, changes in interest reflected the patient’s perceived level of performance. Patients require that a BCI system be multifunctional, accurate, and robust against false positives. System appearance was generally unimportant, whereas other factors, such as preference in electrode type, showed great variability in responses. Some rated electrode type as a very important feature in their decision to use a BCI (desired minimal invasiveness), while others did not find this feature of great importance (were open to more invasive electrode types). The desired accuracy levels of 80-90% are achieved currently using both types of BCI systems, and the cap setup of 10-20 minutes that is 69 current standard practice would be acceptable to many of our patients. Although a training period of 2-5 sessions is sufficient for the P300 speller, a motor-imagery training program generally takes upwards of 20 sessions [153]. An aspect identified by a majority of users for improvement is a desired communication speed of 15 letters per minute. Although this speed is elusive for a scalp-based motor-imagery system, for evoked potential BCIs, speeds of at least 15 letters per minute have been accomplished [213]. The results of this survey were largely in agreement with the literature [208]. A few of the discrepancies between studies are likely due to differences in item description by the research teams. For example, the importance of standby reliability and the utility of a robotic arm were descriptions that likely varied between investigators. The relative lack of interest in those with behavioral impairment in using a BCI is consistent with the apathy and mental rigidity seen commonly as signs of behavioral dysfunction associated with ALS [114, 156]. Those with cognitive impairment were more interested than those with normal cognition, driven by the large percentage of respondents in the impaired group that indicated high interest (score 5). Patients with FTD have notably poor insight and judgment [214, 215], and often fail to acknowledge physical and cognitive deficits. Cognitive deficits associated with FTD lead to reduced capability for abstraction [214]. These two factors may have led to unrealistically positive assessments. As anecdotal support, we have found that patients with cognitive impairment generally rate all aspects of their quality of life as being the highest possible, and often describe their physical deficits as being minimal. The finding that psychological impairment parallels BCI acceptance is of high clinical relevance, as these clinical features influence the course of technological intervention decided on by the patient, caregiver, and physician. In our multivariate analysis, education and driving distance were negative predictors of pilot study participation, while high bulbar function was a positive contributor (Figure 4.5). Long travel distances were an understandable deterrent to attending the four study sessions, while highly educated individuals could find the devices insufficient for their needs and frustrating to use. Alternatively, highly educated individuals may still be in the workforce and unable to participate in the pilot study, or they may have other activities that occupy their time. High bulbar function as a positive factor was somewhat surprising, given that we expected patients with speech difficulties to be more eager to participate. However, bulbar involvement has also been associated with behavioral 70 impairment [156]. It should be mentioned that certain aspects of patient life, such as work and family commitments, were not assessed. Although they may have had an impact on the patient’s decision to participate, these factors remain unresolved in this study. This study has limitations of course. The relatively small sample size of the survey group limits the power of the significant findings, and the method of sampling patients from the clinic on a voluntary basis introduces a bias towards opinions which are favorable to BCI technology. The use of a full neuropsychological screen would have been more accurate in gaging the cognitive and behavioral health of the participants, however the ALS-CBS was used mainly due to limited time during the clinical visit. Finally, the training period of four sessions for the BCI protocol was chosen to provide participants with a practical demonstration of the utility of the communication interface. While this may have been long enough for the P300 speller system, an extended training period may have been required for the users to reach their optimal level of performance with the motor-imagery system. Ratings of the motor-imagery system may have suffered due to this insufficient training period. Critically, this study showed how perception of success caused a change in patient opinion of the devices. Participants who performed well using the systems tended to maintain or increase their interest in pursuing one of the devices, while those who did not achieve satisfactory performance saw a marked decrease in their desire to use the device. This finding indicates the need for trials of these expensive BCI-AAC devices early in the design stage of device development. Such trials, with a patient’s perception of performance, will likely determine whether the patient accepts and will continue using the device. The observed changes in opinion also underline the importance of BCI- informed respondents in the collection of survey data relating to perceived usefulness of such devices. 71

4.2 Cognitive impairment negatively impacts BCI use [4]

4.2.1 Introduction

People with ALS have for some time been considered primary candidates for BCI communication systems. Although the capacity to control a BCI has been shown to be relatively conserved in these patients, only marginal communication has yet been established for those experiencing CLIS of late-stage ALS [44]. One explanation for unsuc- cessful BCI use in CLIS is based on the loss of goal-directed thinking behavior by the user. This phenomenon, proposed as a type of cognitive impairment associated with the ALS/FTD syndrome, results from the complete lack of motor control and subsequent feedback [180]. In this study, we explored the effect of cognitive and behavioral impairment, as well as other clinical variables, as possible indicators for losses in performance in two BCI tasks: the P300 speller and motor-imagery cursor control. The amplitude of the P300 decreases and latency increases with normal aging [216, 217, 218]. ALS has been shown to further prolong the latencies of the P300 response for visual and audio paradigms [133, 141, 219]. Some [133, 220], but not all [141] groups report lower amplitude responses in patients compared to controls. The degree of functional motor impairment, as measured using the ALSFRS-R, was not found to be correlated with P300 abnormality [141, 206, 220, 221]. The altered morphology of the oddball evoked response in ALS has been suggested to result from reduced focus and attention [219] as well as abnormal memory processing [222], and possibly even a failure of cognitive association [141], although there are conflicting results on whether cognitive impairment associated with ALS leads to P300 abnormality [220]. Persons with ALS have also been shown to be able to suppress their SMRs for the purpose of operating a motor-imagery BCI [153], although baseline SMR levels are of lower amplitude in patients with ALS [223] and FTD [224]. Efficacy of SMR modulation is not associated with physical functional disability [180], nor is the locus of motor weakness correlated with the ability to form useful imagery of that limb [155]. Bul- bar involvement, however, has been shown to negatively influence imagery ability and therefore BCI performance [155, 223]. The goal of this study was to identify ALS-related factors that contribute to success 72 using P300 and motor-imagery BCI systems. We expected that cognitive and behavioral impairments would impact successful BCI operation more than physical factors. For the P300 speller, we anticipated that altered cognition would weaken BCI performance due to reduced attention, consistent with the literature linking both ALS and FTD to a reduced P300 response. For the motor-imagery BCI, we expected baseline SMR spectral power levels would be the main predictor of BCI performance, as reported previously [166], but also expected that cognitive impairment would be detrimental to achieving the SMR suppression required for device control.

4.2.2 Methods

Study Procedure

A subset of twenty-five patients from the survey study described in Section 4.1 were enrolled in this study. All met the criteria for definite, probable, probable laboratory- supported, or possible ALS, and were absent of overt dementia. Fifteen control participants were recruited from the nearby area by means of a community bulletin. In addition to being neurologically healthy, control participants were age and gender matched to the patient group. The study protocol was approved by the Institutional Review Board of the Penn State Hershey College of Medicine and all participants provided informed consent. Patients had demographic and clinical variables recorded, and were administered the ALS-CBS, as described in Section 4.1. Control participants were also administered the cognitive portion of the ALS-CBS. Participants were categorized as cognitively impaired if they achieved a score below 17 on the cognitive portion (range 0-20), and were behaviorally impaired if they scored below 37 (range 0-45) on the behavioral assessment. The ALS-CBS was administered before and after the BCI study procedures, and the two scores were averaged. Patient participants completed four sessions of BCI recordings over the course of 1-2 months, with each hour-long session split between two BCI paradigms: a P300 spelling system, and a two-class motor-imagery center-out task. Control participants completed two sessions. Each run of the P300 spelling task consisted of copy spelling a four letter word, where each trial culminated in the selection of one letter after flashing each grid icon 20 times. A checkerboard-type speller with 32 targets was used to evoke the P300 signal from the user [80]. Targets were highlighted for 187.5 ms, and the inter- 73 stimulus interval was 62.5 ms. Targets were randomly assigned into groups of four at the beginning of each letter presentation, and the order of groups was randomized as well. Group order was constrained so that no target flashed consecutively, i.e. a specific letter flashed no more than once every half second. During each session the participants also performed one calibration run in the ‘covert’ spelling setting; they were instructed to focus their eyes on the center of the letter matrix, which contained a fixation cross, and to only direct their attention to the target letter in the grid periphery. P300 responses were defined by amplitude and latency in channels Fz, Cz, and Pz. P300 amplitude was the maximum magnitude over a baseline of the average EEG response in these channels 250-500 ms after the stimulus [78]. Latency was the time after the stimulus at which this maximum occurred. Each run of the motor-imagery task consisted of ten left and right trials, in which the subject was instructed to perform kinesthetic imagery of their left and right hands, as well as ten no-go trials, during which they were instructed to relax. The first run of each session entailed calibration of the classifier without feedback. In the remaining feedback runs, real-time classification of brain features enabled the cursor to move freely from the center of the screen under the control of the user. During the recordings, EEG electrodes were affixed in an electrode cap at nineteen locations in the 10-20 system, with ground at Fpz, and referenced to linked earlobes. Additionally, for purpose of artifact reduction, electrodes were placed on the forehead and lateral canthi to record the electrical activity of eye movement. Signals were amplified and digitized with two g.USBamp amplifiers. Data acquisition, signal processing, and feedback generation were performed by a customized program in BCI2000. Aside from the specifics of the feature extraction step, the processing pipelines for the two systems were very similar, and included artifact reduction and rejection, feature extraction, and classification. Ocular artifact reduction was automated though a regression procedure as described in Section 3.2, followed by rejection of epochs which exceeded an amplitude of 75µV. Features used for the P300 spelling system were the stimu- ± lus time-locked average EEG signals, downsampled to 20 Hz. The features extracted during the motor-imagery task were the spectra in 2 Hz bins from 5-30 Hz from each Laplacian-referenced EEG channel. Classifier generation was implemented using the stepwisefit.m function in MATLAB, which utilized stepwise selection of regression coefficients on training data to generate a classifier for predicting the two target classes 74 online [80]. This classifier was used to generate real-time feedback for the user, and was updated after each feedback run.

Analysis

The online classification accuracy for each task was reported as the percentage of trials finished correctly in the last two runs for each session. For most individuals, this meant that the P300 accuracy statistic was calculated from the percent letters out of 32 (4 sessions with 2 runs spelling a 4 letter word) that were spelled correctly in the online session. The motor-imagery task online accuracy was calculated from the percentage of trials out of 160 (4 sessions with 2 runs of 10 left and 10 right trials) in which the cursor finished on the correct side of the screen. For patients and control participants who completed fewer than four sessions, accuracy statistics were calculated from a reduced number of trials. We also wanted to define the differences in EEG features for the control and patient groups. For the P300 task, we report latency and amplitude of the evoked potential under overt and covert spelling conditions. For the motor-imagery task, we calculated the BCI performance predictor established by Blankertz et al. [166]. This predictor was calculated as the difference of the amplitude of the power spectrum and the fit of a 1/ f noise spectrum in the SMR range. Statistical comparisons of sample means between groups were handled with the Wilcoxon rank-sum test. In addition to reporting accuracy and feature differences between patients and controls, we defined a classifier-independent approach to determining the robustness of the signals recorded from the participants, which we call ‘quality’. The rationale for con- sidering feature quality was to distinguish the potential for BCI aptitude from a performance metric based on an individually-trained classifier. The quality of our two tasks was calculated as the standard distance between data belonging to the two classes of interest: left and right trials in the case of motor-imagery, and target and non-target trials for the P300 speller.

As an example, QC3, the quality of motor-imagery in channel C3, was calculated as the standard distance between left and right average power spectra,

µC3,L µC3,R QC3 = − , (4.1) q 2 2 σC3,L + σC3,R 75

where µC3,L was the average power spectrum for left trials in the channel C3, and σC3,L was the standard deviation over these trials. Motor-imagery quality was defined for each individual and Laplacian-referenced channel for frequencies 5-30 Hz. Quality was defined similarly for P300 as between the target and non-target averages 250-500 ms after the stimulus. To determine whether quality of the P300 and motor-imagery signals were dependent on physical or psychological factors, we performed multiple linear regression, with the quality scores as the dependent variables. The independent variables of interest were the motor, bulbar, and respiration sub-scores of the ALSFRS-R, as well as the behavioral and cognitive scores of the ALS-CBS. Each of these factors were normalized before performing the regression. Model selection was performed through minimization of the Akaike Information Criterion (AIC), a measure that reflects the goodness of t of the model as well as its complexity. The model selection procedure was performed in R [225] using both forward and backward feature selection algorithms. The factors which contributed to a minimized AIC through both forward and backward feature selection were retained in the final reduced model.

4.2.3 Results

Demographics for patients and controls are given in Table 4.2. Controls were well matched for age and gender, but had more years of formal education. Of the 19 patients that provided information on region of onset, 2 presented with bulbar onset, 13 with limb onset, 3 with simultaneous bulbar and limb onset, and 1 with simultaneous limb and respiratory onset. One participant did not complete the ALS-CBS because of severe communication difficulties, and another participant did not complete the behavioral portion of the screen because of the lack of an available caregiver. Among patients, the median cognitive score on the ALS-CBS was 16 (range 9-19), with 14/24 (58%) defined as cognitively impaired. The median behavioral score was 37.5 (range 18-45), with 8/23 (35%) defined as behaviorally impaired. The median cognitive score for the control participants was 17.5 (range 15.5-19), with 3/15 (20%) defined as cognitively impaired. The distribution of cognitive scores in the two groups was quite different. Abnormal controls were just below the cutoff, consisting of 2 scores of 16 and one of 15.5. In contrast, patient 76

Table 4.2: Patient and control participant demographics. Statistics are given as median and range (min-max), if applicable, and p-values compare control and patient samples.

Patients (n=25) Controls (n=15) p-val Age, years 58 (45.5-74) 55 (45-73.5) 0.364 Education, years 14 (11.5-24) 18 (12-24) 0.006 Gender, % male 68 60 0.624 ALSFRS-R 30 (0-46) TSSO, months 32 (12-113) ALSFRS-R = ALS Functional Rating Scale - Revised, TSSO = Time since symptom onset. scores were skewed towards the lower end of the range, with nine patients scoring lower than the poorest performing control subject. Overall cognitive scores were signiﬁcantly higher for the control group (p = .013). Exceptions to the recording procedure described in the methods included two patients (P06, P28) who completed only one session of the study and one patient (P23) who completed three sessions. One control participant (S14) also only completed one session. Electrode impedances often exceeded the recommended 5 kΩ level [12] for individual participants. Of the 19 EEG electrodes attached during each session, 81% registered impedance values less than 10 kΩ, and 90% were less than 30 kΩ. There was no correlation found between electrode impedance and the EEG features relevant to the BCI tasks, nor the performances achieved on those tasks.

P300 Speller

Although P300-BCI performance varied within patient and control participant groups, control participants on average performed at a higher level (p = 0.033, Figure 4.7). The average accuracy achieved by the patients on the P300 task was 68% (range 0-100%). For control participants, the mean accuracy was 86% (range 38-100%). The P300 quality for patients was generally highest in channels along the midline, and strongest in the central electrodes (Figure 4.8b). Timing of the target VEP varied among individuals as well, with most participants having a large positive deflection around 200 ms which was sustained through 400 ms (Figure 4.8c). Many individuals also had a large negative deflection sometime after 400 ms. Control participants had similar topology and timing of P300 quality. The average amplitude of the patients’ P300, that is the largest positive deflection 77

1 Patients (a) Controls 0.9

0.8

0.7

0.6

0.5 Motor-imageryaccuracy 0.4

0.3 0 0.2 0.4 0.6 0.8 1 P300 accuracy

(b) 1 * 1 *

0.5 0.5 MI onlineMI accuracy

P300online accuracy 0 0 Impaired Normal Controls Patients Patients

Figure 4.7: (a) Online accuracy for patients ( ) and control participants ( ) for the P300 and motor-imagery BCI systems. Dashed lines∗ denote the expected accuracy◦ of random performance on each task. (b) ANOVA comparison of task accuracy by participant group. Patients with cognitive impairment had reduced performance on both tasks, sig- niﬁcantly lower than controls on the P300, and lower than cognitively intact patients on the motor imagery task ( indicates pairwise difference at p < 0.05 level.) ∗ of the VEP in channel Cz during the 250-500 ms window after the target stimulus, was 3.0 µV( 1.5 µV), with latency occurring 334.2 ms ( 77.9 ms) after the stimulus. ± ± Control participants had a slightly larger mean P300 amplitude than patients of 3.7 µV ( 1.5 µV, p = 0.105), although their mean latency of 330.7 ms ( 77.7 ms, p = 0.723) ± ± was not different. P300 latency for patients was similar to a previous report, but the amplitude was signiﬁcantly lower (4.06 µV from [206], p = 0.003 for a one-sample t-test). Compared to the overt spelling task, the mean latency of the P300 during the covert spelling condition was longer for both patients (399.3 72.7 ms, p = 0.004) ± and controls (426.3 81.7 ms, p = 0.002), but the amplitude was unchanged for both ± groups. Regression was performed to determine whether the mean absolute value of P300 quality in channels Fz, Cz, and Pz occurring 250-500 ms after the stimulus was correlated with any patient characteristics. Of the factors evaluated, cognitive score of the ALS-CBS was the only variable to positively correlate with P300 quality (Figure 4.9, 78

(a) (b) 0.1 0.2 0.3 0.4 0.5 (c) 0.2 0.4 0.6 0.8 (d)

P01 P01 P02 P02 P05 P05 5 P06 P06 P08 P08

V) P09 P09 Fp1 Fp2 µ P11 P11 P12 P12 P15 P15 Fz 0 P16 P16 F3 F4 P19 P19 P21 P21 F7 F8 P23 P23 Amplitude ( P26 P26 P27 P27 -5 P28 P28 0 0.5 1 P29 P29 T7 C3 Cz C4 T8 P30 P30 P31 P31 P34 P34 P36 P36 P37 P37 P38 P7 P8 0.4 P38 P3 P4 P39 P39 Pz P43 P43

Fp1 Fp2 F7 F3 Fz F4 F8 T7 C3 Cz C4 T8 P7 P3 Pz P4 P8 O1 O2 0 0.5 1 0.2 O1 O2 Channels Seconds after stimulus

Quality 0.4 0.2 0 0.3 0.15 0.2 0 0.5 1 0.1 Time (seconds after cue) 0.1

Figure 4.8: Quality of the evoked response between target and non-target trials. (a) Top: Target (red) and non-target (black) averages in channel Cz for participant P27. Bottom: Quality is defined as the sum of the absolute value of the standard difference between the target and non-target trials in a specified time window, shaded here for the 250 500 ms period. (b) Qualities in each channel at this time period are given for each participant.− (c) Qualities at each time point are also shown for each participant, as the (d) channel average of qualities in Fz, Cz, and Pz (shown here for participant P12). p < 0.05). However, the stepwise model selection procedure, which added and removed factors to minimize the AIC, retained both cognitive and behavioral scores in the final model. Both of these scores were positively associated with P300 quality, indicating that participants who scored high on the psychological screen were more likely to perform well on the spelling task. A break down of sub-scores on the cognitive portion of the ALS-CBS indicated that deficiencies in tasks of attention and tracking were correlated with poor P300 feature quality (r2 = 0.178, p = 0.04 and r2 = .173, p = 0.04).

Motor-imagery task

Group-wide performance was lower on the motor-imagery task (Figure 4.7), with an average accuracy achieved by the patients of 60% (range 45-97%). Five out of the 25 patient participants were able to control the motor-imagery center out task at the 70% level suggested for productive use. For control participants, the mean accuracy was 62% (range 46-90%), with 3 of the 15 control participants achieving 70% accuracy. For the ﬁve patients who were successful in motor-imagery control of the center- out task (P09, P21, P31, P34, P39), channels C3 and C4 produced the highest quality 79

P300 Motor-imagery

FRS FRS respiration respiration

FRS FRS bulbar bulbar

FRS FRS motor motor

Cognition Cognition

Behavior Behavior

-0.1 0 0.1 0.2 0.3 0.4 0.5 -1 -0.5 0 0.5 1 1.5 2 Regression Coefficient Regression Coefficient

Figure 4.9: Cognition positively predicts BCI signal quality. (Left) Regression of P300 quality on patient characteristics. Bars indicate the mean regression coefficients for each factor, along with 95% confidence interval of this estimate. (Right) Regression with motor-imagery quality as the dependent variable. motor-imagery responses (Figure 4.10b). In these individuals, both mu and beta band sensorimotor rhythms were modulated to control the cursor (Figure 4.10c). In other individuals, there was little differentiation in SMR power between trial types (P16, P27), or there was an absence of defined SMRs altogether (P02, P29). The BCI performance predictor, which reflects the prominence of these resting state rhythms, was positively correlated with the quality of the motor imagery signal (r2 = .717, after removal of outlier P16). We found no difference in the BCI performance predictor between patients and controls. The difference in motor-imagery quality between channels C4 and C3, averaged over the 5-30 Hz range served as the dependent variable for the regression on the disease factors. Again, cognitive score of the ALS-CBS was the only variable to positively correlate with motor-imagery quality (Figure 4.9, p < 0.05). The model reduction procedure retained only cognitive score as the factor which minimized the AIC. A breakdown of the sub-scores of the cognitive screen revealed that tasks involving attention and word initiation were significantly correlated with the quality of the motor-imagery features (r2 = 0.26, p = 0.01 for both sub-scores). In some patient participants, strong power in gamma-range frequency bands consistently appeared in frontal and temporal electrodes. The ratio of spectral power in the 40-50 Hz range to task-relevant power in the 8-24 Hz range was calculated in channels Fp1, Tz, and Cz. An ANOVA determined there to be a significant difference in this ratio between the cognitively-impaired patients, cognitively-normal patients, and controls (p < 0.05 in each channel). Subsequent pairwise comparisons revealed that patients 80

(a) (b) -0.4 -0.2 0 0.2 0.4 0.6 (c) 0 1 2 3 (d) )

2 40 V µ

20 P01 P01 Power ( P02 P02 0 P05 P05 P06 P06 P08 P08 1 P09 P09 P11 P11 P12 P12 0 P15 P15 P16 P16 F3 F4 Quality P19 P19 -1 P21 P21 P23 P23 0 10 20 30 40 50 60 P26 P26 Fequency (Hz) P27 P27 P28 P28 P29 P29 C3 C4 P30 P30 P31 P31 ) 2 40 P34 P34 V

µ P36 P36 P37 P37 20 P38 P38 P39 P39 P3 P4 Power ( P43 P43 0 F3 F4 C3 C4 P3 P4 0 20 40 60

1 Channels Frequency (Hz) 0.1 0.4 0 0 Quality -1 0.2 -0.1 0 10 20 30 40 50 60 Fequency (Hz) 0 Figure 4.10: Quality of SMR discrimination between right and left tasks. (a) Quality is deﬁned as the mean standard distance between left (blue) and right (red) spectra in the 5 30 Hz range. (b) Qualities in each channel are given for each participant. (c) Qualities− at each frequency are also shown for each participant. (d) The frequency plots in (c) are calculated from the mean of the difference of the qualities in channels C3 and C4 (shown here for participant P34). with cognitive impairment possessed an elevated power ratio compared to the other two groups. (Figure 4.11). Task dominance, or differential success using one of the systems, was observed in many participants. A metric was generated which described the dominance produced by each participant as the residual of the regression of motor-imagery task accuracy on P300 task accuracy. The main factor which predicted task dominance was the participant’s age, with older subjects performing relatively better using the P300 system than the motor-imagery system (Figure 4.12).

4.2.4 Discussion

Of the factors considered for this study, those related to cognitive function, rather than physical function, were determined to be a major predictor of successful BCI operation. Patients with ALS who scored higher on the cognitive portion of the ALS-CBS were more likely to achieve higher accuracies using the P300 and motor-imagery systems. A possible mechanism for this loss in performance in patients with cognitive impairment 81

1.5 1.5 1.5 Impaired * Normal 1 1 * 1 Controls * 0.5 0.5 0.5 * Power Ratio

0 0 0 Fp1 T7 Cz

Figure 4.11: Cognitively-impaired patients display elevated ratios of power in the 40 50 Hz band over the average power in the 8 24 Hz band during no-go trials of a motor-− imagery task. ANOVA identified significant− differences between cognitively-impaired patients, cognitively-normal patients, and control participants in channels Fp1, T7, and Cz (p<0.05). Significant pairwise differences between groups at the p<0.05 level are indicated by an asterisk. was a decrease in the signal-to-noise ratio within task-relevant EEG frequencies. There was also evidence that behavioral dysfunction negatively affects P300 speller performance. Finally, there was a tendency for older participants to achieve relatively better performance with the P300 system. Riccio et al. [226] asserted that attention is an important factor in predicting P300 BCI performance. Specifically, those who are better at detecting a target letter (self- reported) in a serial visual stream are also better at the P300 task. They attribute this to the ability to update an attentional filter. In our paradigm, we found the quality of the P300 consistent throughout the length of the trial and from the first run of the session to the last, even for patients who scored poorly on measures of attention, indicating that the length of the trial or the session was not a determinant of the quality of the P300 control signal. Poor performance in those with impaired cognition may have been due to deficits in ability to update their attentional filter to recognize the target stimulus. This would be consistent with the executive function deficits seen in FTD, which are typically demonstrated through tests which measure shifting of cognitive set, initiation of behavior, inhibition of inappropriate responses, and control of attention [227]. The prefrontal damage that occurs in ALS can often lead to the presence of reduced focus and attention, hinder- ing the production of a stable P300 [222]. However, we cannot rule out the possibility that visual dysfunction may have played a role in the low performance experienced by some patients, as reported recently [99]. 82

The morphology of the evoked poten- 100 tials between patients and controls was re- 90 r2=0.330, p=0.003 markably similar in both overt and covert 80 70 conditions, albeit with a non-signiﬁcant 60 reduction in the amplitude of the P300 50

Motor-imagery Online Accuracy (%) Online Motor-imagery 40 in the ALS sample as a whole. Among 0 20 40 60 80 100 P300 Online Accuracy (%) patients, P300 amplitude and latency were not signiﬁcantly affected by physi- 80 r2=0.238, p=0.013 cal function, measured by the ALSFRS- 70 R, nor did we observe an increase in 60 P300 latency with age. This may have

Patient Age (years) Age Patient 50 been partly due to limited age range com- 40 pared to studies which reported this effect -20 -10 0 10 20 30 40 Equi-dominance residual [216, 217, 218], as well as the smaller P300 Dominant Motor-imagery Dominant sample size. The mean latency of the Figure 4.12: Older patients perform relatively better on P300 systems. (Top) Scatter P300 found in this study was similar to plot of patients’ online accuracy for the two the 360 ms previously reported [206], al- tasks, with equi-dominance linear fit. (Bot- though the amplitude was lower. Both of tom) Individual residuals from the equi- these metrics are difficult to compare be- dominance line is negatively correlated with associated age. cause of the differences in trial design and stimulation protocol between studies. In addition, the lower amplitude might be due in part to the higher electrode impedances in this study. The observed increase in latency of the P300 from the overt to the covert attention condition reflects the loss of the 200 ms peak during the covert condition, with results very similar to Treder et al. [191]. The extinction of the fixation-related portion of the evoked potential during the covert trial results in a subsequent decrease in BCI system performance [191, 228]. Cognitive impairment was the major disease-related factor that impaired the effective use of the motor-imagery system. Motor-imagery is dependent on the user’s ability to self-generate imagery of a detailed motor action from memory. The user needs to repeat this imagery when cued, as well as switch imageries quickly, a task heavily dependent on executive function. As in the case of the P300, the sub-score of attention was a particularly important indicator of motor-imagery quality, while the initiation sub-score 83 was also positively correlated with signal quality. Both findings implicate executive dysfunction as detrimental to success on the motor-imagery task. Motor dysfunction, as measured by the motor sub-score of the ALSFRS-R, was not correlated with motor- imagery performance, consistent with previous reports [155, 180]. Although age was not found to be a significant factor in predicting BCI performance overall, we did find that there was task dominance for the P300 system in older individuals. This finding implicates P300 systems as potentially more useful for older users, while the motor- imagery systems may be better utilized by younger users. In contrast to previous findings [223], we did not observe a significant difference in the resting state power of the mu rhythms in ALS patients compared to controls, although we did observe an increase in the ratio of task-irrelevant to task-relevant power in patients with cognitive impairment (Figure 4.11), which is in agreement with other studies that showed a reduction of motor rhythm amplitudes in FTD [224]. These findings lead us to believe that the changes in the brain specific to ALS-associated cognitive impairment may reduce the utility of a motor-imagery BCI because of the decrease in signal to noise ratio of the sensorimotor rhythms. On the other hand, with sufficient training, patients without cognitive impairment may be able to modulate SMR well enough to achieve accuracy levels comparable to healthy individuals. Our short training period of four sessions may have been inadequate for the participants to achieve optimal results. The equally poor performance of the control participant group on this task leads us to believe that, given a longer period of training, a greater number of patients could achieve motor-imagery control. The main findings of our study lead us to make to some recommendations for BCI communication devices in ALS. First, the younger individuals in our study demonstrate a dominance for the motor-imagery BCI paradigm, while the older individuals perform better with the P300 system compared to the motor-imagery system. Although performance is only one factor for determining BCI device adoption, this age-paradigm interaction may be a useful tool for recommending systems to patients of different ages. Second, we have shown that both of the standard BCI paradigms tested fall short of sup- porting useful communication for cognitively impaired patients. However, patients with cognitive impairments are just as likely as those without such impairments to desire a BCI for assistive communication use [3]. This presents a meaningful clinical problem if we are to offer BCI systems to patients with ALS, because up to half of individuals 84 with ALS have cognitive or behavioral impairment. For these users, the design of the BCI will have to accommodate for reduced cognitive ability, or they may prove inop- erable. When designing assistive devices, cognitive load can be alleviated by reducing memory loads and distractions and providing information in more intuitive forms [229]. Modifications to BCI systems have been attempted using centralized, serial stimulus presentation (RSVP), and language prediction models [230], as well as using symbols representing common daily activities [231]. Similar modifications will need to be tested for the potential to improve communication for ALS patients with reduced cognitive capacity.

4.3 Repeats of hexanucleotide G4C2 in C9ORF72 correlate with quality of BCI performance [5]

4.3.1 Introduction

In modern health care, the term “personalized medicine” is often associated with treatments and therapeutics which are targeted towards an individual, often on the basis of one’s genetic diversity. The treatment of ALS is no different, and the genetic anomalies that shape the specific course of illness may help to define subsets of patients who will be receptive to new pharmaceuticals, or in the case of the following work, assistive communication devices. As mentioned in Chapter 2, mutations in a number of genes have been implicated in a growing fraction of the total ALS cases. Of these, some have been shown to also contribute to the development of frontotemporal dementia. The repeat expansion in the gene C9ORF72, found in open reading frame 72 on chromosome 9, is one of the most recent discoveries that has provided another biological link between ALS and FTLD. Abnormal expansion of the G4C2 (GGGGCC) hexonucleotide in this sequence occurs in about 40% of familial ALS and 7% of sporadic ALS cases, while being rare in healthy individuals. C9ORF72 repeat expansion is also prevalent at rates <2% in Huntington’s, Alzheimer’s, and Parkinson’s diseases [232, 233], similar to the trinucleotide repeats responsible for an array of additional neurogenetic disorders [234]. ALS patients possessing the pathologic repeat expansion exhibit earlier onset [232, 235, 236, 237, 238, 239], more rapid progression, and earlier death [236, 237, 239, 240]. Penetrance of the mutation increases with age, from 85 a state of non-penetrance at age 35 to full penetrance by age 80, with nearly 100% of those with the expansion developing symptoms by this time [241, 242, 243]. Among ALS pedigrees, there is evidence for genetic anticipation, with earlier age of onset in the descendants of expansion carriers. [241] The question arises as to what defines a pathological repeat length of the C9ORF72 gene? In the pair of papers that originally established the link between the repeat and ALS-FTD, the cutoff for a pathologic expansion was determined to be greater than 30 hexanucleotide repeats [244, 245]. Polymerase chain reaction (PCR) is capable of determining whether a patient has the repeat length of greater than 30, but is not ideal for determining the repeat length that could be in the thousands. For this, Southern Blotting is typically employed. [232, 233, 238, 246]. To complicate matters, significant somatic heterogeneity has been observed, with different expansion lengths found in different tissues, even between regions of the brain [233, 239, 247]. Still uncertainties remain about the phenotypic differences occurring as a result of long expansions and those of intermediate length (7-30 repeats). One study showed that five FTD patients with repeat lengths of 20-22 were similarly cognitively impaired as the four patients with longer repeats (>30) [248], while another study showed that four patients with repeats of 20-22 were phenotypically similar (age of onset, higher prevalence of dementia) to those with >30 [249]. Also, intermediate numbers were shown to have a risk effect in familial FTLD, with lengths of 12-21 associated with earlier onset and shorter survival [241]. On the other hand, others have found no such correlations between sub-threshold repeat lengths and phenotype in ALS [250]. There has been limited work done to determine the structural and functional brain changes that co-occur with the C9ORF72 repeat expansion. MRI studies point to structural changes common among pathologic repeat carriers [236, 237, 251, 252, 253]. For one, there are gray matter changes that extend beyond the degeneration of the corticomotor pathways typically involved in ALS. Transcranial magnetic stimulation was used to assess cortical excitability in ALS patients [243]. Both those with sporadic ALS and those possessing the repeat expansion displayed increased motor evoked potentials compared to controls, indicating similar pathological hyper-excitability of the cortex. In an EEG study of seven ALS/FTLD patients with the repeat expansion, two showed generalized slowing of the background activity, while another two showed intermittent abnormal delta-theta activity temporally [253]. 86

The question of whether genotype, particularly in this case as it applies to expansion of the C9ORF72, produces measurable changes in the performance of an EEG-based BCI task remains to be seen. In this section, we explore the potential of this genetic marker as a screening mechanism for brain-computer interface utility, and describe how the intermediate repeat expansions found in a sample of ALS patients interplay with cognition and performance on two BCI tasks.

4.3.2 Methods

The majority of this protocol was described in the previous section. Twenty-five patients that were being treated at a multidisciplinary ALS clinic were enrolled in the study. Those with clinically significant dementia, as determined by the ALS clinic neurologist, were excluded. Fifteen age and gender matched control participants were also included in the study. In addition to being neurologically healthy, control participants were age and gender matched to the patient group. The first goal of the analysis was to determine whether the repeat lengths, of intermediate size or otherwise, correlated with any of the physical or phsychological symptoms exhibited by the patients in the study. These include correlations between repeat length and bulbar vs. limb onset, impaired cognition and behavior, as well as age at the onset of symptoms. In order to determine whether a genetic screen may contain predictive value for BCI utility, we also evaluated the interaction of repeat length with two measures of BCI performance in each tasks: online accuracy and signal quality. These outcome variables of accuracy and quality were the same as those described in Section 4.2. Motor-imagery quality was defined for each Laplacian-referenced channel. A single measure of ‘classic’ motor-imagery quality for each participant was determined by averaging the quality differences in the paired channels F3-F4, C3-C4, and P3-P4 over the frequency range 5-30 Hz. Quality was defined similarly for P300 as between the target and non-target averages. The timing of the ‘classic’ quality signal was averaged over the 250-500 ms period following the stimulus, in channels Fz, Cz, and Pz. The quality measures were regressed with repeat length to assess for interaction. Additionally, we defined a cutoff level at the median repeat length in order to determine how groups of users possessing a high repeat length compared to those with a low repeat length. For these tests, we used a 87

3 Count 2

0 0 5 10 15 C9ORF72 Repeat length

Figure 4.13: Histogram of GGGGCC repeat lengths within C9ORF72 in 24 ALS patients

15 15 15 15 (a) (b) (c) (d)

10 10 10 10

5 5 5 5 C9ORF72 repeat length repeat C9ORF72

0 0 0 0 YN YN YN 40 60 80 Cognition Normal Behavior Normal Bulbar onset Age at symptom onset

Figure 4.14: Neither (a) cognitive impairment, (b) behavioral impairment, (c) bulbar onset, nor (d) age at symptom onset were correlated with sub-threshold expansions of C9ORF72. the Wilcoxon rank-sum test, or in the case of comparing these two groups with controls, a one-way ANOVA.

4.3.3 Results

Of the patients enrolled in the study, one participant (P38) did not complete the ALS-CBS because of severe communication difficulties, and another participant (P43) did not complete the behavioral portion of the screen because of the lack of an available caregiver. All twenty-five patients and fourteen of fifteen controls provided samples for assessment of the C9ORF72 repeat expansion, although one DNA sample from a patient 88 was of insufficient quality to perform analysis. Therefore, analysis is performed on the twenty-four patients and fourteen controls with complete genetic results. Overall cognitive scores on the ALS-CBS were significantly higher for the control group (p = .019), with a median score for patients of 16 and a median score for controls of 17.5. The median behavioral score for patients was 38.25 (range 18-45). All patients in the study possessed sub-threshold expansion lengths of 15 hexanucleotide repeats or less (Figure 4.13). These sub-threshold repeat lengths in patients were not found to be significantly associated with cognitive impairment, behavioral impairment, bulbar onset, or age at symptom onset (Figure 4.14). Patients were divided into ‘low-repeat’ and ‘high-repeat’ groups based on a repeat length above or below the median length of eight. Figure 4.15 displays the differences in the P300 evoked potential between patients in the two repeat groups and controls. Controls demonstrated consistently better quality in the evoked potential over a wide range, including early potentials at 200 ms, the classic P300 (250-500 ms after the cue), and even late rebound potentials. We focused on the quality of the classic P300, as it is not as affected by fixation of eye gaze as the 200 ms potential. Classic P300 quality, averaged over the channels Fz, Cz, and Pz was found to be most different between the control group and the ALS group with high repeat lengths. Furthermore, the negative correlation between the repeat length and classic P300 quality averaged over the 250-500 ms range was significant (Figure 4.15c, R2 = 0.21, p = 0.024). Control participants, who achieved the highest performance overall on the P300 system, displayed a significantly higher accuracy rate compared to the high-repeat group as determined by ANOVA and subsequent Tukey post-hoc tests (Figure 4.15d). Although there was a tendency for patients in the low repeat group to achieve higher accuracies on the online P300 task than those in the high repeat group, this difference was not significant (p > 0.05). Figure 4.16 displays the same analysis for the motor-imagery task. Here, the three groups have similar quality, as there was slightly better than random performance for all but a few of the best patient and control participants. The variability of imagery ability among participants yielded little differentiation between the qualities in specific frequencies across groups (Figure 4.16a). When contralateral electrode pairs were averaged to generate a measure of classic motor-imagery quality, controls displayed clear peaks around the typical mu and beta ranges (Figure 4.16b). Patients produced a more distributed band of frequency desynchronization, which led to a broader, al- 89 though smaller amplitude quality of motor-imagery. Averaged over the 5-30 Hz range, the classic motor-imagery quality exhibited a significant negative correlation with gene repeat expansion length (Figure 4.16c, R2 = 0.19, p = 0.033). Although the patients with high repeat lengths averaged lower accuracies on the online motor-imagery task (Figure 4.16d), this difference between groups was not significant (p > 0.05). 90

high repeat (a) low repeat Fz controls

0.4 Pz 0.2

0 P300 Quality P300 -0.2 -0.1 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 Time (s)

pval = 0.023995 15 (d) 0.5 high repeat high repeat low repeat (c) 1.2 (b) low repeat controls 0.4 controls 1 * 0.3 10 0.8 0.2

0.6 0.1 P300 accuracy P300 5 Classic P300 Quality Classic C9ORF72 repeat length C9ORF72repeat 0.4 0

0.2 -0.1

-0.2 0 0 0 0.2 0.4 0.6 0.8 1 0 0.05 0.1 0.15 0.2 0.25 Time (s) mean Classic P300 Quality, 250-500 ms

Figure 4.15: Repeat length is inversely related to P300 quality. (a) P300 quality in channels Fz, Cz, and Pz in high-repeat patients (red), low-repeat patients (blue), and controls (black). Greater values indicate better discriminability of the P300 control signal. Hor- izontal bars of magenta, dark red, and dark blue indicate significant ANOVA pairwise differences between high-low, high-control, and low-control groups at p < .05. Thicker horizontal bars indicate significance after Bonferroni correction for all time windows tested. (b) The classic P300 quality, or the average over the three channels. The range of a typical P300 signal (250-500 ms) is shaded in gray. (c) A significant negative correlation exists between the average of the classic P300 quality in the 250-500 ms range and the length of the repeat expansion among patients. (d) Accuracy on the P300 task by repeat group. The asterisk indicates a pairwise difference between the high-repeat and control groups following a significant ANOVA result. 91

F3 (a) F4

C3 C4

high repeat P3 P4 low repeat 0.2 controls 0 -0.2 MI Quality MI -0.4

0 10 20 30 40 50 60 Frequency (Hz)

pval = 0.032604 15 0.5 high repeat high repeat low repeat low repeat (c) 1.2 controls 0.4 (b) controls

0.3 1 10 (d) 0.2 0.8

0.1 0.6

0 Classic MI Quality Classic 5

C9ORF72 repeat length C9ORF72repeat 0.4 Motor-imagery accuracy -0.1 0.2 -0.2

0 0 0 10 20 30 40 50 60 -0.2 0 0.2 0.4 0.6 0.8 Frequency (Hz) mean Classic MI Quality, 5-30 Hz

Figure 4.16: Repeat length is inversely related to motor-imagery quality. (a) Motor- imagery quality in channels F3, F4, C3, C4, P3, and P4 for the three participant groups. Refer to Figure 4.15 for information about horizontal lines. (b) Classic motor-imagery quality, or the average over the paired differences in quality between F3-F4, C3-C4, and P3-P4. The range of a typical SMR modulation (5-30 Hz) is shaded in gray. (c) A signiﬁcant negative correlation exists between the average of the classic motor-imagery quality in the 5-30 Hz range and the length of the repeat expansion among patients. (d) Accuracy on the motor-imagery task by repeat group. 92

4.3.4 Discussion

All of the patients included in the study possessed a sub-threshold repeat expansion, indicating that none had C9ORF72-linked ALS [245]. Furthermore, these patients did not display any of the signs and symptoms typically associated with this linkage. Pa- tients were no more likely to have cognitive or behavioral impairments, as opposed to those with the pathological expansion who often have co-occurring frontotemporal dementia. Studies of FTD patients possessing intermediate length repeats (~12-22) were associated with similar levels of cognitive impairment, age of onset, and prevalence of carrying high-risk alleles as individuals with longer repeats [241, 248, 249, 254], although this finding was not observed in ALS [250]. Many of the patients in our study fell outside of this intermediate range, possessing repeat lengths of eight or less. Certain findings from this study warrant a further look in a larger group of patients, specifically, the association between BCI task proficiency and C9ORF72 repeat expansion length. Both the quality of the control signals for the P300 and motor-imagery tasks were negatively associated with gene expansion length. The online accuracies produced in these individuals also trended in this direction, although the difference between groups was not clear enough to make claims about performance effects of longer repeat lengths. Given that impaired cognition, a factor shown to impede BCI performance, was not associated with these longer repeat lengths, additional assessment is required to determine the size of this intermediate length effect, as well as a possible causal mechanism. CHAPTER 5

PERSONALIZATION - SYSTEM TO USER

The personalization addressed in the previous chapter focused on identifying patient characteristics that predicted BCI performance, so that a user could be paired with a system which was most likely to result in a compatible interface. As the complement to the previous chapter, this chapter explores personalization of the BCI system from the other direction. Here, we identify ways in which we could personalize the operation of the system to optimally utilize the brain signals generated by the specific user. Person- alization of system operation can be achieved through a number of means. In the data collection stage, selective recording of subsets of electrodes [255, 256] can minimize setup time and discomfort. At the level of feature selection, only the most discriminable time and frequency features could be forwarded to the classifier [185]. Personalization can also occur at the level of the interface, whether by cuing modality [48, 257], speed [258], or number of classes [258, 259], to name a few. The need for personalization in the most general sense was made apparent within the first few sessions of recording in the ALS clinic. Things such as screen adjustment and positioning, system pacing, and electrode subsets were small modifications that were sometimes made so that the device was better able to be used. These changes were often made due to physical restrictions, usually as a result of bulky wheelchairs with on-board monitors, but timing changes to the visual stimuli were also made to accommodate for longer trials or slower flash times. Although these are anecdotal examples, I was able to experience firsthand what difference could be made in BCI performance by small 94 alterations in system parameters. The remainder of the chapter focuses on engineering solutions for device personalization, so that adaptation to individual brain patterns may allow for maximum performance. Additionally, we report trends in personalization that occur across neurologically healthy individuals and ALS patients with and without associated cognitive impairment. In this chapter, two projects are described which were accomplished with offline analysis of BCI data. First, optimization was carried out on features generated from both P300 and motor-imagery tasks. This was done to find how the optimal subsets of electrodes and spatio-temporal locations of class-discriminable brain data changed on an individual and group level. Second, alternative task features related to network connectivity patterns were assessed for classification utility. A simple measure of coherence was compared to a novel data assimilation technique that uses a biophysical neural field model to estimate cortical connectivity. The results of these analyses provide a unique perspective on the opportunities and challenges for device deployment in ALS, and presage future work employing these techniques online to boost BCI-AAC performance.

5.1 Feature Optimization

5.1.1 Introduction

Feature selection is a central component in the BCI system as a process that converts large quantities of EEG data into concise chunks of information to be fed to the classifier. The calculation of features allows for the classifier to work with a lower dimensional training set. This is one way to prevent the classifier from over fitting the training data, and it also speeds up the process of training the classifier. Feature selection can be fixed, by performing one set of transformations on the data to extract similar features for all individuals, or it can be optimized in such a way that the smallest set of the most discriminable features are used. Our lab has explored feature optimization techniques using exhaustive search methods to determine optimal electrode configurations [255], and eigenvalue methods for selecting robust frequency features in a motor-imagery task [185]. In this section, we perform feature optimization offline on data recorded from ALS patients and controls during the P300 and motor-imagery BCI tasks described in 95

Section 4.2. As mentioned in Chapter 2, ALS and FTD produce overlapping patterns of degeneration that affect multiple regions within the central nervous system. Patients with primarily ALS show large white matter changes, while those with FTD exhibit substantial gray matter changes. However, little work has been done to describe the spatio-temporal changes to task-speciﬁc patterns of brain activity that occur as a result of disease state. In the following paragraphs, I describe the anticipated differences in feature optimization to be found between ALS patients and controls, as well those occurring as the result of cognitive impairment.

P300 in ALS

The work in Chapter 4 reviews some of the changes to the morphology of the P300 that occur due to normal aging as well as ALS. The most common changes involve reduced amplitude and longer latency of the P300 that occurs with normal aging, with similar morphological changes occurring in ALS and FTD. Although the topographical changes in P300 scalp distribution have not been clearly studied, Hanagasi et al., demonstrated that both patients and controls display a parietal maximum for the P3b component. Sev- erens et al. also showed similar topography of the P300 between early-stage ALS patients and controls [47]. These studies describe a topographical stationarity to the P3b oddball potential despite significant neurological change. It should be noted that a different expectation occurs for the novelty P300, or P3a potential, which is elicited by a slightly modified novelty detection task. ALS patients display smaller amplitude of P3a over frontal sites, with no effect over central and parietal sites [222]. For this reason, Raggi et al. suggest that novelty P300 paradigms may be better suited to study the disruption in frontal networks accompanying cognitive decline in ALS [222]. When heterogeneity of the ALS population is accounted for, differences in the P300 become apparent. Ogawa et al. showed that patients with bulbar disease onset had significantly delayed P300 peaks compared to those with limb onset [260]. The cognitive decline sometimes co-occurring with ALS may also lead to alterations in the P300. Ac- cording to Vierrege et al., dorsolateral prefrontal and anterior cingulate cortices have a role in the direction of attention. Some of the cognitive deficits found in ALS patients include reduced attention and inhibition, and PET studies have shown that regional blood flow in these brain areas are reduced [261]. FTD can cause delay in the P300 signal, 96 although this delay is not as large as in dementia of the Alzheimer’s type [262]. Others have noted that the delay in the P300 may be substantial in some demented patients, but non-existent in others, therefore ruling out the P300 as a reliable diagnostic tool [263]. However, in the case of BCI feature personalization, identifying longer P300 latencies on an individual basis could make the difference between a functional and non-functional BCI device. Therefore, in addition to the weaker amplitude of the P300 signal, we may expect the timing of the positive peak to be delayed and for compensatory reorganization to shift the location of the maximum in certain individuals with cognitive impairment.

Motor-imagery in ALS

The hallmark of ALS is degeneration of upper and lower motor neurons, but it is less clear how the large-scale, oscillatory activity of primary motor and associative areas of the brain are affected. Most studies have been performed with neuroimaging and blood flow analysis to describe the changes in brain function due to movement action and imagery. fMRI has shown increased activity during motor execution in regions primarily anterior but also posterior to the motor cortex, as well as in areas associated with motor learning, as in the basal ganglia and cerebellum [137, 264]. These changes have been confirmed with positron emission tomography (PET), showing increased activation in auxiliary motor areas, premotor, and parietal association areas during a motor task [265]. These studies point to an expansion of the output sensorimotor zone, which may represent cortical plasticity, or ongoing changes that occur to compensate for a loss of pyramidal cells in the motor cortex [264, 265], an effect that has also been observed in stroke patients [264]. Shifts to ipsilateral sensorimotor cortex that occur in these patients are another example of cortical reorganization [264, 266]. Conversely, lower regional cerebral blood flow was observed in prefrontal areas during both free and stereotyped motor tasks, which may be reflected as a correlate of neuropsychological deficits often found in the disease [265]. Again, disease heterogeneity adds nuance to this narrative. Lule et al. point out that this type of functional reorganization is not as evident in cases of purely lower motor neuron degeneration, and is a result of degeneration of both upper and lower motor neurons [264]. Kollewe and colleagues documented differential fMRI BOLD responses to hand and tongue movements in ALS patients with and without bulbar involvement. In 97 both groups of patients, there was an increase in the number of activated voxels during hand movement, but during tongue movement, only patients with bulbar signs showed reduced activation [267]. This study shows that compensatory mechanisms exist for the neurodegeneration of hand motor areas which do not exist in bulbar regions, possibly explaining the faster disease progression in bulbar-onset ALS [267]. Evidence for a different mechanism is at work in motor imagination. While some studies have shown stronger recruitment of premotor cortical areas, as well as cognitive areas related to motor planning, [268], others have demonstrated reduced BOLD activity in the left anterior parietal lobule, the anterior cingulate, and prefrontal cortex [269]. The reduction in activity seen in imagery may reflect the disruption of networks responsible for imagery outside of the primary motor cortex.

Expected Findings

We performed an exhaustive search for the optimal spatio-temporal features for a P300 and motor-imagery task in our patients. We were interested in finding if there was a difference in the optimal feature set between controls and patients with and without cognitive deficits. We expected to observe an overall decrease in feature robustness in the ALS disease state, but also anticipated a general shift in typical activations (central electrodes for motor-imagery and 300 ms delay for P300) to alternate locations and times, providing evidence that more personalized configurations may be optimal for users with neurodegeneration.

5.1.2 Methods

Exhaustive optimization was performed to determine the most effective feature vectors for classification in 25 ALS patients and 15 control participants. This optimization procedure was carried out for P300 and motor-imagery data separately. To facilitate the optimization procedure, all jobs were run on the Penn State computing cluster, with 12 processors dedicated to each person’s feature set. The full feature set consisted of data from 19 (P300, linked-ears reference) or 6 (motor-imagery, Laplacian referenced) channels. For each trial, there were 5 time- binned EEG amplitude features for P300 or 10 frequency-binned EEG power features for motor-imagery, both of which had been down-sampled to reflect averages within the 98 range of 280-500 ms or 5-50 Hz, respectively. The details of the optimization procedure are described for the analysis of P300 data, although the same method was used for the motor-imagery data. The only difference was the size of the spatio-temporal feature vector for each trial. Data from all four sessions was concatenated into a single matrix. Each row within this matrix contained the average EEG of a single stimulus code in one trial of P300 data. This row was associated with a class label, indicating whether it was a target code or a non-target code, and contained 19 channels with 5 time features evenly spaced over the 280-500 ms period, resulting in 95 features per stimulus code. Two sets of optimization were performed for each task, due to computational restrictions. First, the set of time features were kept constant within each feature vector, while the full search of electrode combinations was performed. For the P300 task, this amounted to (219 1) 524,286 subsets of electrodes containing all time features to be − processed through the classifier. As an example, on the first classification iteration only the first channel was used. The data matrix contained ~1000 rows, each being an EEG average over 10 flashes of a single stimulus code within a trial, and 5 columns, for the five time features in the first channel. This was repeated for all combination of electrode channels. Similarly, a second optimization was performed with the electrodes held constant and the space of the time features searched, for an additional (25 1) 31 clas- − sifications. This method of separate electrode and time search avoided the need to do a full search on all possible combinations of 95 electrode and time features with ~4e28 classifications. The data matrix sent to the classifier contained a 1:8 ratio of target to non-target flashes. Data were parcellated into ten groups in order to calculate and average test set error from the classifier using 10-fold cross validation. In each of the ten iterations of the classifier, a random 90% of the stimulus codes (rows) were used for training the LDA classifier which used a 1:8 prior, while the remaining rows were used for classification. The test set error was defined as the fraction of trials in the test group which were misclassified. This error was averaged over the ten folds of the procedure to determine the average accuracy for that electrode combination. For analysis, the number and type of features, specifically electrodes, time points, and frequencies, which yielded optimal classification were identified. To identify the spatio-temporal differences in feature space among participants, the classification error 99 from the iteration using only a single electrode, time point, or frequency was used. Other methods were attempted which used all subsets up to a certain dimension that included a specific feature, but the resulting trends were simply spatially averaged versions of the single feature scheme. We considered how these optimal feature sets changed across participant groups. Finally, we asked how the optimal feature set could be reduced in each group, while maintaining 80% of the optimal error. As an example, for an individual achieving an optimal error of 5%, the allowable error at a level 80% optimal was determined as (5.1).

Error80% = 12.5 (12.5 Erroropt) 0.8 (5.1) − − × = 6.5%

where the theoretical random classiﬁer error was 12.5%, due to the 1:8 prior. The conﬁguration with the smallest number of electrodes reaching this level of accuracy was chosen as the reduced electrode set.

5.1.3 Results

The exhaustive search of feature space was expedited by the resource of the Penn State computing cluster. A maximum of four jobs were able to be run at a time, with each job allocated 12 cores. These 12 cores were utilized within MATLAB through the use of the parfor parallelized computation loop. The bulk of the processing power went into performing classiﬁcation on hundreds of thousands of feature vectors, and when parallelized in this way, each job experienced a roughly 6-fold increase in speed over the local computer. For the largest P300 datasets, a single subject’s data took roughly 1.5 hours to analyze, and as a whole, the time of total computation was reduced from roughly 384 hours to 16 hours.

P300

The optimization scheme used 10-fold cross validation of an LDA classifier, of which the version in MATLAB defaults to an equal prior on classes when not specified. Be- cause the occurrences of target and non-target classes were unequal, a classifier with 100 equal expectation of each class resulted in a relatively high false positive rate (FPR = 1 - true negative rate). To bias the classifier to produce fewer false positives, the classifiers used in this analysis employed a 1:8 target:non-target prior. As can be seen in Figure 5.1, the classifier with this empirically-calculated prior produced lower overall error, due to the significant improvement in the true negative rate (TNR). The first observation, apparent in Fig- ure 5.1, was that the performance of the

30 Classifier with no priors Empirical (1:8) Target:Non-target prior classiﬁer improved with increasing com- 20

10 plexity, or number of electrodes and time

0 Test set error rate (%) rate Test set error 1 2 3 4 5 6 7 8 9 10 11 12 13 points included in the feature vector. The Number of electrodes in classification No-prior classifier Empirical prior classifier exception to this trend were those individ- 1 1 uals who had a limited number of trials 0.8 0.8 in their feature matrix due to fewer test- 0.6 0.6 ing sessions. The relatively smaller num- 0.4 0.4 ber of trials made the classifier over fit 0.2 0.2 Classification rate of test set TPR (sensitivity) the data when greater numbers of features TNR (specificity) 0 0 Increasing classifier complexity were used, resulting in lower test set performance. Figure 5.1: Error among patient participants The critical task for this optimization by classifier complexity. Top: The classifier that employed an empirical prior (calculated exercise was to find whether the optimal from the relative frequencies of the training electrode locations were different depend- data) achieved consistently lower test set er- ing on the study group. The cross vali- ror. Bottom: The reduction in error came from a drastic improvement in the TNR, at dation error for each subject that resulted the expense of the TPR. from using only a specific electrode is shown in the top of Figure 5.2. The differences in classification across electrodes within most individuals was relatively small, although in some (S14, P39) the differential in test set error due to electrode location was quite large. When visualized across groups, some differences appeared between controls and patients (Figure 5.2), although ANOVA statistics for each electrode revealed no significant differences in electrode errors between groups, when Bonferroni corrected for 19 comparisons. In the control participants, the locus of the P300 signal had a distinct posterior maximum, with little discriminative information contained in the frontal channels, typical of a P3b oddball potential. The central, parietal, and 101 occipital regions defined the optimal regions for classification in these 15 individuals. On the other hand, in the ALS disease state, a more distributed response resulted from individuals benefiting from both frontal channels (P16, P34) and parieto-occipital channels (P31, P39). This distributed response occurred in the cognitively impaired group as well, although the general quality of the signals was lower and the resulting error higher in nearly all channels. Control and patient participants displayed similar levels of optimal accuracy (Figure 5.3, left). Data were examined to determine what subset of the optimal configuration would be required to produce at minimum 80% of the optimal accuracy. The majority of those in the ALS group could achieve the accuracy at 80% their max using the same number of electrodes as control participants (Figure 5.3, center), with high performers in both groups generating 80% optimal performance using anywhere from 50-80% fewer electrodes (Figure 5.3, right). The reduction in required electrodes appears to scale with performance, indicating that there is high information content over few electrodes in the high performers, and little information content over many electrodes in the low performers. Finally, optimal times for classification within the P300 task were examined by con- sidering features from five time bins. Errors achieved by the classifier using only features from one time point are given for each participant in the top of Figure 5.4. Error was minimized for the control group during the time bins centered on 325 and 375 ms, and the lowest errors achieved by ALS patients were in the 475 ms window. The accuracy at each time point was not significantly different between groups, due to the high variability of accuracy within groups (Figure 5.4). 102

S01 S02 S03 S04 S05 S07 S08 S09

S10 S11 S12 S13 S14 S15 S16 P01c

P02c P05cb P06cb P08c P09b P11 P12c P15c

P16 P19cb P21b P23cb P26 P27c P28 P29b

P30c P31 P34 P36cb P37c P38** P39 P43c*

Controls Cog. Normal Cog. Impaired Fp1 Fp2 F3 Fz F4 F7 F8

T7C3 Cz C4T8

Absolute Error Absolute P7 P8 P3 Pz P4 O1 O2

Figure 5.2: (Top) Error in each electrode for each participant. Errors are on the same scale, which shows relative performance between subjects. (Bottom) Absolute error in each electrode, averagedControls over participantCog. groups. Normal Red colorsCog. indicate Impaired high error rates, blue colors indicateFp1 lowFp2 error rates F3 Fz F4 F7 F8

T7C3 Cz C4T8

ErrorRank P7 P8 P3 Pz P4 O1 O2 103

14 14 14 Controls 12 12 12 Patients 10 10 10

8 8 8

6 6 6

OptimalError 4 OptimalError 4 OptimalError 4

2 2 2

0 0 0 0 5 10 15 0 5 10 15 -100 -50 0 Number of channels used Number of channels % reduction in channels for optimal error to maintain 80% optimal error to maintain 80% optimal error

Figure 5.3: P300 error from reduced electrode sets in patients ( ) and controls (o). (Left) Optimal error achieved over all electrode combinations plotted∗ against the number of channels used in that optimal combination. (Center) The optimal error vs. the minimum number of channels needed to maintain 80% of the optimal error level. (Right) Optimal error vs. the percentage reduction in channels of the reduced set.

S01 S02 S03 S04 S05 S07 S08 S09

20 20 20 20 20 20 20 20

0 0 0 0 0 0 0 0

S10 S11 S12 S13 S14 S15 S16 P01c

20 20 20 20 20 20 20 20

0 0 0 0 0 0 0 0

P02c P05cb P06cb P08c P09b P11 P12c P15c

20 20 20 20 20 20 20 20

0 0 0 0 0 0 0 0

P16 P19cb P21b P23cb P26 P27c P28 P29b

20 20 20 20 20 20 20 20

0 0 0 0 0 0 0 0

P30c P31 P34 P36cb P37c P38** P39 P43c*

20 20 20 20 20 20 20 20 Error (%) Error 0 0.28 0.33 0.38 0.42 0.47 0 0 0 0 0 0 0

Control Cog. Normal Cog. Impaired 15

5 Test set error rate (%) rate Test set error

0 0.28 0.33 0.38 0.42 0.47

Time bin (s)

Figure 5.4: (Top) Error at each time point for each participant. (Bottom) Error at each time point, averaged over participant groups. 104

Controls Cog. Normal Cog. Impaired Fp1 Fp2 F3 Fz F4 F7 F8

T7 C3 Cz C4T8

Absolute Error Absolute P7 P8 P3 Pz P4 O1 O2

Control Cog. Normal Cog. Impaired 60 Controls Cog. Normal Cog. Impaired Fp1 Fp2 40 F3 Fz F4 F7 F8

20 T7 C3 Cz C4T8 Test set error rate (%) rate set error Test 7.5 12.5 17.5 22.5 27.5 32.5 37.5 42.5 47.5 52.5

ErrorRank 0 P7 P8 P3 Pz P4 O1 Freq.O2 bins (Hz) Figure 5.5: Electrode errors in each participant group from features from individual channels (top) and frequencies (bottom), on the same scale. For the electrode maps, red colors indicate high error, and blue low error.

Motor-imagery

Analysis for the motor-imagery task was performed in the same way, except feature vectors were constructed from Laplacian-referenced EEG data that had been transformed into frequency space and the spectrum from each trial parcellated into 10- 5 Hz bins from 5-50 Hz. Similar to the results of the P300 optimization, increasing the complexity/channels fed into the classifier resulted in a lower error rate, although the error rates as a whole were higher in this task, near random level for many individuals. Due to the inconsistent performance of users on the motor-imagery task, these results should be interpreted cautiously. Comparing participant groups, the cognitively impaired patients produced the highest error rates, while the cognitively normal patients produced the lowest error rates. The control participants, possibly because of the shorter training period of two sessions, performed within the range of the two patient groups. The relatively poor accuracies seen across groups masked the good performance of a minority of participants in the control 105 and cognitively normal groups. Averaging electrode performance over groups shows the relative difference in electrode importance between the three groups (Figure 5.5, top row). In cognitively normal individuals, the source of motor-imagery signals was in the C3/C4 pair of electrodes. These are the electrodes which putatively sample the activity of the primary motor cortex, which, along with the pre-motor cortices, are the primary generators of imagery signals. An ANOVA indicated that there was a significant difference in the accuracy of the three groups due classification based on features extracted from channel C4, after Bonferroni correction for the six channels tested. A post-hoc analysis revealed that the controls and cognitively normal ALS patients achieved higher accuracy from the features extracted from this channel than those with cognitive impairment. The cognitively impaired patients, while performing overall at a random level, seemed to achieve little discriminability of motor-imagery signals in any channel tested. Optimization was also run to evaluate which frequency features were most useful for discrimination of classes. The results are less clear in this case, but useful frequency features tended to occupy the mu and beta ranges (7.5 Hz, 12.5 Hz, 17.5 Hz, 22.5 Hz, and 27.5 Hz). Again, increasing the number of frequency features generally improved classification, but as in the case of the P300, this effect plateaued. Averaging over groups, a large frequency band over in the SMR range facilitated the classification of motor-imagery (Figure 5.5, bottom row). No groups appeared to use frequencies above 35 Hz to control the motor-imagery BCI, and an ANOVA indicated that there was a significant difference in the accuracy of the three groups due to classification based on features in the frequency bin centered at 22.5 Hz, Bonferroni corrected for the ten frequency bins tested. A post-hoc analysis revealed that the cognitively-normal ALS patients achieved higher accuracy from the features extracted at this frequency. There were no obvious patterns for electrode retention at 80% optimal accuracy, but a large percentage of controls and patients saw no electrode reduction at the 80% level, due to the fact that one or two channels made up the optimal set (Figure 5.6). On the other hand, there were quite a few individuals who would be able to reduce the number of channels by half in order to maintain 80% of their optimal error level. These individuals were of a wide range of proficiencies, and included control as well as patient participants. 106

50 50 50

40 40 40

30 30 30

20 20 20 OptimalError OptimalError OptimalError

10 10 Controls 10 Patients

0 0 0 1 2 3 4 5 1 2 3 4 5 -80 -60 -40 -20 0 Number of channels used Number of channels % reduction in channels for optimal error to maintain 80% optimal error to maintain 80% optimal error

Figure 5.6: Motor-imagery accuracy from reduced electrode sets in patients ( ) and controls (o). (Left) Optimal error achieved over all electrode combinations plotted∗ against the number of channels used in that optimal combination. (Center) The optimal error vs. the minimum number of channels needed to maintain 80% of the optimal error level. (Right) Optimal error vs. the percentage reduction in channels of the reduced set.

5.1.4 Discussion

The reason for performing feature optimization for a BCI task is two fold. First, it allows the classifier to account for diversity among individuals by utilizing unique features, which are sometimes outside the realm of ‘normal’ ranges expected for these tasks. With this optimal feature set, the user can perform at a higher level than if the feature extraction were a fixed process. Additionally, the dimension reduction accomplished by feature selection makes simple linear classifiers more robust to over-fitting, decreases online computation time, and requires the use of fewer sensors. The optimization performed in this section is relatively inelegant compared to other feature reduction schemes, although the method used guarantees finding the feature set producing the lowest error. The obvious drawback is the increase in computational demands, which was mitigated by parallelizing the optimization procedure. As is, this technique is reasonable to do in a multi-session BCI environment, as the current procedure took at most three hours per participant. However, if immediate feature reduction is required, stepwise algorithms for feature selection are regularly used, as well as dimensionality reduction techniques, both of which have their downsides. The optimization performed over electrodes and time points in the P300 task indicate that there may be subtle differences in the optimal feature set produced by ALS patients. As expected from the literature, the times closer to the end of the 500 ms window were most useful for classification in patients, while times in the 300-400 ms window yielded the lowest error in controls. Additionally, ALS patients exhibited a wider distribution 107 of P300 features in frontal electrodes, while in controls, discrimination between classes was exclusively found in centro-parietal regions. For all performers, the relative benefit to feature reduction scales with performance. For high performers, the use of many channels produces redundancy. These users can maintain high levels of performance even after eliminating up to 80% of the electrodes, while low performing users need an increased feature space to maximize the P300 averaging effect. Not much can be conclusively said about feature differences across groups in the motor-imagery task. This is due to the majority of individuals unable to effectively modulate their motor rhythms in response to hand imagery. The differences between groups represent the biases introduced by one or two successful motor-imagery users, averaged over the larger cohort of near-random performers. Among high performers in control and patient groups, the channels with greatest discriminability were the central electrodes C3 and C4. Although there was no observed shift in frequencies or electrode locations of discriminable data in the motor-imagery task due to ALS, there is substantial reduction of the feature space possible by utilizing only these two channels for classification.

5.2 Features describing neural connectivity for BCI

5.2.1 Event-related coherence changes

Introduction

To define the neural features that accompany a mental task, the majority of current BCIs use individual sensor activations; firing rates from local populations of neurons, regional metabolic changes, or the transient changes in the brain’s electromagnetic field recorded at the level of the cortical surface and scalp. By limiting features to the activity of individual sensors, information about neural interaction is lost. With that in mind, diverse methods have been used to define regional coupling, causality, and synchrony in EEG data. Reviews on the nature and applicability of these methods, which range from coherence [94], Granger causality [270], phase synchronization [271], as well as nonlinear descriptors of synchrony [272] are given elsewhere [273, 274, 275, 276]. Evaluation of these tools in the analysis of nonlinear EEG data has revealed potential pitfalls in their interpretation [276], stemming from the inherent variability in 108 the resting-state network compared to changes due to functional coupling. However, when combined with traditional BCI metrics, connectivity features have been shown to provide an additional boost in performance [277, 278]. In this analysis, we consider a simple measure of coherence, in comparison to traditionally used power features utilized in a motor-imagery task. We determine whether there are significant differences between features across participants and whether this translates to improved classification of imagery state.

Methods

Based on the ﬁndings of the online BCI study described in Section 4.2, we were interested in determining the utility of coherence features across three groups: control participants, cognitively normal patients, and cognitively impaired patients. Of interest was whether features describing the interaction between electrode sites in the mu and beta bands provided additional information for classiﬁcation, and how this information varied across group type. Spectral features were calculated using Welch’s method, implemented in MATLAB. Power spectral density features were calculated from the same six Laplacian-referenced channels used online, while coherence was calculated for each of the 19 electrodes referenced to linked earlobes. For each method, the time period from 0-3 seconds after the presentation of the cue was used, encompassing the assumed period of imagery. Win- dows of one second length were passed through a hanning window, and the periodogram calculated. The window was shifted by half a second and the process repeated, with the output being the power spectral density, or average of these periodograms normalized by the sampling frequency. The magnitude squared coherence was calculated as the ratio of the cross spectral density magnitude to the product of the autospectral densities.

2 Pxy( f ) Cxy( f ) = | | (5.2) Pxx( f )Pyy( f ) In addition to describing changes in power and coherence among participants, we compared the accuracy achieved in ofﬂine classiﬁcation using these features in a 10- fold cross validation scheme. The averages of the test set errors over the ten folds using typical power features vs. coherence features were compared. There were twelve features in the case of band power, accounting for the average band power in the six 109 channels over two frequency bands. Although there were 30 total coherence features between the 6 channels F3/4, C3/4, and P3/4, this was reduced to 9 features, retaining all connections to and from channels C3 and C4. Accounting for both frequency bands, this was 18 features total for the coherence data matrix. This reduction in the size of the coherence feature vector was done to minimize error discrepancies between feature types due to the size of the feature vector. For each individual, cross validation error was determined for each of the feature sets, and an error differential was calculated as the difference between the two errors. This error differential was compared across patient groups using a one-way ANOVA to determine whether there was an advantage of using coherence or power features depending on the user’s neurological state.

Results

The power differences across left and right trials followed the pattern expected from SMR desynchronization, with the best performers achieving differential power changes in multiple channels per hemisphere (Figure 5.7). Significant differences in mean power between left and right trials at the p < 0.05 level, as determined by repeated t-tests, are shown as colored circles at the electrode, with larger circles indicating where this interaction was significant after the test statistic had been Bonferroni corrected for six inferences. In general, channels in the left hemisphere (F3, C3, P3) displayed increased power during left trials compared to right trials as a result of mu (and beta) suppression in these channels during right hand imagery. The opposite was true for right hemisphere channels. Not shown is the plot for beta power differences, in which participants such as S16, P36, and P39, who achieved relatively high online motor-imagery accuracies, showed more consistent patterns of hemispheric desychronization. As expected, those with poor performance in the motor-imagery task failed to display significant mu power modulation depending on trial type. There are similarities and differences apparent when comparing power to coherence changes in these individuals. In Figure 5.8, coherence differences between left and right trials in the mu band can be seen for patients and controls. Significant differences in mean coherence between left and right trials are shown as lines between two of the 19 unipolar EEG channels. Thicker lines indicate where this interaction was significant after the test statistic had been Bonferroni corrected for 171 ([Nelec (Nelec 1)]/2) elec- × − trode pair inferences. Of note are the large hemispheric differences in coherence seen 110

S01 (63%) S02 (46%) S03 (61%) S04 (60%) S05 (61%) S07 (66%) S08 (64%) S09 (73%)

S10 (53%) S11 (49%) S12 (90%) S13 (68%) S14 (53%) S15 (59%) S16 (72%) P01 (51%)

P02 (62%) P05 (53%) P06 (45%) P08 (60%) P09 (97%) P11 (71%) P12 (60%) P15 (60%)

P16 (61%) P19 (51%) P21 (68%) P23 (51%) P26 (58%) P27 (51%) P28 (45%) P29 (56%)

P30 (51%) P31 (65%) P34 (75%) P36 (73%) P37 (63%) P38 (46%) P39 (74%) P43 (54%)

Legend

F3 F4 p<.05, B.C.Increased power p<.05 during left trials C3 C4 Increased power during right trials P3 P4

Figure 5.7: Power spectral density in the mu frequency band in each Laplacian- referenced channel, compared across left and right motor-imagery trials in controls and patients. Signiﬁcant power differences across trial type, as determined by repeated 2- sample t-tests, are indicated by colored circles at each electrode. Larger circles indicate signiﬁcance after Bonferroni correction. for the high-performing individuals. The pattern of these differences indicated increased coherence for right hemisphere channels during left trials and increased coherence for left hemisphere channels during right trials. As opposed to the clear hemispheric differentiation of coherence observed in high-performing subjects, there were large scale, whole brain coherence changes between left and right trials in other participants, e.g. P02, P05, P29, and P43. Interestingly, these users all achieved relatively low online performance, and did not demonstrate strong hemispheric power differences in either the mu or beta bands. To determine whether coherence features could be useful in motor-imagery clas- 111

S01 (63%) S02 (46%) S03 (61%) S04 (60%) S05 (61%) S07 (66%) S08 (64%) S09 (73%)

S10 (53%) S11 (49%) S12 (90%) S13 (68%) S14 (53%) S15 (59%) S16 (72%) P01 (51%)

P02 (62%) P05 (53%) P06 (45%) P08 (60%) P09 (97%) P11 (71%) P12 (60%) P15 (60%)

P16 (61%) P19 (51%) P21 (68%) P23 (51%) P26 (58%) P27 (51%) P28 (45%) P29 (56%)

P30 (51%) P31 (65%) P34 (75%) P36 (73%) P37 (63%) P38 (46%) P39 (74%) P43 (54%)

Legend FP1 FP2 F3 Fz F4 p<.05, B.C.Increased coherence p<.05 F7 F8 during left trials T7 C3 Cz C4 T8 Increased coherence during right trials P7 P3 Pz P4 P8 O1 O2

Figure 5.8: Coherence in the mu frequency band in each channel, compared across left and right motor-imagery trials in controls and patients. Significant coherence differences across trial type, as determined by repeated 2-sample t-tests, are indicated by lines between electrodes. Thicker lines indicate significance after Bonferroni correction. sification, an offline classifier was implemented. The classifier compared the errors achieved when using typical power features vs. coherence features. The results for each individual, shown in Figure 5.9, indicated high variability both in overall performance and relative utility of coherence features. The error differential between classifiers using power and coherence features was not different across groups of control participants, cognitively normal patients, and cognitively impaired patients, although certain patients in the cognitively impaired group did achieve substantial differential benefit from using the classifier based on coherence features. Notably, patients P06 and P43 saw improvement in their test set errors from 46.25% to 34.5% and from 47.9% to 37.1% when switching from power to coherence features for classification. 112

S01 S02 S03 S04 S05 S07 S08 S09

Power Coherence

S10 S11 S12 S13 S14 S15 S16 P01

Control Cog. Normal Cog. Imp P02 P05 P06 P08 P09 P11 P12 P15 2

P16 P19 P21 P23 P26 P27 P28 P29 -2

-4

-6 P30 P31 P34 P36 P37 P38 P39 P43

Error (%) Power-Coherence 50 -8 Error(%) 0 -10

Figure 5.9: Classiﬁcation errors from ofﬂine cross validation using power and coherence features separately. (Bottom right) Differences in these errors averaged over participant groups. Error differentials are not different between groups.

Discussion

The spectral measure of coherence has been previously used to describe the interaction between electrode sites during motor planning and execution. Rappelsberger at al. described a contralateral coherence increase in the alpha band in premotor and motor areas just prior to movement onset [94]. Leocani et al. described event related coherence increases due to self-paced movements that were localized spatio-temporally with event related desychronization in frontotemporal areas [93]. The contralateral increase in coherence described in these studies was similar to what was observed in the high-performing motor imagery users in this analysis. Furthermore, coherence changes in response to imagery overlapped spatially and temporally with suppression of SMR power. For most individuals there was no appreciable difference in intensity of coherence between imagery states. Of the subjects which did show consistent coherence changes, 113 the high-performing users demonstrated clear hemispheric lateralization of this effect. However, there were also individuals who demonstrated whole brain changes to coherence values which were hemisphere invariant. Although the functional interpretation of whole-brain coherence changes is less clear, for both of these groups, these features should theoretically allow for classification into left and right states. Furthermore, coherence in higher frequency bands has been demonstrated to produce complementary and somatotopically specific changes as a result of motor action [279]. Functional connectivity changes at the gamma (30-100 Hz) level were not assessed in this analysis, although they may prove useful in this type of imagery task. Overall, power features proved to be more useful for the classification task. This does not mean that for certain individuals (e.g. P06, P43) there existed a greater level of discriminable information in features of coherence compared to traditional power changes. It should be noted that both of these individuals were indicated as cognitively impaired by the ALS-CBS, thus generating the non-significant trend that coherence features were more useful for those with cognitive impairment compared to those with intact cognition. Despite this trend, the majority of individuals in the cognitively impaired group performed at a random level, and it would be unfounded to make any statements about the optimal features to use in this group of individuals. That being said, on an individual level, feature selection including coherence measures could prove beneficial in classification. In the case of P06, discrimination between classes was due to hemispheric lateralization of coherence found in the beta band, while in P43, there existed a spatially diffuse increase in coherence during right trials over a large range of frequencies. Measures of coherence, although overlooked in healthy individuals, may be a more robust mechanism for control in those with diffuse neurodegeneration, although further confirmation of this required. Most likely, a unique combination of both coherence and power features will result in the optimal classifier. 114

5.2.2 State estimation based on neural ﬁeld modeling

Introduction

In line with our goal for device personalization, we sought to use additional features as input to the classification system. In this section, we move beyond empirical measures of functional coupling discussed earlier in the chapter, in order to use the accumulated knowledge about brain anatomical networks that produce large scale electrical activity. In doing so, we constrain the estimates of population functional connectivity to a plausible subset of realistic interactions, thus reducing the influence of irrelevant features. Neural models are effective tools used in neuroscience to accomplish such data assimilation, and can provide a physiologically-plausible structural basis for estimates of brain connectivity. Lumped-parameter neural models are able to emulate different aspects of brain activity, from thalamo-cortical generators of occipital alpha rhythms [280, 281, 282], to high frequency rhythms and epileptic activity [283], sensorimotor generators [284], and oddball responses [285]. Although the dynamics of these models are not nearly as complex as those defined using populations of individual neurons [283, 286], they have the benefit of being parameterized by a low number of variables. The original ‘neural mass’ models [280, 287], consisting of discrete interacting cortical structures, have recently been generalized to occupy a continuum, resulting in ‘neural field’ models [288, 289]. Continuum models such as these are adept at defining activity found between cortical columns, such as traveling waves. Neural mass/field models integrate the postsynaptic potentials of a neural subpopula- tion, and transform this membrane potential into a spike train that influences neighboring populations. These time-evolving networks are then forward mapped to sensor-space and fit to bioelectric data. Our proposed neural field model follows the framework given in Freestone et al. [290]. Their procedure improves on other models by converting the continuous field potential into a discretized state-space by Galerkin projection onto a set of basis functions, followed by tracking of states and parameters using a variation on the Kalman Filter. This allows for assimilation of neural data in order to track parameters which represent connectivity strengths between populations, as well as time constants. With this model, we attempted to infer differential parameter distributions that exist due to the intentional states of a BCI paradigm. 115

Methods

The estimation framework of Freestone et al. [290]

To develop a set of BCI features representing connectivity between brain regions during a motor-imagery task, we looked towards the work of Freestone et al. [290]. This group applied a neural field model for the purpose of estimating connectivity kernels between neural populations recorded at the cortical surface. In this section, their framework, which was originally suited for microelectrode recordings on the cortex, was modified to enable estimation of connectivity strengths between larger scale populations measured by EEG. The estimation framework is based on knowledge of the generative model that is assumed to underly electrical phenomena in the brain. Their discretized integro-difference equation (IDE) describes the evolution of a neural field on the surface of the cortex at locations r as

Z νt+Ts(r) = ξνt(r) + Ts w(r,r0) f (νt(r0))dr0 + et(r) (5.3) Ω

where νt is the membrane potential of the spatially distributed neural field over domain Ω at time t, Ts is the time step of the field, and ξ represents the memory in the field, and is related to the time step and the synaptic time constant of the underlying dynamics. These dynamics are represented by a sigmoidal function f that transforms the membrane potential of a population into a firing rate as in

1 f (ν(r0,t)) = . (5.4) 1 + exp(ς(ν0 ν(r ,t))) − 0

This function is defined by the firing threshold ν0 and the slope ς. To update the membrane potential from one state to the next, the firing rate is convolved with the spatial connectivity kernel w, which is composed of three Guassian functions ψ(r,r0), scaled by amplitudes θ, following

w(r,r0) = ψ>(r,r0)θ. (5.5) 116

Here, ψ is a 3x1 vector of Gaussian basis functions, θ is a 3x1 vector of scaling parameters, and > represents the matrix transpose. It is assumed that the width of the connectivity basis functions are known, and the scaling parameters are to be estimated from the data. For the simulated field, by which the authors assess the efficacy of their framework, θ =[100 -80 5]>, generating a connectivity kernel between populations that facilitates center excitation, surround inhibition, and weak long-range excitation, sometimes referred to as a Mexican Hat function (Figure 5.10). Finally, the IDE includes a disturbance term et defined by spatial covariance γ(r r ). − 0 In the paper of Freestone et al., the field of membrane voltage is propagated forward into electrode space containing sensors y that exist at locations rn, according to

Z yt(rn) = m(rn r0)νt(r0)dr0 + εt(rn). (5.6) Ω −

This equation describes the mapping between the membrane potential and electrode space, with sensor noise εt deﬁned by a multivariate normal distribution, and using a Gaussian observation kernel m, which deﬁnes the falloff of sensitivity of the electrode 2 with width σm.

! (rn r0)0(rn r0) m(rn r0) = exp − 2 − (5.7) − − σm

Thus far described has been the underlying generative model for field potential and sensor activation. In the application of this framework to EEG data, we had available sensor activations y(rn), and no information about the underlying field. However, from this model we were able to perform a projection into state space so that estimation of the underlying field could be performed. In order to perform this estimation, Freestone et al. decompose the neural field νt into a set of states xt and associated basis functions φ(r) of the form

νt(r) φ >(r)xt (5.8) ≈ 117

! (rb r0)0(rb r0) φ(rb r0) = exp − 2 − . (5.9) − − σφ

Here, the basis functions with centers at nodes rb are Gaussian shaped with width 2 σφ . To project the field into state space, the basis decomposition of νt is substituted into the IDE model, followed by pre-multiplication of the equations by the field basis functions φ. Following simplification provided in [290], the state space model takes the following form:

Z xt+1 = Ψ(r0) f (φ >(r0)xt)dr0θ + ξxt + et (5.10) Ω

yt = Cxt + εt (5.11)

With relevant state-space matrices taking the form:

Z ∆ 1 [Ψ(r0)]:i = TsΓ− φ(r)ψi(2ci + r0 r)dr (5.12) Ω − ∆ Z Γ = φ(r)φ >(r)dr (5.13) Ω Z ∆ 1 et = Γ− φ(r)et(r)dr (5.14) Ω ∆ Z Ci j = m(ri r0)φ j(r0)dr0 (5.15) Ω −

These four entities, which are employed in the estimation procedure, are predefined analytically before entering the iterative estimation loop. The job of the estimator is to assimilate the data from the sensors with knowledge about the underlying connectivity among populations of the unobserved field in order to find the states xt, the connectivity kernel parameters θ, and the synaptic dynamics ξ. This is accomplished with a two-part iterative algorithm, in which state estimation is performed by the unscented Rauch-Tung-Striebel smoother (URTSS), and parameter estimation is done through minimization of the squared error of the state prediction. For each trial of data, the URTSS is run forward and backward to generate the best estimate of the states given the current estimate of the connectivity kernel. Then the parameter vector is updated with a least squares estimator which minimizes the sum of the 118

Sensor Space (rn) Sensor activations, yt Observation function C

State Space (rb)

Field states, xt Basis functions φ(rb r ) − 0 Continuum (r)

Connectivity Kernel w(r,r0) Node dynamics f (ν(r0,t)) -10 -8 -6 -4 -2 0 2 4 6 8 10 Space (cm)

Figure 5.10: 2D representation of field space (black), state space (blue), and sensor space (red). The points at each level represent the relative spacing of nodes. Each node of the field (black points) has assumed dynamics which transform the membrane voltage into a firing rate. The black line shows the connectivity kernel, or the influence the central node has on the surrounding nodes in the field. The blue line represents the basis function used for the Galerkin projection; it shows the relative contribution of the field nodes to the central basis node. The red line shows the state space representation of a single row of the observation matrix, showing the influence of each basis function on the amplitude of the sensor (filled red circle). squared errors of the predicted state update. Explicit formulations of the URTSS and least squares algorithms are given in [290]. The estimation procedure stops when the parameters change < 5% from their previous estimate. For replication of the simulations performed by the original authors, 50 instances of the neural field were generated, resulting in 50 parameter estimates.

Modiﬁcations to the estimation framework

The transition from cortical measurements to the scalp EEG required significant reformulation of the model in order to accommodate for incongruities with scalp data. The domain of EEG space was an order of magnitude larger, scaling from tens of millimeters to tens of centimeters. Many variables were able to scale with the increasing dimensions, like the discretization of the field, and the spacing and width of the basis functions. Oth- ers, such as the connectivity kernel width, sensor and field disturbance variance were 119 kept in proportion. Two data sets were available for testing, one with a high-density 129 electrode montage, and another with a low density 19 electrode montage. Modifications are summarized in Table 5.1 and Figure 5.11, and describe primarily accommodations for the increased spatial domain of the field, although there were changes to the sampling frequency as well the formulation of the connectivity kernel w. All changes are detailed further here:

• Dimensions of field, bases, and sensors. All dimensions related to the neural field were increased by ten-fold. Dimensions originally 10 mm in the x and y ± direction have been modified to 10 cm. ± • Node spacing. Spacing of the node locations (r) increased, along with the spacing

of the basis functions (rb) and sensors (rn). The locations of the sensors were constrained to a grid governed by the actual locations of the electrodes in their respective geodesic net and 10-20 system conﬁgurations. From the high-density recording, 46 of the central electrodes were retained in the estimation procedure, while all 19 of the electrodes were used in the low-density estimate.

• Parameterization of observation and basis functions. Width of observation kernels and field basis functions increased in proportion to the spacing of nodes in order to achieve similar falloff with distance. Due to the increased distance between sensors, the observation kernel width was increased so that each electrode sampled from a larger proportion of field space. This is consistent with the greater spread of the field to sensor space in EEG compared to microgrid measurements. The new spacing and width of the basis functions limited the maximum spatial frequency able to be reconstructed by the field to 0.06 cycles/cm. In the original framework, this maximum frequency was 0.12 cycles/mm (this has not been scaled by a factor of 10, so in effect, the spatial resolution of the original field was double the new field, due to increased basis function spacing).

• Parameterization of the connectivity kernel. The variance of the connectivity kernel was doubled in the case of the high-density EEG field, and tripled in the case of the low-density EEG field. These increases were based on the spatial extent of the resulting connectivity kernel in terms of the influence of nearby basis functions. 120

Table 5.1: Parameters used in the neural ﬁeld estimation framework. The left column replicates the values of Freestone et al. [290], while the values in the remaining columns indicate changes to accommodate for high- and low-density EEG data.

Freestone et al. [290] High-Density EEG Low-Density EEG Spatial domain, Ω 10 mm 10 cm 10 cm Spatial discretization, ∆ ±.5 mm± 1 cm± 1 cm Time step, Ts 0.001 s 0.004 s 0.004 s 1 Synaptic time constant, τ .01 s− -- Firing threshold, ν0 1.8 mV - - 1 Activation function slope, ς, .056 mV− -- Connectivity kernel widths, σψ [1.8, 2.4, 3.6] mm [3.6, 4.8, 12] cm [5.4, 7.2, 18] cm Number of sensors, ny 196 46 19 Distance between sensors, ∆y 1.5 mm 2.66 cm 5 cm Observation kernel width, σm 0.9 mm 1.6 cm 3 cm 2 2 2 Observation noise variance, Σε .1 In mm .5 In cm 1.67 In cm × y × y × y Disturbance spatial cov. width, σγ 1.3 mm 1.3 cm 1.3 cm Disturbance variance, σd 0.1 mV - - Number of basis functions, nx 81 16 9 Dist. between basis functions, ∆φ 2.5 mm 5 cm 7.5 cm Width of basis functions, σφ 1.58 mm 3.12 cm 4.68 cm

• Synaptic dynamics. We assumed the same model for field generation as the original model. The membrane potential of the field is transformed via a parameterized sigmoidal function to a firing rate. This firing rate influences neighboring neural populations through Mexican hat connectivity. Amplitude of the EEG data was scaled to the same size as the simulated data, so that the sigmoid parameters acted on the data in a similar manner.

• Sampling frequency. The simulations were carried out at 1000 samples/second. The EEG recordings were acquired at 250 samples/second, which prompted us to change the sampling parameter in the model. This quadrupling of the time step of the ﬁeld affected both the time constant parameter ξ and the state space connectivity matrix Ψ.

To assess the validity of the estimation procedure, we ﬁrst looked at the reproduction of the sensor activations y from the reconstructed states x. This allowed us to determine whether the iterative state tracking procedure effectively assimilated the data fed in by the sensors or if errors accumulated, resulting in unreasonable estimates for the parameters. 121

Original High-Density EEG Low-Density EEG 10 100 100

0 0 0

-10 -100 -100 -10 0 10 -100 0 100 -100 0 100

Figure 5.11: Modifications to the estimation framework are given for the high-density EEG recordings (center column) and the low-density recordings (right column), with the original simulation framework on the left. The top row shows the extent and spacing between the nodes in the field, in mm. The second row displays the locations of the field states in blue, as well as the associated basis function for a single state. The third row displays the locations of the sensors in red, as well as the observation kernel for the center-most sensor. The bottom row once again shows the locations of the states, along with the connectivity kernel of the state at the center of the field. All rows use the same color scale except the last row, for which the amplitude of the EEG connectivity kernels are four times larger due to the increased time step. 122

The main goal was to find the distributions of kernel amplitude parameters, θ, across imagery states. A major issue for assessing the validity of the kernel estimates was that there was no information about the actual nature of the underlying connectivity kernels. The expectation was that the estimated kernels would display a similar center excitation, surround inhibition pattern, as this was not expected to change with scalp recordings. From a standpoint of utility as a feature for classifying individual BCI trials, the values of the parameters were not as critical as the consistency of estimates across trials. We performed classification on the first two kernel parameters of θ, describing the amplitude of center and surround influence, in order to determine if the active state of mental imagery could be distinguished from the state of rest. This required the classifier to be trained on left and right imagery data in a single class, with no-go data belonging to the other class. Ten-fold cross validation of the LDA classifier was performed on the roughly 480 (160 per cue) trials for each individual, and the test set error was reported.

Results

Replication of the simulated estimation framework

The first goal of the neural field estimation procedure was to replicate the findings of Freestone et al., which was accomplished through generation of a custom MATLAB code. As can be seen in Figure 5.12, the estimated kernels for three different field simulations, representing three cases of field connectivity, matched those estimates presented in the original paper. Between the original and reproduced model, there were no significant differences in the distribution of any of the [θ; ξ] parameters yielded by the estimation scheme. The influence of the synaptic dynamics on kernel reconstruction were also replicated as in [290]. The ideal values for ς and ν0 were close to the .56 and 1.8, with these parameters producing the lowest root mean squared error between the simulated kernel and the estimated kernel. Critically, increasing the field size does not substantially alter these dynamics, which was determined by repeating this simulation on a field which had been enlarged by a factor of ten. Reconstruction fidelity could be directly assessed in the case of the simulated field by comparing the estimated states, transformed by the field basis functions (φ >x) with the underlying true field, ν. As seen in Figure 5.13, field reconstruction captured the 123

40 40 60

30 30 50 40 20 20 30 10 10 20 0 0 10 -10 -10 0 -20 -20 -10 -10 -5 0 5 10 -10 -5 0 5 10 -10 -5 0 5 10

Figure 5.12: (Top) Selected simulations (A-C) using three different kernel amplitudes, from [290]. (Bottom) Replication of this framework produces nearly identical kernel estimates. The solid line is the actual kernel used to generate the data, the dashed line is the mean of the trial estimates, and the shaded region is the confidence interval for the mean. dynamics at large spatial scales, but missed higher frequency components of the field. This was an inherent limitation of the reconstructed estimate due to the greater spread of basis functions compared to the nodes of the field. The field and its reconstruction were analyzed to determine their spectral distributions. Power spectra in two dimensions were computed using the 2D Fourier transform (Figure 5.14). In the simulated case, these power spectra were able to be calculated for the field ν, as shown in equation 5.17.

X 1Y 1 1 − − 2πi( kx + ly ) νˆ (k,l) = ∑ ∑ e− X Y ν(x,y) (5.16) XY x=0 y=0 2 Pν = νˆ (k,l) (5.17) | |

Evaluation of the power spectral density demonstrated the spatial spectral falloff for the observations from the simulated electrode grid (Py) and the reconstructed subspace (P ). These two should roughly match because the conﬁguration of the basis functions φ >x are chosen to account for the full spatial bandwidth of the observations. As can be seen from the ﬁgure, the relatively greater spacing of the basis functions generated a loss of 124

Field (8) Reconstruction (?′x) Error (?′x ! 8) -10 2 1 -5 1 1 0 0 0 0 5 -1 -1 -1 10 -10 0 10

Electrode (y) Projection (Cx) Error (Cx ! y) -10 4 1 -5 2 2 0 0 0 0 5 -2 -2 -1 10 -10 0 10 Figure 5.13: Field reconstruction can be directly assessed with a simulated ﬁeld. (Top) The underlying ﬁeld ν is subtracted from the reconstruction in state space, transformed by basis functions φ. (Bottom) Electrode activations and the estimated states projected into sensor space, and the resulting error.

P P ! P P8 Py ?⊤x ?⊤x 8 0 0 0 45 0 35 45 45

40 0.1 30 40 40 0.5 35 0.5 35 0.5 35 0.2 25 30 30 30 0.3 25 20 25 25 1 1 1 0 0.5 1 0 0.1 0.2 0.3 0 0.5 1 0 0.5 1

Figure 5.14: Plots of spectral falloff for each field from left to right: The simulated neural field, the electrode space, the reconstructed field, the error between the field and the reconstruction. The reconstruction captures the dynamics at large spatial scales, but misses higher frequency components of the field. high frequency information in the reconstruction. This analysis provided an example of the spatial frequencies that could be expected to be lost in the estimation procedure. Of course, these high frequency losses could be mitigated by choosing a higher density of basis functions, at the cost of significantly higher computational complexity. In the case of estimation using EEG data, the reason a greater number of basis functions was not used was due to the problems encountered with over-fitting sensor data to a higher- dimensional basis space.

Results with EEG data

The modiﬁcations to the estimation framework involved altering the spatial dimensions of the ﬁeld, the number of node, basis, and sensor elements, and the extent of the con- 125

Electrode (y) Projection (Cx) Error (Cx ! y) 0 200 -50 -200 -200 100 -400 0 0 -400 -600 -100 50 -800 -200 -600 -1000 -300 -50 0 50

Electrode (y) Projection (Cx) Error (Cx ! y)

-100 200 100 200 0 -50 0 -100 0 0 -200 -200 50 -300 -400 -200 100 -400

-100 0 100

Figure 5.15: Observed sensor activity at one time point (left column), estimated states projected into sensor space (middle column), and the resulting error for high density (top) and low density (bottom) EEG data. The state-space formulation tracks the gross features of the field, but cannot resolve observation dynamics on smaller spatial scales. nectivities between these elements (Figure 5.11). With the EEG data, there was no underlying field to assess the reconstruction fidelity of the estimation, since there is no information about the cortical generators that lead to the measured signal. We could, however, compare the sensor activations to a projection of the estimated states into sensor space (Figure 5.15). This provides a check that the states and parameters of the model were reconstructing the observed dynamics, and that there were no systematic errors accumulating in the estimation procedure. The relative ratio of the error in the observed field was on a similar order for the high density recordings, although the error increased as the number of basis functions decreased, as in the case of the low density field. We could also assess if the basis functions were sensitive to the frequency content of the observations. As can be seen in Figure 5.16, the spatial frequency content of the observation kernel was still above the -3 dB point at the Nyquist frequency. However, the falloff pattern was similar to the falloff of the basis functions, which both scaled by a factor of two when going from the high-density EEG field to the low-density field. As can be seen from Figure 5.17, there was a tendency for the Mexican Hat connectivity kernel estimates to exhibit a negative central peak. This was the case for all 5 subjects with the low-density EEG recordings. The exception to this was high-density EEG from S002, whose kernel distribution displayed a positive central peak. Interest- ingly, this individual also achieved the greatest consistency among kernel estimates and 126

-3 P P x 10 y ?⊤x 70 0 0 30 5 60 0.02 25 10 50 15 0.04 20 40 20 0 10 20 0 0.02 0.04 -3 x 10 -3 x 10 0 24 0 60 22 20 50 0.02 5 18 16 40 0.04 14 10

0 5 10 0 0.02 0.04 -3 x 10

Figure 5.16: For the high-density (top) and low-density (bottom) fields, only power (in dB) in electrode space and reconstruction space are able to be analyzed. Both of these display much lower frequency content than the simulated field, due to the expansion of the field and the increased spacing of nodes. exhibited the greatest differentiability of kernels between mental states. For both S001 and S002, the magnitude of the kernel estimate is largest for the no- go trials. For individuals with low-density EEG recordings, this trend appeared to be reversed, although in these individuals there were no significant differences in kernels between the three mental states. A reasonable question to ask, especially of the kernel estimates of participant S002, is whether these features could be used in a classifier to predict state of mental imagery in an online BCI. For the 10-fold cross validation of a classifier trained on imagery and non-imagery trials, the test set classification errors aligned with the variability of the kernel estimates. The lowest error of 19.4% resulted from the relatively high discriminability between no-go and imagery trials in participant S002. On the other hand, all of the participants with low density recordings generated errors >40%, or near random.

Discussion

Freestone et al. provide a novel framework for tracking the states of a neural field, while simultaneously estimating the parameters of spatial connectivity within a simulated cortical field. We set out to modify this framework to estimate relative connectivity strengths between states of the neural field underlying motor-imagery EEG recordings. With our modified framework, we achieved limited success in differentiating between 127

S001 S002 P09 P11 P34 P36 P39

0.02 0.01 0 -0.01 -0.02 -100 0 100 10-fold Cross-validation error 49.6% 19.4% 42.3% 46.5% 42.0% 45.9% 45.8%

Figure 5.17: From left to right: High-density recordings in participants S001 and S002, low-density recordings in participants P09, P11, P34, P36, and P39. Estimates of kernels from left (blue), right (red), and no-go (gray) trials. The dotted line is the mean of the kernel distribution, and the shaded region is the standard error. At the bottom is the cross validated test set error of the classification between imagery and non-imagery trials. the active states of imagery and rest periods using filtered EEG data. Although there were not as many subjects with high-density EEG recordings available, the use of a greater number of sensors appears to be better suited for distinguishing between mental states. We anticipate the estimation of the low-density EEG scheme suffers because of the focal nature of the signal of interest. While a large array of electrodes was used in the space of state estimation, only a small portion of these electrodes are expected to sample from the region of scalp exhibiting task-relevant changes in spectral power and coherence. Certainly, this framework would be more appropriately suited for a high-density EEG array or even an ECoG grid, both of which pack a higher density of sensors potentially along a region of brain that is engaged in the motor-imagery task. There are other aspects of the modified estimation scheme which have the potential for improvement. Many of the modifications made to the model were straightforward, such as alteration of the size and spacing of elements in estimation space. However, the model was originally designed for estimating field connectivity from the measurements provided by a microelectrode grid, and the difference in the spatial scale of the model also requires additional experimental confirmation of parameters, and re-imagining of the assumed generative dynamics. The first aspect of the model to be addressed in the future is the generative model 128 for the dynamics of the unobserved field. As opposed to a microgrid array, which may reasonably record from neuronal ensembles that experience the influence of center excitation and surround inhibition from its neighbors, the large scale, synchronous postsynaptic activity recorded by EEG may require a different implementation. A Gaussian or Laplacian connectivity kernel, rather than the Mexican Hat type, may better describe the influence of nearby cortical activity. The fixed width of the connectivity kernel bases may pose additional limitations by not allowing interactions to occur on multiple scales. In a follow-up paper, the authors of the original model utilized a family of B-spline wavelets and associated scaling functions to create a multi-resolution neural field estimation framework [291]. This modification allows for inference of connection strengths on multiple scales, which would be of great use with EEG data, as often the precise scale of interaction is unknown. The convolution of this kernel with the sigmoidal firing rate function may also be inappropriate for use when the characteristic scale is an order of magnitude larger. Sim- plification of the connectivity kernel may be less desirable than a more realistic neural mass model for SMR rhythm generation that includes multiple neuron types [285]. Other groups have used discretized damped linear wave equations to describe current source density [292] fit to EEG, or estimated parameters using the spectra of the EEG rather than the time series representation [284]. In addition to the changes in the proposed model for field generation, there are some additional aspects of the current model that are lacking. Specifically, the omission of time delays becomes more critical at larger spatial scales. Delays in signaling among distant neural populations may be included through implementation of a defined prop- agation velocity. Finally, the observation kernel in the modification remained of the Gaussian form. A more appropriate observation kernel for EEG applications might take the form of a lead field matrix, including information about the location of the source, the electrode, and the intervening media. Ideally, the observation kernel would be created on an individual basis to represent personal differences in geometry and distribution of skull tissue. One use of this type of data assimilation is to track the single trial parameter differences for the purpose of classification, as in the case of a BCI. This will always be a more difficult task than finding differences in the average connectivity over a number of trials, and will require reformulation of the estimation algorithm to make it operate 129 in an online scenario. While this is a worthwhile goal, this model could also be used to assess the functional changes that occur as a result of the neurodegeneration inherent to ALS. Using the tools of regional connectivity elaborated in the previous section, this estimation framework may be employed to describe motor and non-motor related network changes in the ALS brain. CHAPTER 6

CONCLUSIONS, FUTURE DIRECTIONS, AND ETHICAL CONSIDERATIONS

6.1 Personalized deployment and design of BCI devices in ALS

6.1.1 Prospects for gating

In Chapter 3, trial gating is demonstrated to improve accuracy and speed of the BCI interface for certain individuals. All three of tested gating mechanisms are derived from EEG data that is supplementary to the primary task, and therefore may be considered a form of hybrid BCI that restricts the timing of the primary classification to periods of high task potential. This allows for asynchronous timing of the system controlled by the user’s state of vigilance or desire to use the device. The study on BCI gating was performed offline using motor imagery data, and indicated that most individuals would benefit from gating trials with low baseline mu amplitude, while gating by the robustness of the VEP was also beneficial for some. These procedures require validation in an online implementation. A future study will employ similar analysis for defining EEG features which predict single trial motor imagery performance. Something as simple as a thresholding procedure for mu amplitude may be used to determine whether motor-imagery trials are allowed or gated. Alternatively, the hybrid system could em- 131 ploy multivariate classification on available gating signals to make this decision. Online gating of low-predictive trials will allow us to assess whether this type of procedure improves performance and more importantly makes device use more enjoyable. What remains to be seen is how the feedback of the gating procedure affects the stationarity of these signals, whether the user is able to adapt to repeated gating by modulating EEG correlates of vigilance, and whether user-defined asynchrony is effective using this method. This is a valuable avenue of study, as the potential for improvement is greatest for low performing users, and could possibly offer a solution to BCI inefficiency seen in a significant proportion of users.

6.1.2 System Targeting

Chapter 4 focused on how the unique capabilities and limitations presented by ALS patients affect the utility of standard BCI systems. These results of this study are of high clinical relevance, as disease heterogeneity can influence the course of technological intervention decided on by the patient, as well as the level of achievement possible with different devices. It was found that behavioral impairment, which can affect up to half of ALS patients, was associated with a relative lack of interest for pursuing BCI as an assistive communication technology. This finding is consistent with the behavioral signs of apathy and mental rigidity associated with the disease [114, 156]. At the same time, those with cognitive impairment were more interested in pursuing BCI than those with normal cognition, possibly as a result of loss of insight for cognitive deficits. As my research with BCI-AAC devices in ALS continues, additional studies will help confirm these findings and probe further the reasons for device rejection. Assess- ment of patients’ neuropsychological states in greater detail is required to achieve a better understanding of their cognitive and behavioral limitations. The ALS-CBS was used primarily due to time restrictions, but more comprehensive tests for cognition and behavior in ALS may serve to acquire a better understanding of changes in these domains specific to the disease [293]. Of course, permitting more time for BCI training would be a goal of future work, as it would allow more users to reach their optimal performance level, so that they could better make decisions about long-term BCI use. This is especially true of the motor-imagery system, for which lack of overall control likely contributed to poor post-training evaluations of this system. 132

Of foremost importance in future study is to identify the reasoning behind disinterest in BCI-AACs. Depression may affect motivation for exploring new technologies, and its impact on BCI interest will need to be quantified. Self-reported motivation has been shown to affect the quality of P300 signals in a spelling task [158], while increasing motivation using monetary incentives or immersion in virtual feedback environments significantly improve performance on BCI tasks, most notably in poor performers [294, 108]. Nijboer et al. [158] present a more comprehensive assessment of motivation that encompasses four factors: mastery confidence, incompetence fear, interest, and challenge. It should be interesting to determine which of these factors play a role in device rejection among behaviorally impaired patients, and which associate with higher device acceptance among those who are cognitively impaired. With regard to system targeting based on factors of the disease, the only variable which correlated with differential performance on the BCI tasks was age. The correlation was not particularly strong, but does indicate that older patients perform relatively better using a P300 system. This finding needs to be reproduced in an additional sample, taking into account motivational and cognitive factors. It has been shown that both P300 amplitude and motor imagery ability decline with age [217, 295], so individual assessment of P300 generation and imagery ability will be needed to make decisions about optimal BCI paradigms.

6.1.3 System Personalization

Of the disease factors measured in this study, those related to cognitive function, rather than physical function, were determined to predict successful BCI operation. Specifi- cally, the cognitive sub-score of attention was associated with BCI proficiency in both P300 and motor-imagery tasks. This is consistent with the executive function deficits seen in FTD, in which prefrontal damage can often lead to the presence of reduced focus and attention. Although the studies in this dissertation did not directly assess how performance and cognition are related in locked-in syndrome, the loss of BCI capability in CLIS may in part be considered the extrapolation of worsening performance with increasing cognitive limitations found here. The morphology of the P300 and resulting task performance is reduced with age and cognitive dysfunction. Modifications to P300 systems have attempted to overcome 133 these limitations by reducing cognitive load. Some teams have used centralized spellers and language prediction models, by which distractions and memory load are reduced by removing the need to search for letters in a grid. Symbolic spellers allow for rapid communication of the common needs of daily living, and have been shown to be invariant to the cognitive status of the patient. Finally, entirely new paradigms eliciting alternative evoked potentials such as the N400 in response to familiar faces have been shown to be robust against cognitive decline compared to the P300. Modifications such as these may make evoked potential-based tasks more viable for patients with subtle cognitive impairment, and will need to be tested for improved communication in ALS patients with reduced cognitive capacity. Modifications to motor-imagery tasks that address cognitive decline are less studied, and will require more intensive participant training to evaluate. Certainly, the low number of training sessions factored into the poor performance of the ALS participants on the motor-imagery task. Future work will involve at least ten sessions of training. The use of a sufficient training interval may reveal the differential timing of training effects (or lack of such effects) that occur in individuals with cognitive limitations. As was the case with the P300 speller, the cognitive sub-score of attention was a particularly important indicator of motor-imagery quality. In addition to the results of the cognitive screen, the presence of elevated EEG power in frequencies unrelated to the task was also common in low-performing individuals. Further work will study the linkage between these two findings, along with methods to reduce this task-irrelevant signal. This may involve utilizing alternative forms of mental imagery that originate from brain networks which are less affected by the neurodegenerative processes occurring in ALS, such as auditory imagery and spatial navigation. Additional study is required to determine whether sub-threshold genetic markers for C9ORF72-linked ALS/FTD correlate with cognitive impairment, a trend which was not indicated in the study performed in this dissertation. This linkage would provide evidence for graded cognitive decline with repeat length, a finding whose impact would extend far beyond the field of BCI. Without this, the relationship between repeat length and performance on both of the tasks needs to be assessed for alternative causal influences. Finally, the limitations of patients with CLIS were not directly assessed in this dissertation, as all ALS patients retained at least residual eye movement. Up to this point, 134 there has been very little success of achieving reliable communication in these patients, although a number of reasons for this are becoming evident. First, the cuing and feedback system may be inappropriate for use in these patients who often develop visual impairments. Second, scalp-based electromagnetic measures of brain activity may not be appropriate, given that the only minor success for these patients was achieved with near infrared spectroscopy, which measures blood oxygenation over long time scales. Last, the work presented here points to gradual loss in cognition, specifically attention, as a possible mechanism for the extinction of BCI use in CLIS. For any of these modes of failure, we have a better idea of how to personalize systems to CLIS users by using modified inputs, recording methods, and training techniques to facilitate device success.

6.1.4 Alternative features

Substantial improvements to BCI performance can be made by optimization of neural features extracted from ALS patients, the beginnings of which are detailed in Chapters 3 & 5. System gating, as described above, employs unused features recorded during the primary BCI task, in order to make decisions about task readiness. As was demonstrated offline, this type of gating could significantly improve the performance and speed of the interface. Personalized feature optimization allows the classifier to account for diversity among individuals by utilizing unique brain signatures, which are sometimes outside of the ‘normal’ ranges expected for these tasks. With this optimal feature set, the user can perform at a higher level than if feature extraction were a fixed process. Additionally, the dimension reduction accomplished by feature selection makes simple linear classifiers more robust to over-fitting, decreases online computation time, and requires the use of fewer sensors. Future studies could employ feature reduction algorithms based on exhaustive search, and measure the stationarity of these optimal signals both inter- and intra- session. From the results of our P300 study, we expect high performing users to achieve the greatest reduction in sensor number, while ALS patients would benefit from classification periods around 500 ms after the presentation of the stimulus. Assessment of the changes to the optimal feature set in motor-imagery will rely on a more extensive training period. Results from the high-performing users indicate imagery in controls and 135

ALS patients could both be classified using a low number of channels, specifically C3 and C4 over the primary motor regions, leading to a substantial reduction in electrodes and feature space for classification. The use of coherence as a measure of interaction between electrode sites during motor imagery proved to be relevant for classification in a subset of individuals. Of the subjects which did show consistent coherence changes resulting from motor imagery, the high-performing users demonstrated clear hemispheric lateralization, displaying a contralateral increase in coherence within the SMR frequency bands. There were also individuals who demonstrated whole-brain changes in coherence values which were hemisphere invariant. Measures of coherence, although overlooked in healthy individuals, were found to be somewhat more effective with ALS patients demonstrating cognitive impairment, possibly due to compensation of functional networks through large-scale brain recruitment. Future work will address whether features of coherence used in conjunction with SMR power features can be utilized in online classification more effectively than either alone, and whether this improvement applies to cognitively impaired users as well. We used a novel method for data assimilation using a neural field as the underlying generative model and a modified Kalman filter to track the states of this field. This model was a modified version of that developed by Freestone at al. [290]. Such a model was used to generate estimates of functional connectivity of the underlying neural field that exist during the intentional states of a BCI paradigm. With our modified framework, we achieved limited success in differentiating between the active states of imagery and rest periods using filtered EEG data, although the high-density sensor system produced less variable results, and appears to be the most promising avenue for future work. The next step for such an estimation scheme is to develop a scalp-based grid with closely packed sensors localized to the region of interest. This will allow for assessment of kernel validity using high-density recordings in a larger group of subjects. Before exploring this, three modifications to the estimation procedure are apparent. These include the development of an observation function based on lead field models, the addition of signaling delays, and connectivity kernels which allow for multi-scale level interactions between field nodes. If kernel estimates arising from these recordings were to prove viable, the next step would be to implement the single trail tracking of kernel parameters for use in a BCI classification scheme. Additionally, this model could be used to assess 136 the functional changes that occur as a result of the neurodegeneration that accompanies ALS.

6.2 Ethical Considerations

A small section has been devoted to the ethical discussion that has accompanied the development of BCI. For simplicity, this overview is limited to matters surrounding research and application of brain-computer interfaces for communicative purposes in locked-in patient populations. Other discussions arising from the topics of BCIs in the classroom, the courtroom, and the military are given elsewhere [296, 297, 298]. The ethical basis for brain-computer interfacing emerges from the relatively new field of neuroethics, itself a recent extension of bioethics. It is from two major accomplish- ments in bioethics that we build the foundation of ethical discourse on the topic: The Declaration of Helsinki and the Belmont Report. The Declaration of Helsinki is a series of 37 ethical principles established by the World Medical Association in 1964 for physicians and those performing medical research involving human subjects and data [299]. Last modified in 2013, it establishes the life, health, dignity, self-determination, and confidentiality of research subjects as paramount importance in medical research. It dictates that the benefits associated with research should outweigh the risks, and for researchers to continuously monitor this relationship and modify study procedures accordingly. The declaration outlines the process of study oversight by research ethics committees, which are set up to approve and monitor research on human participants. It documents the procedures of acquiring informed consent, for placebo use, vulnerable populations, testing unproven clinical interventions, and the dissemination of study results to the public. The Belmont report was published by the U.S. government in 1979 as a set of guidelines for biomedical research on human subjects [300]. Within the report, the Belmont commission put forward three general principals which serve as a framework for investigators and review boards, which are respect for persons, beneficence, and justice. Respect for persons stipulates that individuals are treated with appropriate autonomy over their course of treatment, and protections given to those with diminished autonomy. Beneficence is an obligation to minimize harm while at the same time maximizing the benefits delivered through the intervention. Both the judgment of patient autonomy and 137 risks and benefits should be assessed regularly. The mandate of justice ensures that consideration is given towards the equal distribution of burden and reward that accompanies scientific inquiry. This is to insure that those who are financially or socially impover- ished do not bear the majority of the burden (undergoing scientific procedures) without bearing any of the reward (access to resulting treatments). In practice, the framework of the Belmont report serves as a regulatory model for modern institutional review boards. The applications of such a framework pertain to the process of informed consent and voluntariness, subject selection, and the evaluation of risks and benefits.

6.2.1 Making judgments about BCI use

As an experimental technology, the main barrier to BCI deployment is the achievement of sufficient communication throughput to merit use in therapeutic and rehabilitative intervention. The level at which experimental technology should be implemented comes down to a judgment relating the risks associated with device use to the benefit achieved by the system. This assessment must account for the many avenues of failure that could occur, whether from implanted electrode rejection, improper classifier calibration, or effector failure in the case of wheelchair or prosthesis. For devices that carry significant risks, only those patients who stand to benefit the most from the system, like a locked-in patient with little residual communication, would be considered good candidates. This application of invasive technology to an already compromised patient population requires consideration of the responsibilities we have to these patients, and our obligations as researchers to improve the quality of life through low risk, high benefit communication systems while also respecting their autonomy. Less rigorous selection criteria may be used if the device carries a lower risk, and there appears to be a clear divide among researchers about current risk/benefit ratios when comparing non-invasive to invasive BCI systems [301]. The difference between research and treatment should be made clear to prospective users of BCI. These systems currently occupy a space between research and treatment, often in an experimental stage, but also capable of providing benefit. If the intent of a BCI intervention is mainly as research, patients may experience “therapeutic miscon- ception”, which can lead to decisions that might not be in their best interest [302]. As an example, the prospect of BCI communication can influence decisions about life support, 138 specifically the choice to receive invasive mechanical ventilation. This decision should not be made without full understanding of the expected outcomes of the BCI protocol. As BCI systems advance towards long-term clinical implementation, some suggest that they be offered only after the individual has chosen to be ventilated [303]. On the other hand, options for BCI communication should be discussed with the patient early in their treatment course. Even though these topics may be upsetting to patients and family, it has been shown that such discussions allow for an enhanced sense of control over the disease [304]. Lastly, the choice a patient makes about communication technologies is not independent of the financial cost to themselves and their family. At this point, BCI technologies remain quite expensive and pose an additional burden to patients and their families. This creates a problem for equal distribution of the technology, which may in part be alleviated by reductions in system prices as commercial systems enter the market, as well as future coverage by insurance policies.

6.2.2 Informed Consent

The procedure of informed consent should instill in the participant an accurate depiction of what to expect from the research or therapeutic intervention, with special care given to the operation of the system and expected performance [305]. In progressive diseases such as ALS, the process of informed consent should begin early, so that the user and family have enough time reach an informed decision [305]. BCI experts agree that the research team should have a common interaction with the patient about the potential risks and beneﬁts [301]. The practice of integrated team care is the standard within ALS clinics, and BCI experts should be working closely with the clinical staff to deliver the same message relating to informed consent and decision making [305]. The goal of the device is to maximize the quality of life for the patient and their family; conveying disjointed expectations about potential beneﬁts could lead to alterations in end-of-life decisions, which could result in deleterious effects on the user and their loved ones [301, 302]. The process of obtaining informed consent when the user is in a non-communicative state relies on the consent of legal representative and possibly the assent of the user. A legal representative may be a family member or a neutral party, depending on the laws 139 of the country. In either case, major issues with representative consent come from the evidence that patients may rate their quality of life as satisfactory and worth preserving, while others may ascribe them to have a poor quality of life [178]. There is general agreement among experts that an attempt should be made to establish BCI communication with a completely locked-in patient if consent of their legal representative is given. However, this does not apply to systems that involve invasive recordings [301].

6.2.3 Issues arising from continuous BCI use

Once a user consents to BCI intervention, special concerns arise from regular and continuous use of the system. One such concern is privacy of bioelectric data generated by the user. With improving sensors and algorithms, the researcher is able to tell more and more about a person’s state of mind or intention [302]. Regulation on the use of that data, which is generally controlled in a research setting, will be less well controlled with commercially-developed products [306]. Also, the issue of handling incidental findings, although not unique to BCI development, could be problematic for all users, but especially for recreational users of the devices. If we are able to record from the brain such detailed information to be able to make assessments about voluntary intention, then it is not implausible to regularly encounter gross abnormalities in related bioelectric phenomena. The question of reporting these incidental findings then becomes highly relevant, as a plan of action needs to be in place for situations affecting otherwise healthy users. A major issue in BCI prosthetic and mobility applications is determining liability in the case of failure or misuse. For BCI applications with high risk, such as wheelchair navigation, at what point is the user responsible for the actions of an imperfect machine? Jens Clausen argues that “...humans are often in control of dangerous and unpredictable tools such as cars and guns. Brain-machine interfaces represent a highly sophisticated case of tool use, but they are still just that. In the eyes of the law, responsibility should not be much harder to disentangle” [307]. Similarly, others note that there are legal structures in place which deal with ascription of liability in our use of machines [308], and many BCI experts anticipate that BCI users will maintain responsibility for the actions of the BCI device [301]. Grubler¨ elaborates on this viewpoint, speculating that proper regulation of high-risk BCI systems will require minimum requirements for de- 140 vice reliability, documentation of user training, regular licensure, and acquisition of liability insurance [309]. Continuation of treatment directives becomes a concern when a patient who has been using the BCI for communication is no longer deemed competent to make their own decisions. Right-to-die decisions made by patients with disorders of consciousness have come under scrutiny due to controversial cases related to brain injuries resulting in disorders of consciousness [310]. Like traumatic brain injury, ALS can eventually lead to a brain state in which communication is no longer feasible, and the prospect of recovery is without precedent. Locked-in syndrome is not considered a disorder of consciousness, but it is necessary for patients to communicate end-of-life directives at regular intervals, in the event further communication becomes infeasible. Treatment directives stated by patients before becoming non-communicative should not be overwritten eas- ily, and should incorporate the legal representative. Ideally, the patient should make decisions about treatment directives in the case of reduced autonomy due to dementia [304]. Other ethical questions for the future-thinking BCI researcher may involve societal impacts [306], unintended changes to the brain and personality [303, 307, 311], especially in BCI-enabled neurorehabilitation regimes [302], the actions of sub-personal mental agency [308], and deleterious side effects of a hyper-efficient mental connection in a physical world [306]. These topics are given a passing glance in this discussion because I, like other researchers in the field [301], feel an inadequate body of research exists to make judgments about these potential consequences of BCI use. APPENDIX A

SUPPLEMENTARY MATERIALS BCI Technology Patient Survey Patient Identiﬁer: 1

Brain-computer interface technology patient survey

Thank you for taking part in this survey. The survey will take approximately 10 minutes to complete. H Section 1: Patient Information

1. Education (years): 2. Technology Use Never Bi-monthly Weekly Bi-weekly Daily Computer/Tablet Use: Last month –––––––––– –––––––––– –––––––––– –––––––––– Month before diagnosis –––––––––– –––––––––– –––––––––– –––––––––– Phone Use: Last month –––––––––– –––––––––– –––––––––– –––––––––– Month before diagnosis –––––––––– –––––––––– –––––––––– ––––––––––

H Section 2: Current assistive technology usage

3a. Have you ever used an assistive device for verbal communication? Yes No Answer questions 3b&c if you answered ‘Yes’ to the previous question. 3b. What forms of assistive verbal communication do you currently use or have used in the past? Text-to-speech synthesizer Eye/forehead tracking Other 3c. How has your quality of life improved with the addition of your assistive verbal communication device? Not At All Minor Improvement Major Improvement

4a. Have you ever used an assistive device for written communication? Yes No Answer questions 4b&c if you answered ‘Yes’ to the previous question. 4b. What forms of assistive written communication do you currently use or have you used in the past? Symbolic communication board Speech-to-text transcriber Eye/forehead tracking Other 4c. How has your quality of life improved with the addition of your assistive written communication device? Not At All Minor Improvement Major Improvement

5a. Do you currently have a feeding tube for feeding assistance? Yes No Answer question 5b if you answered ‘Yes’ to the previous question. 5b. How has your quality of life improved with the addition of your feeding tube? Not At All Minor Improvement Major Improvement BCI Technology Patient Survey Patient Identiﬁer: 2

6a. Have you ever used an assistive device to aid in your mobility during your time with ALS? Yes No Answer questions 6b&c if you answered ‘Yes’ to the previous question. 6b. Which assistive devices have you used now or in the past course of your ALS treatment to aid in your mobility? Cane Walker Manual/Power Wheelchair Brace Other 6c. How has your quality of life improved with the addition of your assistive mobility device? Not At All Minor Improvement Major Improvement

7a. Do you currently use a BiPAP/breathing device? Yes No Answer questions 7b,c,d&e if you answered ‘Yes’ to the previous question. 7b. How often do you use the device per day while not sleeping? Never Less than 2 hours 2-6 hours 6-10 hours More than 10 hours –––––––––––––– –––––––––––––– –––––––––––––– –––––––––––––– 7c. Does it ease your daily activities? Yes No 7d. Does it help you sleep? Yes No 7e. How has your quality of life improved with the addition of your BiPAP/breathing device? Not At All Minor Improvement Major Improvement

8. What are your reasons for not adopting assistive technologies to aid with Communication, Feeding, Mobility, and Breathing? Financial Time to Learn Not Effective Enough Aesthetics Will make me weaker Other 9. Please explain your answers to the previous question in a little more detail.

Section 3: Opinions toward BCI technology

10. Knowing the prognosis for ALS patients includes decreased verbal and written communication ability, how interested would you be in pursuing these BCI assistive communication technologies at some point in the future? Rate your interest in each device on the scale below, with (1) indicating no interest, and (5) indicating great interest. Not Interested –––––––––––––––– Very Interested (1) (2) (3) (4) (5) P300 Speller –––––––– –––––––– –––––––– –––––––– Dasher –––––––– –––––––– –––––––– –––––––– BCI Technology Patient Survey Patient Identiﬁer: 3

11. If at some point in the future you were unable to perform these tasks, in which other BCI functions for assistive technology would you be interested? Rate your preference for each function on the scale below, with (1) indicating no interest, and (5) indicating great interest. Not Interested –––––––––––––––– Very Interested (1) (2) (3) (4) (5) Bed Controls –––––––– –––––––– –––––––– –––––––– Computer Use –––––––– –––––––– –––––––– –––––––– Lift –––––––– –––––––– –––––––– –––––––– Light Switch –––––––– –––––––– –––––––– –––––––– Wheelchair Control –––––––– –––––––– –––––––– –––––––– Recline –––––––– –––––––– –––––––– –––––––– Robotic Arm –––––––– –––––––– –––––––– –––––––– Temperature Control –––––––– –––––––– –––––––– –––––––– Speaker Phone –––––––– –––––––– –––––––– –––––––– Television Control –––––––– –––––––– –––––––– –––––––– 12. If you were to use a BCI system as an assistive technology, specify the importance of each of the following system features. Rate the importance of each item on the scale below, with (1) meaning the feature is not important to you, and (5) indicating great importance. Not Important to me ––––––––––– Very Important (1) (2) (3) (4) (5) Accuracy –––––––– –––––––– –––––––– –––––––– Appearance –––––––– –––––––– –––––––– –––––––– BCI Functions –––––––– –––––––– –––––––– –––––––– Setup Simplicity –––––––– –––––––– –––––––– –––––––– Setup Time –––––––– –––––––– –––––––– –––––––– Speed –––––––– –––––––– –––––––– –––––––– Standby Reliability –––––––– –––––––– –––––––– –––––––– Training Location –––––––– –––––––– –––––––– –––––––– Training Time –––––––– –––––––– –––––––– –––––––– Type of Electrodes –––––––– –––––––– –––––––– –––––––– 13. What would be the minimum accuracy requirement for you to use a BCI device? Less than 60% 60% 70% 80% 90% 95% 100% 14. What would be your required speed of a BCI device used for communication (letters per minute)? Less than 5 5-9 10-14 15-19 20-24 25+ 15. What is the maximum setup time you would tolerate (in minutes)? Less than 10 10-20 21-30 31-45 46-60 More than 60 16. What is the maximum number of training sessions you would desire to reach optimal performance? 1 only 2-5 6-10 11-15 16-20 20+ 17. What is the minimum time you would tolerate for the system to incorrectly leave standby mode? Less than 15 min 30 min 1 hour 2-5 hours 5+ hours Thank you for participating in this survey! 145

Non-technical abstract

A brain-computer interface (BCI) is defined by four criteria. It must (1) record activity from the brain that is (2) intentionally modulated by the user. (3) The processing and classification of neural activity must occur in real time, after which (4) the user receives feedback of the result. The functions of a brain-computer interface can range from communication interfaces and rehabilitation tools, to biofeedback, gaming, training, and vigilance monitoring. In this thesis, I focus on the personalization of BCI systems applied as assistive communication tools for patients with amyotrophic lateral sclerosis (ALS). Also known as Lou Gehrig’s disease, ALS is a neurodegenerative disorder with approximately 6000 new cases in the United States each year. The disease produces degeneration of motor neurons, leading to eventual cessation of voluntary muscular activity, and the need for alternative forms of communication. BCI communication in advanced ALS has achieved limited success, with users often performing at a lower level than young, healthy individuals in whom the devices are regularly tested. Furthermore, only marginal communication has been established in patients whom are completely locked-in, or without any means of communication with the outside world. In this dissertation, personalization of the BCI system is used to overcome some of the disadvantages faced by ALS patients, as well as provide potential solutions to the BCI inefficiency found in late-stage ALS. Three projects were undertaken, the first of which focused on novel predictors of BCI performance in healthy individuals. We employed previously unused brain signatures to assess vigilance before the BCI task, in order to selectively gate the operation of the system. By utilizing one of these features, the amplitude of ongoing motor-related brain oscillations, the communication speed of the interface was able to be substantially increased for certain individuals. The second study focused on the user-to-system personalization of a BCI system for ALS. We determined that psychological factors, more than physical factors, contributed to the acceptance of BCI communication systems among these individuals, and that cognition was also a major determinant of device success. Finally, we performed offline analysis on the BCI data to explore possible avenues of system-to-user personalization. Specifically, feature optimization and novel tools for defining brain connectivity were assessed for applicability within this population. Significant improvements in system performance were made possible by optimiz- 146 ing the feature set, indicating that functional brain changes occurring in ALS warrant an individualized approach to feature extraction. For patients with ALS, the need for assistive BCI devices occurs when other methods for communication fail. The results of this study inform both the clinical guidelines for device prescription and the optimal methods of feature extraction. These were often found to be modulated by disease heterogeneity, specifically psychological changes that often co-occur in the disorder. Future work involving online confirmation of these findings will further validate these tools as essential components of BCI systems. BIBLIOGRAPHY

[1]G ERONIMO,A.,M.KAMRUNNAHAR, and S. J. SCHIFF (2010) “Cue Variation in a Motor Im- agery Task,” in BCI Meeting 2010: 4th International Meeting, Asilomar, CA, poster abstract. [2] ——— (2015) “The use of single trial gating signals to optimize motor-imagery brain-computer interfacing,” Manuscript in preparation.

[3]G ERONIMO,A.,H.E.STEPHENS,S.J.SCHIFF, and Z. SIMMONS (2014) “Acceptance of brain- computer interfaces in amyotrophic lateral sclerosis,” Amyotrophic Lateral Sclerosis and Fron- totemporal Degeneration, (Early Online - November), pp. 1–7.

[4]G ERONIMO,A.,Z.SIMMONS, and S. J. SCHIFF (2015) “Performance predictors of brain- computer interfaces in patients with amyotrophic lateral sclerosis,” Manuscript submitted. [5] ——— (2015) “Sub-pathological threshold repeat lengths of GGGGCC in C9ORF72 correlate with brain-computer interface performance,” Manuscript in preparation.

[6]P FURTSCHELLER,G.,B.Z.ALLISON,C.BRUNNER, ETAL. (2010) “The Hybrid BCI,” Frontiers in Neuroscience, 4(April), p. 11.

[7]P FURTSCHELLER,G.,C.GUGER,G.MULLER¨ ,G.KRAUSZ, and C. NEUPER (2000) “Brain oscillations control hand orthosis in a tetraplegic,” Neuroscience Letters, 292(3), pp. 211 – 214.

[8]W OLPAW,J.R.,N.BIRBAUMER,D.J.MCFARLAND,G.PFURTSCHELLER, and T. M. VAUGHAN (2002) “Brain-computer interfaces for communication and control,” Clinical Neuro- physiology, 113(6), pp. 767 – 791.

[9]B IRBAUMER, N. and P. SAUSENG (2010) “Brain-Computer Interface in Neurorehabilitation,” in Brain-Computer Interfaces (B. Graimann, G. Pfurtscheller, and B. Allison, eds.), The Frontiers Collection, Springer Berlin Heidelberg, pp. 155–169.

[10]B EDE, P., A. BOKDE,M.ELAMIN, ETAL. (2012) “Grey matter correlates of clinical variables in amyotrophic lateral sclerosis (ALS): a neuroimaging study of ALS motor phenotype heterogeneity and cortical focality,” Journal of Neurology, Neurosurgery & Psychiatry, 84(7), pp. 766–773.

[11]B IRBAUMER, N. (2006) “Breaking the silence: brain-computer interfaces (BCI) for communication and motor control.” Psychophysiology, 43(6), pp. 517–32.

[12]S CHOMER, D. L. and F. L. DA SILVA (2011) Niedermeyer’s Electroencephalography: Basic Prin- ciples, Clinical Applications, and Related Fields, sixth ed., Lippincott Williams & Wilkins.

[13]B ERGER, H. (1929) “Uber¨ das Elektrenkephalogramm des Menschen,” Archiv fur¨ Psychiatrie und Nervenkrankheiten, 87. 148

[14]B IRBAUMER, N., T. HINTERBERGER,A.KUBLER, and N. NEUMANN (2003) “The thought- translation device (TTD): neurobehavioral mechanisms and clinical outcome,” Neural Systems and Rehabilitation Engineering, IEEE Transactions on, 11(2), pp. 120 –123.

[15]F ARWELL, L. and E. DONCHIN (1988) “Talking off the top of your head: toward a mental prosthesis utilizing event-related brain potentials,” Electroencephalography and Clinical Neurophysi- ology, 70(6), pp. 510 – 523.

[16]P FURTSCHELLER, G. and A. ARANIBAR (1979) “Evaluation of event-related desynchronization (ERD) preceding and following voluntary self-paced movement,” Electroencephalography and Clinical Neurophysiology, 46(2), pp. 138 – 146.

[17]G EORGOPOULOS,A.,J.KALASKA,R.CAMINITI, and J. MASSEY (1982) “On the relations between the direction of two-dimensional arm movements and cell discharge in primate motor cortex,” The Journal of Neuroscience, 2(11), pp. 1527–1537.

[18]G EORGOPOULOS, A. P.,A. B. SCHWARTZ, and R. E. KETTNER (1986) “Neuronal Population Coding of Movement Direction,” Science, 233, pp. 1416–1419.

[19]C ARMENA,J.M.,M.A.LEBEDEV,R.E.CRIST, ETAL. (2003) “Learning to Control a Brain- Machine Interface for Reaching and Grasping by Primates,” PLoS Biol, 1(2), p. e42.

[20]H OCHBERG,L.R.,D.BACHER,B.JAROSIEWICZ, ETAL. (2012) “Reach and grasp by people with tetraplegia using a neurally controlled robotic arm,” Nature, 485(7398), pp. 372–375.

[21]L EUTHARDT,E.C.,G.SCHALK,J.R.WOLPAW,J.G.OJEMANN, and D. W. MORAN (2004) “A brain-computer interface using electrocorticographic signals in humans,” Journal of Neural Engineering, 1(2), p. 63.

[22]N UNEZ, P. L. and R. SRINIVASAN (2006) Electric Fields in the Brain: The Neurophysics of EEG, second ed., Oxford University Press, New York.

[23]S TERNICKEL, K. and A. I. BRAGINSKI (2006) “Biomagnetism using SQUIDs: status and perspectives,” Superconductor Science and Technology, 19(3), p. S160.

[24]C UFFIN, B. N. and D. COHEN (1979) “Comparison of the magnetoencephalogram and electroencephalogram,” Electroencephalography and Clinical Neurophysiology, 47(2), pp. 132 – 146.

[25]C OHEN, D. and B. N. CUFFIN (1983) “Demonstration of useful differences between magnetoencephalogram and electroencephalogram,” Electroencephalography and Clinical Neurophysiology, 56(1), pp. 38–51.

[26]M ALMIVUO, J., V. SUIHKO, and H. ESKOLA (1997) “Sensitivity distributions of EEG and MEG measurements,” Biomedical Engineering, IEEE Transactions on, 44(3), pp. 196–208.

[27]P HILLIPS,J.,R.LEAHY,J.MOSHER, and B. TIMSARI (1997) “Imaging neural activity using MEG and EEG,” IEEE Eng Med Biol Mag., 16(3), pp. 34–42.

[28] DA SILVA, F. L., H. WIERINGA, and M. PETERS (1991) “Source localization of EEG versus MEG: empirical comparison using visually evoked responses and theoretical considerations,” Brain Topogr., 4(2), pp. 133–142.

[29]C OHEN, D. and B. N. CUFFIN (1991) “EEG versus MEG localization acccuracy: theory and experiment,” Brain Topogr., 4(2), pp. 95–103.

[30]K NAKE,S.,E.HALGREN,H.SHIRAISHI, ETAL. (2006) “The value of multichannel MEG and EEG in the presurgical evaluation of 70 epilepsy patients,” Epilepsy Research, 69(1), pp. 80 – 86.

[31]S CHWAB,K.,C.LIGGES, T. JUNGMANN, ETAL. (2006) “Alpha entrainment in human electroencephalogram and magnetoencephalogram recordings,” NeuroReport, 17(17), pp. 1829–1833. 149

[32]G OLDENHOLZ, D. M., S. P. AHLFORS,M.S.HAM¨ AL¨ AINEN¨ , ETAL. (2009) “Mapping the signal-to-noise-ratios of cortical sources in magnetoencephalography and electroencephalography,” Human Brain Mapping, 30(4), pp. 1077–1086.

[33] VAN DEN BROEK, S., F. REINDERS,M.DONDERWINKEL, and M. PETERS (1998) “Volume conduction effects in EEG and MEG,” Electroencephalography and Clinical Neurophysiology, 106(6), pp. 522 – 534.

[34]M ELLINGER,J.,G.SCHALK,C.BRAUN, ETAL. (2007) “An MEG-based brain-computer interface (BCI).” NeuroImage, 36(3), pp. 581–593.

[35]W ALDERT,S.,H.PREISSL,E.DEMANDT, ETAL. (2008) “Hand Movement Direction Decoded from MEG and EEG,” The Journal of Neuroscience, 28(4), pp. 1000–1008.

[36]M ULLER¨ , M. M., W. TEDER, and S. A. HILLYARD (1997) “Magnetoencephalographic recording of steady-state visual evoked cortical activity,” Brain Topography, 9(3), pp. 163–168.

[37]B RUNNER, P., A. L. RITACCIO, J. F. EMRICH,H.BISCHOF, and G. SCHALK (2011) “Rapid Communication with a P300 Matrix Speller Using Electrocorticographic Signals (ECoG),” Fron- tiers in Neuroscience, 5(February), p. 9.

[38]S IMERAL, J. D., S.-P. KIM,M.J.BLACK, J. P. DONOGHUE, and L. R. HOCHBERG (2011) “Neural control of cursor trajectory and click by a human with tetraplegia 1000 days after implant of an intracortical microelectrode array.” Journal of Neural Engineering, 8(2), p. 025027.

[39]R OSS,D.A.,T.R.HENRY, and L. D. DICKINSON (1993) “A percutaneous epidural screw electrode for intracranial electroencephalogram recordings,” Neurosurgery, 33(2), pp. 332–334.

[40]D RURY,I.,L.SCHUH,D.ROSS, ETAL. (1997) “Ictal patterns in temporal lobe epilepsy recorded by epidural screw electrodes,” Electroencephalography and Clinical Neurophysiology, 102(3), pp. 167 – 174.

[41]W EISKOPF,N.,K.MATHIAK, S. W. BOCK, ETAL. (2004) “Principles of a brain-computer interface (BCI) based on real-time functional magnetic resonance imaging (fMRI),” IEEE Transactions on Biomedical Engineering, 51(6), pp. 966–970.

[42]L UU, S. and T. CHAU (2009) “Decoding subjective preference from single-trial near-infrared spectroscopy signals.” Journal of Neural Engineering, 6(1), p. 016003.

[43]M ILLER,K.J.,S.ZANOS,E.E.FETZ,M. DEN NIJS, and J. G. OJEMANN (2009) “Decoupling the Cortical Power Spectrum Reveals Real-Time Representation of Individual Finger Movements in Humans,” The Journal of Neuroscience, 29(10), pp. 3132–3137.

[44]G ALLEGOS-AYALA,G.,A.FURDEA,K.TAKANO, ETAL. (2014) “Brain communication in a completely locked-in patient using bedside near-infrared spectroscopy,” Neurology, 82(21), pp. 1930–1932.

[45]Z ANDER, T. O. and C. KOTHE (2011) “Towards passive brain-computer interfaces: applying brain-computer interface technology to human-machine systems in general,” Journal of Neural Engineering, 8(2), p. 025005.

[46]A LLISON,B.Z.,D.J.MCFARLAND,G.SCHALK, ETAL. (2008) “Towards an independent brain-computer interface using steady state visual evoked potentials.” Clinical Neurophysiology, 119(2), pp. 399–408.

[47]S EVERENS, M., M. V. DER WAAL,J.FARQUHAR, and P. DESAIN (2014) “Comparing tactile and visual gaze-independent brain-computer interfaces in patients with amyotrophic lateral sclerosis and healthy users,” Clinical Neurophysiology, 125(11), pp. 2297 – 2304. 150

[48]K UBLER¨ ,A.,A.FURDEA,S.HALDER, ETAL. (2009) “A Brain-Computer Interface Controlled Auditory Event-Related Potential (P300) Spelling System for Locked-In Patients,” Annals of the New York Academy of Sciences, 1157(1), pp. 90–100.

[49]A LLISON, B. (2011) “Trends in BCI Research: Progress Today, Backlash Tomorrow?” XRDS, 18(1), pp. 18–22.

[50]C URRAN, E., P. SYKACEK,M.STOKES, ETAL. (2004) “Cognitive tasks for driving a brain- computer interfacing system: a pilot study,” Neural Systems and Rehabilitation Engineering, IEEE Transactions on, 12(1), pp. 48–54.

[51]F RIEDRICH, E. V., R. SCHERER, and C. NEUPER (2013) “Stability of event-related (de-) synchronization during brain-computer interface-relevant mental tasks,” Clinical Neurophysiology, 124(1), pp. 61 – 69.

[52]D YSON, M., F. SEPULVEDA,J.GAN, and S. ROBERTS (2009) “Sequential classiﬁcation of mental tasks vs. idle state for EEG based BCIs,” in Neural Engineering, 2009. NER ’09. 4th International IEEE/EMBS Conference on, pp. 351–354.

[53]F ARADJI, F., R. K. WARD, and G. E. BIRCH (2011) “Toward development of a two-state brain- computer interface based on mental tasks,” Journal of Neural Engineering, 8(4), p. 046014.

[54]S TINEAR, C., W. BYBLOW,M.STEYVERS,O.LEVIN, and S. SWINNEN (2006) “Kinesthetic, but not visual, motor imagery modulates corticomotor excitability,” Experimental Brain Research, 168(1-2), pp. 157–164.

[55]N EUPER,C.,R.SCHERER,M.REINER, and G. PFURTSCHELLER (2005) “Imagery of motor actions: Differential effects of kinesthetic and visual-motor mode of imagery in single-trial EEG,” Cognitive Brain Research, 25(3), pp. 668 – 677.

[56]M ALOUIN, F., C. L. RICHARDS, P. L. JACKSON, ETAL. (2007) “The Kinesthetic and Visual Imagery Questionnaire (KVIQ) for Assessing Motor Imagery in Persons with Physical Disabilities: A Reliability and Construct Validity Study,” Journal of Neurologic Physical Therapy, 31(2).

[57]G REGG,M.,C.HALL, and A. BUTLER (2010) “The MIQ-RS: A Suitable Option for Examining Movement Imagery Ability,” Evidence-Based Complementary and Alternative Medicine, 7(2), pp. 249–257.

[58]J EANNEROD, M. (2001) “Neural Simulation of Action: A Unifying Mechanism for Motor Cogni- tion,” NeuroImage, 14(1), pp. S103 – S109.

[59]L OTZE, M. and U. HALSBAND (2006) “Motor imagery,” Journal of Physiology-Paris, 99(46), pp. 386 – 395.

[60]L ACOURSE,M.G.,E.L.ORR,S.C.CRAMER, and M. J. COHEN (2005) “Brain activation during execution and motor imagery of novel and skilled sequential hand movements,” NeuroImage, 27(3), pp. 505 – 519.

[61]G UILLOT,A.,C.COLLET, V. A. NGUYEN, ETAL. (2008) “Functional neuroanatomical networks associated with expertise in motor imagery,” NeuroImage, 41(4), pp. 1471 – 1483.

[62]M ATTAY, V. S. and D. R. WEINBERGER (1999) “Organization of the human motor system as studied by functional magnetic resonance imaging,” European Journal of Radiology, 30(2), pp. 105–114.

[63]G ASTAUT, H. (1952) “Etude electrocorticopraphique de la reativite des rhythmes rolandiqes,” Re- vue Neurologique, 82(2), pp. 176–182.

[64]P INEDA, J. A. (2005) “The functional signiﬁcance of mu rhythms: Translating seeing and hearing into doing,” Brain Research Reviews, 50(1), pp. 57 – 68. 151

[65]K UHLMAN, W. N. (1978) “Functional topography of the human mu rhythm.” Electroencephalog- raphy and Clinical Neurophysiology, 44(1), pp. 83–93.

[66]B OUYER,J.,C.TILQUIN, and A. ROUGEUL (1983) “Thalamic rhythms in cat during quiet wake- fulness and immobility,” Electroencephalography and Clinical Neurophysiology, 55(2), pp. 180 – 187.

[67]S TERIADE, M., P. GLOOR,R.LLINS, and M.-M. LOPESDA SILVA, F.H. AND-MESULAM (1990) “Basic mechanisms of cerebral rhythmic activities,” Electroencephalography and clinical neurophysiology, 76(6), pp. 481–508.

[68]G OLDMAN,R.I.,J.M.STERN,J.ENGEL JR., and M. S. COHEN (2002) “Simultaneous EEG and fMRI of the alpha rhythm,” NeuroReport, 13(18), pp. 2487–2492.

[69]S OLODKIN, A., P. HLUSTIK,E.E.CHEN, and S. L. SMALL (2004) “Fine modulation in network activation during motor execution and motor imagery.” Cerebral Cortex, 14(11), pp. 1246–1255.

[70]K ASESS,C.H.,C.WINDISCHBERGER,R.CUNNINGTON, ETAL. (2008) “The suppressive in- ﬂuence of SMA on M1 in motor imagery revealed by fMRI and dynamic causal modeling,” Neu- roImage, 40(2), pp. 828 – 837.

[71]D ARVAS, F., R. SCHERER,J.G.OJEMANN, ETAL. (2010) “High gamma mapping using EEG.” NeuroImage, 49(1), pp. 930–938.

[72]S ZURHAJ, W., P. DERAMBURE,E.LABYT, ETAL. (2003) “Basic mechanisms of central rhythms reactivity to preparation and execution of a voluntary movement: a stereoelectroencephalographic study.” Clinical Neurophysiology, 114(1), pp. 107–119.

[73]G ROSSE-WENTRUP,M.,B.SCHOLKOPF¨ , and J. HILL (2011) “Causal inﬂuence of gamma oscillations on the sensorimotor rhythm.” NeuroImage, 56(2), pp. 837–842.

[74]M CFARLAND,D.,L.MINER, T. VAUGHAN, and J. WOLPAW (2000) “Mu and Beta Rhythm Topographies During Motor Imagery and Actual Movements,” Brain Topography, 12, pp. 177– 186.

[75]H ATSOPOULOS, N. G. (2009) “Rhythms in motor processing: functional implications for motor behavior,” Short CourSe II, p. 37.

[76]P FURTSCHELLER, G. and F. H. L. DA SILVA (1999) “Event-related EEG/MEG synchronization and desynchronization: basic principles,” Clinical Neurophysiology, 110(11), pp. 1842 – 1857.

[77]L INDEN, D. E. J. (2005) “The P300: Where in the Brain Is It Produced and What Does It Tell Us?” The Neuroscientist, 11(6), pp. 563–576.

[78]P OLICH, J. (2007) “Updating P300: An integrative theory of P3a and P3b,” Clinical Neurophysi- ology, 118(10), pp. 2128 – 2148.

[79]F ARWELL, L. A. and E. DONCHIN (1988) “Talking off the top of your head: Toward a mental prosthesis utilizing event-related brain potentials,” Electroencephalography and Clinical Neuro- physiology, 70, pp. 510–523.

[80]T OWNSEND,G.,B.K.LAPALLO,C.B.BOULAY, ETAL. (2010) “A novel P300-based brain- computer interface stimulus presentation paradigm: moving beyond rows and columns.” Clinical Neurophysiology, 121(7), pp. 1109–1120.

[81]M ASON,S.,A.BASHASHATI,M.FATOURECHI,K.NAVARRO, and G. BIRCH (2007) “A Com- prehensive Survey of Brain Interface Technology Designs,” Annals of Biomedical Engineering, 35, pp. 137–169. 152

[82]B ASHASHATI,A.,M.FATOURECHI,R.K.WARD, and G. E. BIRCH (2007) “A survey of signal processing algorithms in brain-computer interfaces based on electrical brain signals,” Journal of Neural Engineering, 4(2), p. R32.

[83]L OTTE, F., M. CONGEDO,A.LECUYER´ , F. LAMARCHE, and B. ARNALDI (2007) “A review of classiﬁcation algorithms for EEG-based brain-computer interfaces,” Journal of Neural Engineer- ing, 4(2), p. R1.

[84]S CHLOGL¨ ,A.,C.KEINRATH,D.ZIMMERMANN, ETAL. (2007) “A fully automated correction method of EOG artifacts in EEG recordings,” Clinical Neurophysiology, 118(1), pp. 98 – 104.

[85]C ROFT,R.J.,J.S.CHANDLER,R.J.BARRY,N.R.COOPER, and A. R. CLARKE (2005) “EOG correction: A comparison of four methods,” Psychophysiology, 42(1), pp. 16–24.

[86]S CHIFF, S. J. (2005) “Dangerous phase,” Neuroinformatics, 3(4), pp. 315–317.

[87]N AEEM,M.,C.BRUNNER,R.LEEB,B.GRAIMANN, and G. PFURTSCHELLER (2006) “Seper- ability of four-class motor imagery data using independent components analysis,” Journal of Neu- ral Engineering, 3(3), p. 208.

[88]D ALY,I.,M.BILLINGER,R.SCHERER, and G. MULLER-PUTZ (2013) “On the Automated Re- moval of Artifacts Related to Head Movement From the EEG,” Neural Systems and Rehabilitation Engineering, IEEE Transactions on, 21(3), pp. 427–434.

[89]B ISHOP, C. M. (2006) Pattern Recognition and Machine Learning (Information Science and Statistics), Springer-Verlag New York, Inc., Secaucus, NJ, USA.

[90]K OLES, Z. (1991) “The quantitative extraction and topographic mapping of the abnormal components in the clinical EEG,” Electroencephalography and Clinical Neurophysiology, 79(6), pp. 440 – 447.

[91]M CFARLAND,D.,C.ANDERSON,K.-R.MULLER,A.SCHLOGL, and D. KRUSIENSKI (2006) “BCI meeting 2005-workshop on BCI signal processing: feature extraction and translation,” Neu- ral Systems and Rehabilitation Engineering, IEEE Transactions on, 14(2), pp. 135–138.

[92]S AMAR, V. J., A. BOPARDIKAR,R.RAO, and K. SWARTZ (1999) “Wavelet Analysis of Neuro- electric Waveforms: A Conceptual Tutorial,” Brain and Language, 66(1), pp. 7 – 60.

[93]L EOCANI,L.,C.TORO, P. MANGANOTTI, P. ZHUANG, and M. HALLETT (1997) “Event-related coherence and event-related desynchronization/synchronization in the 10 Hz and 20 Hz EEG during self-paced movements,” Electroencephalography and Clinical Neurophysiology/Evoked Poten- tials Section, 104(3), pp. 199 – 206.

[94]R APPELSBERGER, P., G. PFURTSCHELLER, and O. FILZ (1994) “Calculation of event-related coherence – A new method to study short-lasting coupling between brain areas,” Brain Topography, 7(2), pp. 121–127.

[95]P FURTSCHELLER,G.,C.NEUPER,A.SCHLOGL¨ , and K. LUGGER (1998) “Separability of EEG signals recorded during right and left motor imagery using adaptive autoregressive parameters,” Rehabilitation Engineering, IEEE Transactions on, 6(3), pp. 316 –325.

[96]B LANKERTZ,B.,K.-R.MULLER¨ ,D.KRUSIENSKI, ETAL. (2006) “The BCI competition III: validating alternative approaches to actual BCI problems,” Neural Systems and Rehabilitation En- gineering, IEEE Transactions on, 14(2), pp. 153 –159.

[97]S CHLOGL¨ ,A.,C.VIDAURRE, and K.-R. MULLER¨ (2010) “Adaptive Methods in BCI Research - An Introductory Tutorial,” in Brain-Computer Interfaces (B. Graimann, G. Pfurtscheller, and B. Allison, eds.), The Frontiers Collection, Springer Berlin Heidelberg, pp. 331–355. 153

[98]W U, W., Y. GAO,E.BIENENSTOCK, J. P. DONOGHUE, and M. J. BLACK (2006) “Bayesian population decoding of motor cortical activity using a Kalman ﬁlter.” Neural computation, 18(1), pp. 80–118.

[99]M CCANE,L.M.,E.W.SELLERS,D.J.MCFARLAND, ETAL. (2014) “Brain-computer interface (BCI) evaluation in people with amyotrophic lateral sclerosis,” Amyotrophic Lateral Sclerosis and Frontotemporal Degeneration, 15(3-4), pp. 207–215.

[100]H ILL,N.J.,E.RICCI,S.HAIDER, ETAL. (2014) “A practical, intuitive brain-computer interface for communicating yes or no by listening,” Journal of Neural Engineering, 11(3), p. 035003.

[101]S ELLERS, E. W.,T. M. VAUGHAN, and J. R. WOLPAW (2010) “A brain-computer interface for long-term independent home use,” Amyotrophic Lateral Sclerosis, 11(5), pp. 449–455.

[102]P OKORNY,C.,D.S.KLOBASSA,G.PICHLER, ETAL. (2013) “The auditory P300-based single- switch brain-computer interface: Paradigm transition from healthy subjects to minimally conscious patients,” Artiﬁcial Intelligence in Medicine, 59(2), pp. 81 – 90.

[103]M ILLAN´ , J., F. GALAN´ ,D.VANHOOYDONCK, ETAL. (2009) “Asynchronous non-invasive brain- actuated control of an intelligent wheelchair,” in Engineering in Medicine and Biology Society, 2009. EMBC 2009. Annual International Conference of the IEEE, pp. 3361–3364.

[104]M CFARLAND, D. J., W. A. SARNACKI, and J. R. WOLPAW (2010) “Electroencephalographic (EEG) control of three-dimensional movement,” Journal of Neural Engineering, 7(3), p. 036007.

[105]L AFLEUR,K.,K.CASSADY,A.DOUD, ETAL. (2013) “Quadcopter control in three-dimensional space using a noninvasive motor imagery-based brain-computer interface,” Journal of Neural En- gineering, 10(4), p. 046003.

[106]P FURTSCHELLER,G.,R.SCHERER,R.LEEB, ETAL. (2007) “Viewing moving objects in virtual reality can change the dynamics of sensorimotor EEG rhythms,” Presence: Teleoper. Virtual Environ., 16(1), pp. 111–118.

[107]B AYLISS, J. (2003) “Use of the evoked potential P3 component for control in a virtual apartment,” Neural Systems and Rehabilitation Engineering, IEEE Transactions on, 11(2), pp. 113–116.

[108]L EEB, R., F. LEE,C.KEINRATH, ETAL. (2007) “Brain-Computer Communication: Motivation, Aim, and Impact of Exploring a Virtual Apartment,” Neural Systems and Rehabilitation Engineer- ing, IEEE Transactions on, 15(4), pp. 473–482.

[109]S CHWARTZ, A. B., X. T. CUI,D.J.WEBER, and D. W. MORAN (2006) “Brain-controlled interfaces: movement restoration with neural prosthetics.” Neuron, 52(1), pp. 205–220.

[110]M ITCHELL, J. and G. BORASIO (2007) “Amyotrophic lateral sclerosis,” The Lancet, 369(9578), pp. 2031 – 2041.

[111]R OWLAND, L. P. and N. A. SHNEIDER (2001) “Amyotrophic Lateral Sclerosis,” New England Journal of Medicine, 344(22), pp. 1688–1700.

[112]R OTHSTEIN, J. D. (2009) “Current hypotheses for the underlying biology of amyotrophic lateral sclerosis.” Annals of Neurology, 65(S1), pp. S3–S9.

[113]A NDERSEN, P. M. and A. AL-CHALABI (2011) “Clinical genetics of amyotrophic lateral sclerosis: what do we really know?” Nat Rev Neurol, 7(11), pp. 603–615.

[114]S TRONG,M.J.,G.M.GRACE,M.FREEDMAN, ETAL. (2009) “Consensus criteria for the diagnosis of frontotemporal cognitive and behavioural syndromes in amyotrophic lateral sclerosis,” Amyotrophic Lateral Sclerosis, 10(3), pp. 131–146. 154

[115]C HIO` ,A.,G.LOGROSCINO,O.HARDIMAN, ETAL. (2009) “Prognostic factors in ALS: A critical review.” Amyotrophic lateral sclerosis ofﬁcial publication of the World Federation of Neurology Research Group on Motor Neuron Diseases, 10(5-6), pp. 310–323.

[116]T URNER,M.R.,M.C.KIERNAN, P. N. LEIGH, and K. TALBOT (2009) “Biomarkers in amyotrophic lateral sclerosis.” The Lancet, 8(1), pp. 94–109.

[117]C EDARBAUM,J.M.,N.STAMBLER,E.MALTA, ETAL. (1999) “The ALSFRS-R: a revised ALS functional rating scale that incorporates assessments of respiratory function,” Journal of the Neurological Sciences, 169(1-2), pp. 13 – 21.

[118]Z IMMERMAN, E. K., P. J. ESLINGER,Z.SIMMONS, and A. M. BARRETT (2007) “Emotional perception deﬁcits in amyotrophic lateral sclerosis.” Cognitive and behavioral Neurology, 20(2), pp. 79–82.

[119]L ILLO, P., E. MIOSHI,J.R.BURRELL, ETAL. (2012) “Grey and White Matter Changes across the Amyotrophic Lateral Sclerosis-Frontotemporal Dementia Continuum,” PLoS ONE, 7(8), p. e43993.

[120]M USCULAR DYSTROPHY ASSOCIATION, “Amyotrophic lateral sclerosis: signs and symptoms,” http://mda.org/disease/amyotrophic-lateral-sclerosis/signs-and-symptoms, ac- cessed: 1-20-2015.

[121]R AAPHORST,J.,M. DE VISSER, W. H. J. P. LINSSEN,R.J. DE HAAN, and B. SCHMAND (2010) “The cognitive proﬁle of amyotrophic lateral sclerosis: A meta-analysis,” Amyotrophic Lateral Scler., 11(1-2), pp. 27–37.

[122]L ING,S.-C.,M.POLYMENIDOU, and D. W. CLEVELAND (2013) “Converging Mechanisms in ALS and FTD: Disrupted RNA and Protein Homeostasis,” Neuron, 79(3), pp. 416 – 438.

[123]W OOLLEY,S.C.,D.H.MOORE, and J. S. KATZ (2010) “Insight in ALS: Awareness of behavioral change in patients with and without FTD,” Amyotrophic Lateral Sclerosis, 11(1-2), pp. 52–56.

[124]P HUKAN, J., N. P. PENDER, and O. HARDIMAN (2007) “Cognitive impairment in amyotrophic lateral sclerosis,” The Lancet Neurology, 6(11), pp. 994 – 1003.

[125]N EARY,D.,J.S.SNOWDEN, and D. M. MANN (2000) “Cognitive change in motor neurone disease/amyotrophic lateral sclerosis (MND/ALS).” Journal of the Neurological Sciences, 180(1- 2), pp. 15–20.

[126]I RWIN, D., C. F. LIPPA, and J. SWEARER (2007) “Cognition and Amyotrophic Lateral Sclerosis (ALS),” American Journal of Alzheimer’s Disease and Other Dementias, 22(4), pp. 300–312.

[127]A BRAHAMS, S., P. LEIGH,A.HARVEY, ETAL. (2000) “Verbal ﬂuency and executive dysfunction in amyotrophic lateral sclerosis (ALS),” Neuropsychologia, 38(6), pp. 734 – 747.

[128]P HUKAN,J.,M.ELAMIN, P. BEDE, ETAL. (2012) “The syndrome of cognitive impairment in amyotrophic lateral sclerosis: a population-based study,” Journal of Neurology, Neurosurgery & Psychiatry, 83(1), pp. 102–108.

[129]J ELSONE-SWAIN,L.M.,C.PERSAD,K.L.VOTRUBA, ETAL. (2012) “The relationship between depressive symptoms, disease state, and cognition in amyotrophic lateral sclerosis.” Frontiers in Psychology, 3(542).

[130]T AYLOR,L.J.,R.G.BROWN,S.TSERMENTSELI, ETAL. (2012) “Is language impairment more common than executive dysfunction in amyotrophic lateral sclerosis?” Journal of Neurology, Neu- rosurgery & Psychiatry, 84(5), pp. 494–498. 155

[131]G IORDANA, M. T., P. FERRERO,S.GRIFONI, ETAL. (2011) “Dementia and cognitive impairment in amyotrophic lateral sclerosis: a review.” Neurological sciences ofﬁcial journal of the Ital- ian Neurological Society and of the Italian Society of Clinical Neurophysiology, 32(1), pp. 9–16.

[132]R OBERTS-SOUTH,A.,K.FINDLATER,M.J.STRONG, and J. ORANGE (2012) “Longitudinal Changes in Discourse Production in Amyotrophic Lateral Sclerosis,” Semin Speech Lang, 33(01), pp. 79–94.

[133]H ANAGASI,H.A.,I.GURVIT,N.ERMUTLU, ETAL. (2002) “Cognitive impairment in amyotrophic lateral sclerosis: evidence from neuropsychological investigation and event-related potentials,” Cognitive Brain Research, 14(2), pp. 234 – 244.

[134]R OBBERECHT, W. and T. PHILIPS (2013) “The changing scene of amyotrophic lateral sclerosis,” Nat Rev Neurosci, 14(4), pp. 248–264.

[135]M ORRIS,H.,A.WAITE,N.WILLIAMS,J.NEAL, and D. BLAKE (2012) “Recent Advances in the Genetics of the ALS-FTLD Complex,” Current Neurology and Neuroscience Reports, 12(3), pp. 243–250.

[136]F ECTO, F. and T. SIDDIQUE (2012) “What is repeated in ALS and FTLD,” The Lancet Neurology, 11(1), pp. 25 – 27.

[137]A GOSTA, F., A. CHIO` ,M.COSOTTINI, ETAL. (2010) “The Present and the Future of Neu- roimaging in Amyotrophic Lateral Sclerosis,” American Journal of Neuroradiology, 31(10), pp. 1769–1777.

[138]S TRONG, M. J. (2001) “Progress in clinical neurosciences: the evidence for ALS as a multisystems disorder of limited phenotypic expression,” The Canadian Journal of Neurological Sciences, 28(4), pp. 283–298.

[139]M EZZAPESA,D.,A.CECCARELLI, F. DICUONZO, ETAL. (February 2007) “Whole-Brain and Regional Brain Atrophy in Amyotrophic Lateral Sclerosis,” American Journal of Neuroradiology, 28(2), pp. 255–259.

[140]F RIEDLANDER, W. J. (1956) “EEG changes in amyotrophic lateral sclerosis,” Electroencephalog- raphy and Clinical Neurophysiology, 8(4), pp. 678 – 681.

[141]G IL,R.,L.ZAI, J. P. NEAU, ETAL. (1995) “Event-related auditory evoked potentials and amyotrophic lateral sclerosis.” Archives of Neurology, 52(3), pp. 182–187.

[142]B EUKELMAN,D.,S.FAGER, and A. NORDNESS (2011) “Communication Support for People with ALS,” Neurology research international, 2011, p. 714693.

[143]L ULE´,D.,C.ZICKLER,S.HACKER¨ , ETAL. (2009) “Life can be worth living in locked-in syndrome,” in Coma Science: Clinical and Ethical Implications (N. D. S. Steven Laureys and A. M. Owen, eds.), vol. 177 of Progress in Brain Research, Elsevier, pp. 339 – 351.

[144]M URGUIALDAY,A.R.,J.HILL,M.BENSCH, ETAL. (2011) “Transition from the locked in to the completely locked-in state: A physiological analysis,” Clinical Neurophysiology, 122(5), pp. 925 – 933.

[145]M ITSUMOTO, H. and J. G. RABKIN (2007) “Palliative Care for Patients With Amyotrophic Lat- eral Sclerosis,” JAMA: The Journal of the American Medical Association, 298(2), pp. 207–216.

[146]B IRBAUMER,N.,A.R.MURGUIALDAY, and L. COHEN (2008) “Brain-computer interface in paralysis,” Current opinion in neurology, 21(6), pp. 634–638.

[147]B EUKELMAN,D.R.,S.FAGER,L.BALL, and A. DIETZ (2007) “AAC for adults with acquired neurological conditions: A review,” Augmentative and Alternative Communication, 23(3), pp. 230– 242. 156

[148]Z ICKLER, C., V. D. DONNA, V. KAISER, ETAL. (2009) “Brain Computer Interaction Appli- cations for People with Disabilities: Deﬁning User Needs and User Requirements,” in AAATE, p. 5.

[149]B ALL,L.J.,D.R.BEUKELMAN, and G. L. PATTEE (2004) “Acceptance of Augmentative and Alternative Communication Technology by Persons with Amyotrophic Lateral Sclerosis,” Aug- mentative and Alternative Communication, 20(2), pp. 113–122.

[150]B ALL,L.J.,D.R.BEUKELMAN,E.ANDERSON, ETAL. (2007) “Duration of AAC technology use by persons with ALS,” Journal of Medical Speech Language Pathology, 15(4), p. 371.

[151]P IRES,G.,U.NUNES, and M. CASTELO-BRANCO (2011) “Statistical spatial ﬁltering for a P300- based BCI: tests in able-bodied, and patients with cerebral palsy and amyotrophic lateral sclerosis.” Journal of Neuroscience Methods, 195(2), pp. 270–281.

[152]S ELLERS, E. W. and E. DONCHIN (2006) “A P300-based brain-computer interface: initial tests by ALS patients.” Clinical Neurophysiology, 117(3), pp. 538–548.

[153]K UBLER¨ , A., F. NIJBOER,J.MELLINGER, ETAL. (2005) “Patients with ALS can use sensorimotor rhythms to operate a brain-computer interface,” Neurology, 64(10), pp. 1775–1777.

[154]B AI, O., P. LIN,D.HUANG, D.-Y. FEI, and M. K. FLOETER (2010) “Towards a user-friendly brain-computer interface: initial tests in ALS and PLS patients.” Clinical Neurophysiology, 121(8), pp. 1293–1303.

[155]K ASAHARA, T., K. TERASAKI, Y. OGAWA, ETAL. (2012) “The correlation between motor impairments and event-related desynchronization during motor imagery in ALS patients.” BMC neuroscience, 13(1), p. 66.

[156]G IBBONS,Z.C.,A.RICHARDSON,D.NEARY, and J. S. SNOWDEN (2008) “Behaviour in amyotrophic lateral sclerosis,” Amyotrophic Lateral Sclerosis, 9(2), pp. 67–74.

[157]H ILL, N. J., T. N. LAL,M.SCHRODER¨ , ETAL. (2006) “Classifying EEG and ECoG signals without subject training for fast BCI implementation: comparison of nonparalyzed and completely paralyzed subjects,” IEEE Transactions on Neural Systems and Rehabilitation Engineering, 14(2), pp. 183–186.

[158]N IJBOER, F., N. BIRBAUMER, and A. KUBLER¨ (2010) “The inﬂuence of psychological state and motivation on brain-computer interface performance in patients with amyotrophic lateral sclerosis - a longitudinal study.” Frontiers in neuroscience, 4(July), p. 13.

[159]L EEB,R.,D.FRIEDMAN,G.R.MULLER¨ -PUTZ, ETAL. (2007) “Self-paced (Asynchronous) BCI Control of a Wheelchair in Virtual Environments: A Case Study with a Tetraplegic,” Intell. Neuroscience, 2007, pp. 7:1–7:12.

[160]D IEZ, P., V. MUT,E.AVILA PERONA, and E. LACIAR LEBER (2011) “Asynchronous BCI control using high-frequency SSVEP,” Journal of NeuroEngineering and Rehabilitation, 8(1), p. 39.

[161]P FURTSCHELLER, G., T. SOLIS-ESCALANTE,R.ORTNER, P. LINORTNER, and G. MULLER¨ - PUTZ (2010) “Self-Paced Operation of an SSVEP-Based Orthosis With and Without an Imagery- Based “Brain Switch:” A Feasibility Study Towards a Hybrid BCI,” Neural Systems and Rehabil- itation Engineering, IEEE Transactions on, 18(4), pp. 409–414.

[162]T ONIN,L.,R.LEEB,M.TAVELLA,S.PERDIKIS, and J. DEL MILLAN´ (2010) “The role of shared-control in BCI-based telepresence,” in Systems Man and Cybernetics (SMC), 2010 IEEE International Conference on, pp. 1462–1466.

[163]M ASON, S. and G. BIRCH (2000) “A brain-controlled switch for asynchronous control applications,” Biomedical Engineering, IEEE Transactions on, 47(10), pp. 1297–1307. 157

[164]S CHERER, R., F. LEE,A.SCHLOGL¨ , ETAL. (2008) “Toward Self-Paced Brain-Computer Com- munication: Navigation Through Virtual Worlds,” Biomedical Engineering, IEEE Transactions on, 55(2), pp. 675–682.

[165]G ALAN´ , F., M. NUTTIN,E.LEW, ETAL. (2008) “A brain-actuated wheelchair: Asynchronous and non-invasive Brain-computer interfaces for continuous control of robots,” Clinical Neurophys- iology, 119(9), pp. 2159 – 2169.

[166]B LANKERTZ,B.,C.SANNELLI,S.HALDER, ETAL. (2010) “Neurophysiological predictor of SMR-based BCI performance,” NeuroImage, 51(4), pp. 1303 – 1309.

[167]H AMMER,E.M.,S.HALDER,B.BLANKERTZ, ETAL. (2012) “Psychological predictors of SMR-BCI performance,” Biological Psychology, 89(1), pp. 80 – 86.

[168]H ALDER,S.,B.VARKUTI,M.BOGDAN, ETAL. (2013) “Prediction of brain-computer interface aptitude from individual brain structure,” Frontiers in Human Neuroscience, 7(105).

[169]H ALDER,S.,C.A.RUF,A.FURDEA, ETAL. (2013) “Prediction of P300 BCI Aptitude in Severe Motor Impairment,” PLoS ONE, 8(10), p. e76148.

[170]G ROSSE-WENTRUP, M. and B. SCHOLKOPF¨ (2013) “A Review of Performance Variations in SMR-Based Brain-Computer Interfaces (BCIs),” in Brain-Computer Interface Research (C. Guger, B. Z. Allison, and G. Edlinger, eds.), SpringerBriefs in Electrical and Computer Engineering, Springer Berlin Heidelberg, pp. 39–51.

[171]H ALDER,S.,D.AGORASTOS,R.VEIT, ETAL. (2011) “Neural mechanisms of brain-computer interface control,” NeuroImage, 55(4), pp. 1779 – 1790.

[172]G ROSSE-WENTRUP, M. and B. SCHOLKOPF¨ (2012) “High gamma-power predicts performance in sensorimotor-rhythm brain-computer interfaces,” Journal of Neural Engineering, 9(4), p. 046001.

[173]M AHMOUDI, B. and A. ERFANIAN (2006) “Electro-encephalogram based brain-computer interface: improved performance by mental practice and concentration skills,” Medical and Biological Engineering and Computing, 44(11), pp. 959–969.

[174]B URDE, W. and B. BLANKERTZ (2006) “Is the locus of control of reinforcement a predictor of brain-computer interface performance?” in Proceedings of the 3rd International BrainComputer Interface Workshop and Training Course, pp. 1–2.

[175]V UCKOVIC, A. and B. A. OSUAGWU (2013) “Using a motor imagery questionnaire to estimate the performance of a Brain-Computer Interface based on object oriented motor imagery,” Clinical Neurophysiology, 124(8), pp. 1586 – 1595.

[176]E SKANDARI, P. and A. ERFANIAN (2008) “Improving the performance of brain-computer interface through meditation practicing,” in Engineering in Medicine and Biology Society, 2008. EMBS 2008. 30th Annual International Conference of the IEEE, pp. 662–665.

[177]S PATARO,R.,M.CIRIACONO,C.MANNO, and V. LA BELLA (2014) “The eye-tracking computer device for communication in amyotrophic lateral sclerosis,” Acta Neurologica Scandinavica, 130(1), pp. 40–45.

[178]B IRBAUMER,N.,G.GALLEGOS-AYALA,M.WILDGRUBER,S.SILVONI, and S. SOEKADAR (2014) “Direct Brain Control and Communication in Paralysis,” Brain Topography, 27(1), pp. 4– 11.

[179]D E MASSARI,D.,C.RUF,A.FURDEA, ETAL. (2013) “Towards Communication in the Com- pletely Locked-In State: Neuroelectric Semantic Conditioning BCI,” in Brain-Computer Interface Research (C. Guger, B. Z. Allison, and G. Edlinger, eds.), SpringerBriefs in Electrical and Com- puter Engineering, Springer Berlin Heidelberg, pp. 111–118. 158

[180]K UBLER¨ , A. and N. BIRBAUMER (2008) “Brain-computer interfaces and communication in paralysis: Extinction of goal directed thinking in completely paralysed patients?” Clinical Neurophysi- ology, 119(11), pp. 2658 – 2666.

[181]W EI, Q., Y. WANG,X.GAO, and S. GAO (2007) “Amplitude and phase coupling measures for feature extraction in an EEG-based brain-computer interface,” Journal of Neural Engineering, 4(2), p. 120.

[182]A NDREASSI,J.,H.OKAMURA, and M. STERN (1975) “Hemispheric asymmetries in the visual cortical evoked potential as a function of stimulus location,” Psychophysiology, 12(5), pp. 541– 546.

[183]R UGG, M. and J. BEAUMONT (1978) “Interhemispheric asymmetries in the visual evoked response: Effects of stimulus lateralisation and task,” Biological Psychology, 6(4), pp. 283 – 292.

[184]S CHLOGL¨ , A., F. LEE,H.BISCHOF, and G. PFURTSCHELLER (2005) “Characterization of four- class motor imagery EEG data for the BCI-competition 2005,” Journal of Neural Engineering, 2(4), p. L14.

[185]D IAS,N.,M.KAMRUNNAHAR, P. MENDES,S.SCHIFF, and J. CORREIA (2010) “Feature selection on movement imagery discrimination and attention detection,” Medical & Biological En- gineering & Computing, 48(4), pp. 331–341.

[186]S AUSENG, P., W. KLIMESCH,C.GERLOFF, and F. HUMMEL (2009) “Spontaneous locally re- stricted EEG alpha activity determines cortical excitability in the motor cortex,” Neuropsychologia, 47(1), pp. 284 – 288.

[187]T HUT, G. and C. MINIUSSI (2009) “New insights into rhythmic brain activity from TMS EEG studies,” Trends in Cognitive Sciences, 13(4), pp. 182 – 189.

[188]B ISHOP, G. H. (1932) “Cyclic changes in excitability of the optic pathway of the rabbit,” American Journal of Physiology – Legacy Content, 103(1), pp. 213–224.

[189]P LONER,M.,J.GROSS,L.TIMMERMANN,B.POLLOK, and A. SCHNITZLER (2006) “Oscilla- tory activity reﬂects the excitability of the human somatosensory system,” NeuroImage, 32(3), pp. 1231 – 1236.

[190]K RUGLIKOV, S. Y. and S. J. SCHIFF (2003) “Interplay of electroencephalogram phase and auditory-evoked neural activity.” Journal of Neuroscience, 23(31), pp. 10122–10127.

[191]T REDER, M. and B. BLANKERTZ (2010) “(C)overt attention and visual speller design in an ERP- based brain-computer interface,” Behavioral and Brain Functions, 6(1), p. 28.

[192]H ILLYARD, S. A. and L. ANLLO-VENTO (1998) “Event-related brain potentials in the study of visual selective attention,” Proceedings of The National Academy of Sciences, 95, pp. 781–787.

[193]R AILO,H.,M.KOIVISTO, and A. REVONSUO (2011) “Tracking the processes behind conscious perception: A review of event-related potential correlates of visual consciousness,” Consciousness and Cognition, 20(3), pp. 972 – 983.

[194]D IAS,N.,L.R.JACINTO, P. MENDES, and J. CORREIA (2009) “Visual gate for brain-computer interfaces,” in Engineering in Medicine and Biology Society, 2009. EMBC 2009. Annual Interna- tional Conference of the IEEE, pp. 532–535.

[195]M CFARLAND,D.J.,L.M.MCCANE, S. V. DAVID, and J. R. WOLPAW (1997) “Spatial ﬁlter selection for EEG-based communication,” Electroencephalography and Clinical Neurophysiology, 103(3), pp. 386 – 394.

[196]M ITRA, P. and H. BOKIL (2008, http://chronux.org) Observed Brain Dynamics, Oxford University Press. 159

[197]G ASSER, T., P. BACHER¨ , and J. MOCKS¨ (1982) “Transformations towards the normal distribution of broad band spectral parameters of the EEG,” Electroencephalography and Clinical Neurophys- iology, 53(1), pp. 119 – 124.

[198]O DOM, J. V., M. BACH,M.BRIGELL, ETAL. (2010) “ISCEV standard for clinical visual evoked potentials (2009 update),” Documenta ophthalmologica, 120(1), pp. 111–119.

[199]O PPENHEIM, A. V. (ed.) (1978) Applications of Digital Signal Processing, chap. Applications of Digital Signal Processing to Radar, Prentice–Hall, p. 250.

[200]L EVANON, N. and E. MOZESON (2004) “Matched Filter,” in Radar Signals, Wiley, pp. 20–33.

[201]L AGE-CASTELLANOS,A.,E.MART´INEZ-MONTES,J.A.HERNANDEZ´ -CABRERA, and L.GALAN´ (2010) “False discovery rate and permutation test: An evaluation in ERP data analysis,” Statistics in Medicine, 29(1), pp. 63–74.

[202]M AKI¨ , H. and R. J. ILMONIEMI (2010) “EEG oscillations and magnetically evoked motor potentials reﬂect motor system excitability in overlapping neuronal populations,” Clinical Neurophysi- ology, 121(4), pp. 492 – 501.

[203]H ALDER,S.,D.AGORASTOS,R.VEIT, ETAL. (2011) “Neural mechanisms of brain-computer interface control,” NeuroImage, 55(4), pp. 1779 – 1790.

[204]R ISNER,M.L.,C.J.AURA,J.E.BLACK, and T. J. GAWNE (2009) “The Visual Evoked Potential is independent of surface alpha rhythm phase,” NeuroImage, 45(2), pp. 463 – 469.

[205]M ACKAY, W. A. (1997) “Synchronized neuronal oscillations and their role in motor processes,” Trends in Cognitive Sciences, 1(5), pp. 176 – 183.

[206]N IJBOER, F., E. W. SELLERS,J.MELLINGER, ETAL. (2008) “A P300-based brain-computer interface for people with amyotrophic lateral sclerosis.” Clinical Neurophysiology, 119(8), pp. 1909–1916.

[207]M IOSHI,E.,J.CAGA, P. LILLO, ETAL. (2013) “Neuropsychiatric changes precede classic motor symptoms in ALS and do not affect survival,” Neurology, 82(2), pp. 149–155.

[208]H UGGINS, J. E.,P. A. WREN, and K. L. GRUIS (2011) “What would brain-computer interface users want? Opinions and priorities of potential users with amyotrophic lateral sclerosis.” Amy- otrophic lateral sclerosis ofﬁcial publication of the World Federation of Neurology Research Group on Motor Neuron Diseases, 12(5), pp. 1–8.

[209]B ROOKS,B.R.,R.G.MILLER,M.SWASH, and T. L. MUNSAT (2000) “El Escorial revisited: Revised criteria for the diagnosis of amyotrophic lateral sclerosis.” Amyotrophic Lateral Sclerosis & Other Motor Neuron Disorders, 1(5), pp. 293 – 299.

[210]W OOLLEY,S.C.,M.K.YORK,D.H.MOORE, ETAL. (2010) “Detecting frontotemporal dysfunction in ALS: Utility of the ALS Cognitive Behavioral Screen (ALS-CBS™),” Amyotrophic Lateral Sclerosis, 11(3), pp. 303–311.

[211]C ZAPLINSKI,A.,A.YEN, and S. APPEL (2006) “Amyotrophic lateral sclerosis: early predictors of prolonged survival,” Journal of Neurology, 253(11), pp. 1428–1436.

[212]G ROSSMAN,A.B.,S.WOOLLEY-LEVINE, W. G. BRADLEY, and R. G. MILLER (2007) “De- tecting neurobehavioral changes in amyotrophic lateral sclerosis.” Amyotrophic Lateral Scler., 8(1), pp. 56–61.

[213]B IN,G.,X.GAO, Y. WANG, ETAL. (2011) “A high-speed BCI based on code modulation VEP,” Journal of Neural Engineering, 8(2), p. 025015. 160

[214]N EARY,D.,J.S.SNOWDEN,L.GUSTAFSON, ETAL. (1999) “Frontotemporal lobar degeneration: a consensus on clinical diagnostic criteria.” Neurology, 51(6), pp. 1546–1554.

[215]M ENDEZ, M. F. and J. S. SHAPIRA (2011) “Loss of emotional insight in behavioral variant frontotemporal dementia or “frontal anosodiaphoria”,” Consciousness and Cognition, 20(4), pp. 1690 – 1696.

[216]K UBA,M.,J.KREMLA´ CEKˇ ,J.LANGROVA´ , ETAL. (2012) “Aging effect in pattern, motion and cognitive visual evoked potentials,” Vision Research, 62(0), pp. 9 – 16.

[217]J UCKEL,G.,S.KARCH, W. KAWOHL, ETAL. (2012) “Age effects on the P300 potential and the corresponding fMRI BOLD-signal,” NeuroImage, 60(4), pp. 2027 – 2034.

[218]P OLICH, J. (2011) Handbook of event-related potential components, chap. Neuropsychology of P300, Oxford University Press.

[219]P AULUS,K.,I.MAGNANO,M.PIRAS, ETAL. (2002) “Visual and auditory event-related potentials in sporadic amyotrophic lateral sclerosis,” Clinical Neurophysiology, 113(6), pp. 853 – 861.

[220]S ILVONI,S.,C.VOLPATO,M.CAVINATO, ETAL. (2009) “P300-Based Brain-Computer Inter- face Communication: Evaluation and Follow-up in Amyotrophic Lateral Sclerosis,” Frontiers in neuroscience, 3(June), p. 12.

[221]M ARCHETTI, M., F. PICCIONE,S.SILVONI,L.GAMBERINI, and K. PRIFTIS (2013) “Covert Visuospatial Attention Orienting in a Brain-Computer Interface for Amyotrophic Lateral Sclerosis Patients,” Neurorehabilitation and Neural Repair, 27(5), pp. 430–438.

[222]R AGGI,A.,S.IANNACCONE, and S. F. CAPPA (2010) “Event-related brain potentials in amyotrophic lateral sclerosis: A review of the international literature,” Amyotrophic Lateral Sclerosis, 11(1-2), pp. 16–26.

[223]S ANTHOSH,J.,M.BHATIA,S.SAHU, and S. ANAND (2004) “Quantitative EEG analysis for assessment to ‘plan’ a task in amyotrophic lateral sclerosis patients: a study of executive functions (planning) in ALS patients,” Cognitive Brain Research, 22(1), pp. 59 – 66.

[224]L INDAU, M., V. JELIC,A.C.JOHANSSON,S.E.,L.WAHLUND, and O. ALMKVIST (2003) “Quantitative EEG abnormalities and cognitive dysfunctions in frontotemporal dementia and Alzheimer’s disease,” Dementia and Geriatric Cognitive Disorders, 15, pp. 106–14.

[225]RC ORE TEAM (2013) R: A Language and Environment for Statistical Computing, R Foundation for Statistical Computing, Vienna, Austria, ISBN 3-900051-07-0.

[226]R ICCIO,A.,L.SIMIONE, F. SCHETTINI, ETAL. (2013) “Attention and P300-based BCI performance in people with amyotrophic lateral sclerosis,” Frontiers in Human Neuroscience, 7(732).

[227]S ALMON, D. P. and D. T. STUSS (2013) “Executive functions can help when deciding on the frontotemporal dementia diagnosis,” Neurology, 80(24).

[228]B RUNNER, P., S. JOSHI,S.BRISKIN, ETAL. (2010) “Does the ‘P300’ speller depend on eye gaze?” Journal of Neural Engineering, 7(5), p. 056013.

[229]B EUKELMAN, D. and P. MIRENDA Augmentative and Alternative Communication, second edition ed., Brookes.

[230]O RHAN,U.,D.ERDOGMUS,B.ROARK, ETAL. (2011) “Fusion with language models improves spelling accuracy for ERP-based brain computer interface spellers,” in Engineering in Medicine and Biology Society,EMBC, 2011 Annual International Conference of the IEEE, pp. 5774–5777. 161

[231]S ALTYKOV,K.,E.BARK, and M. KOULIKOV (2014) “Characteristics of event-related potentials in response to symbolical and alphabetical stimulation matrices used in a P300-based brain- computer interface,” Human Physiology, 40(4), pp. 367–374.

[232]B ECK,J.,M.POULTER,D.HENSMAN, ETAL. (2013) “Large C9orf72 Hexanucleotide Repeat Expansions Are Seen in Multiple Neurodegenerative Syndromes and Are More Frequent Than Expected in the UK Population,” The American Journal of Human Genetics, 92(3), pp. 345 – 353.

[233]C OOPER-KNOCK, J., P. J. SHAW, and J. KIRBY (2014) “The widening spectrum of C9ORF72- related disease; genotype/phenotype correlations and potential modiﬁers of clinical phenotype,” Acta Neuropathologica, 127(3), pp. 333–345.

[234]T AN, E.-C. and P. S. LAI (2005) “Molecular diagnosis of neurogenetic disorders involving trinucleotide repeat expansions,” Expert Review of Molecular Diagnostics, 5(1), pp. 101–109.

[235]G IJSELINCK, I., T. V. LANGENHOVE,J. VAN DER ZEE, ETAL. (2012) “A C9orf72 pro- moter repeat expansion in a Flanders-Belgian cohort with disorders of the frontotemporal lobar degeneration-amyotrophic lateral sclerosis spectrum: a gene identiﬁcation study,” The Lancet Neu- rology, 11(1), pp. 54 – 65.

[236]B YRNE,S.,M.ELAMIN, P. BEDE, ETAL. (2012) “Cognitive and clinical characteristics of patients with amyotrophic lateral sclerosis carrying a C9orf72 repeat expansion: a population-based cohort study,” The Lancet Neurology, 11(3), pp. 232 – 240.

[237]I RWIN, D. J.,C. T. MCMILLAN,J.BRETTSCHNEIDER, ETAL. (2013) “Cognitive decline and reduced survival in C9orf72 expansion frontotemporal degeneration and amyotrophic lateral sclerosis,” Journal of Neurology, Neurosurgery & Psychiatry, 84(2), pp. 163–169.

[238]H UBERS¨ ,A.,N.MARROQUIN,B.SCHMOLL, ETAL. (2014) “Polymerase chain reaction and Southern blot-based analysis of the C9orf72 hexanucleotide repeat in different motor neuron diseases,” Neurobiology of Aging, 35(5), pp. 1214.e1–e6.

[239] VAN BLITTERSWIJK,M.,M.DEJESUS-HERNANDEZ,E.NIEMANTSVERDRIET, ETAL. (2013) “Association between repeat sizes and clinical and pathological characteristics in carriers of C9ORF72 repeat expansions (Xpansize-72): a cross-sectional cohort study,” The Lancet Neurol- ogy, 12(10), pp. 978 – 988.

[240]C OOPER-KNOCK,J.,C.HEWITT,J.R.HIGHLEY, ETAL. (2012) “Clinico-pathological features in amyotrophic lateral sclerosis with expansions in C9ORF72,” Brain, 135(3), pp. 751–764.

[241]B ENUSSI,L.,G.ROSSI,M.GLIONNA, ETAL. (2014) “C9ORF72 Hexanucleotide Repeat Num- ber in Frontotemporal Lobar Degeneration: A Genotype-Phenotype Correlation Study,” Journal of Alzheimer’s Disease, 38(4), pp. 799–808.

[242]M AJOUNIE,E.,A.E.RENTON,K.MOK, ETAL. (2012) “Frequency of the C9orf72 hexanucleotide repeat expansion in patients with amyotrophic lateral sclerosis and frontotemporal dementia: a cross-sectional study,” The Lancet Neurology, 11(4), pp. 323 – 330.

[243]W ILLIAMS,K.L.,J.A.FIFITA,S.VUCIC, ETAL. (2013) “Pathophysiological insights into ALS with C9ORF72 expansions,” Journal of Neurology, Neurosurgery & Psychiatry, 84(8), pp. 931–935.

[244]D EJESUS-HERNANDEZ,M.,I.R.MACKENZIE,A.L.BOEVE,BRADLEY F. ANS BOXER, ETAL. (2011) “Expanded GGGGCC Hexanucleotide Repeat in Noncoding Region of C9ORF72 Causes Chromosome 9p-Linked FTD and ALS,” Neuron, 72(2), pp. 245–256.

[245]R ENTON,A.E.,E.MAJOUNIE,A.WAITE, ETAL. (2011) “A Hexanucleotide Repeat Expansion in C9ORF72 Is the Cause of Chromosome 9p21-Linked ALS-FTD,” Neuron, 72(2), pp. 257 – 268. 162

[246]C RUTS,M.,I.GIJSELINCK, T. V. LANGENHOVE,J. VAN DER ZEE, and C. V. BROECKHOVEN (2013) “Current insights into the C9orf72 repeat expansion diseases of the FTLD/ALS spectrum,” Trends in Neurosciences, 36(8), pp. 450 – 459.

[247]D OLS-ICARDO,O.,A.GARCA-REDONDO,R.ROJAS-GARCA, ETAL. (2014) “Characteriza- tion of the repeat expansion size in C9orf72 in amyotrophic lateral sclerosis and frontotemporal dementia,” Human Molecular Genetics, 23(3), pp. 749–754.

[248]G OMEZ´ -TORTOSA,E.,J.GALLEGO,R.GUERRERO-LOPEZ´ , ETAL. (2013) “C9ORF72 hexanucleotide expansions of 20-22 repeats are associated with frontotemporal deterioration,” Neurology, 80(4), pp. 366–370.

[249]B YRNE,S.,M.HEVERIN,M.ELAMIN,C.WALSH, and O. HARDIMAN (2014) “Intermediate repeat expansion length in C9orf72 may be pathological in amyotrophic lateral sclerosis,” Amy- otrophic Lateral Sclerosis and Frontotemporal Degeneration, 15(1-2), pp. 148–150.

[250]R UTHERFORD,N.J.,M.G.HECKMAN,M.DEJESUS-HERNANDEZ, ETAL. (2012) “Length of normal alleles of C9ORF72 GGGGCC repeat do not inﬂuence disease phenotype,” Neurobiology of Aging, 33(12), pp. 2950.e5 – 2950.e7.

[251]B EDE, P., A. L. BOKDE,S.M.BYRNE, ETAL. (2013) “Multiparametric MRI study of ALS stratiﬁed for the C9orf72 genotype,” Neurology, 81(4), pp. 361–369.

[252]S IMON´ -SANCHEZ´ , J.,E. G. P. DOPPER, P. E. COHN-HOKKE, ETAL. (2012) “The clinical and pathological phenotype of C9ORF72 hexanucleotide repeat expansions,” Brain, 135(3), pp. 723– 735.

[253]K AIVORINNE,A.-L.,M.K.BODE,L.PAAVOLA, ETAL. (2013) “Clinical Characteristics of C9ORF72 -Linked Frontotemporal Lobar Degeneration,” Dementia and Geriatric Cognitive Dis- orders, 3, pp. 251–262.

[254] VAN DER ZEE,J.,I.GIJSELINCK,L.DILLEN, ETAL. (2013) “A Pan-European Study of the C9orf72 Repeat Associated with FTLD: Geographic Prevalence, Genomic Instability, and Inter- mediate Repeats,” Human Mutation, 34(2), pp. 363–373.

[255]K AMRUNNAHAR,M.,N.S.DIAS, and S. J. SCHIFF (2009) “Optimization of electrode channels in Brain Computer Interfaces.” in Annual International Conference of the IEEE Engineering in Medicine and Biology Society, pp. 6477–80.

[256]K RUSIENSKI,D.J.,E.W.SELLERS,D.J.MCFARLAND, T. M. VAUGHAN, and J. R. WOLPAW (2008) “Toward Enhanced P300 Speller Performance,” J Neurosci Methods, 167(1), pp. 15–21.

[257]K AUFMANN, T., E. M. HOLZ, and A. KUBLER¨ (2013) “Comparison of tactile, auditory, and visual modality for brain-computer interface use: a case study with a patient in the locked-in state,” Frontiers in Neuroscience, 7(129), pp. 1–12.

[258]M CFARLAND, D. J., W. A. SARNACKI, and J. R. WOLPAW (2003) “Brain-computer interface (BCI) operation: optimizing information transfer rates,” Biological Psychology, 63(3), pp. 237 – 251.

[259]J IN,J.,B.ALLISON,E.SELLERS, ETAL. (2011) “Optimized stimulus presentation patterns for an event-related potential EEG-based brain-computer interface,” Medical & Biological Engineering & Computing, 49(2), pp. 181–191.

[260]O GAWA, T., H. TANAKA, and K. HIRATA (2009) “Cognitive deﬁcits in amyotrophic lateral sclerosis evaluated by event-related potentials,” Clinical Neurophysiology, 120(4), pp. 659–664. 163

[261]V IEREGGE, P., B. WAUSCHKUHN,I.HEBERLEIN,J.HAGENAH, and R. VERLEGER (1999) “Selective attention is impaired in amyotrophic lateral sclerosis – a study of event-related EEG potentials,” Cognitive Brain Research, 8(1), pp. 27 – 35.

[262]J IMENEZ´ -ESCRIG,A.,J.FERNANDEZ-LORENTE,A.HERRERO, ETAL. (2002) “Event-related evoked potential P300 in frontotemporal dementia,” Dementia and Geriatric Cognitive Disorders, 13(1), pp. 27–32.

[263]P FEFFERBAUM,A.,J.M.FORD,B.G.WENEGRAT, W. T. ROTH, and B. S. KOPELL (1984) “Clinical application of the P3 component of event-related potentials. II. Dementia, depression, and schizophrenia,” Electroencephalography and Clinical Neurophysiology/Evoked Potentials Section, 59(2), pp. 104–124.

[264]L ULE´,D.,A.C.LUDOLPH, and J. KASSUBEK (2009) “MRI-based functional neuroimaging in ALS: An update,” Amyotrophic Lateral Sclerosis, 10(5-6), pp. 258–268, pMID: 19922112.

[265]K EW,J.J.M.,P.N.LEIGH,E.D.PLAYFORD, ETAL. (1993) “Cortical function in amyotrophic lateral sclerosis: A positron emission tomography study,” Brain, 116(3), pp. 655–680.

[266]M OHAMMADI,B.,K.KOLLEWE,A.SAMII,R.DENGLER, and T. F. MUNTE¨ (2011) “Func- tional neuroimaging at different disease stages reveals distinct phases of neuroplastic changes in amyotrophic lateral sclerosis,” Human Brain Mapping, 32(5), pp. 750–758.

[267]K OLLEWE, K., T. F. MUNTE¨ ,A.SAMII, ETAL. (2011) “Patterns of cortical activity differ in ALS patients with limb and/or bulbar involvement depending on motor tasks,” Journal of Neurology, 258, pp. 804–810.

[268]L ULE´, D., V. DIEKMANN,J.KASSUBEK, ETAL. (2007) “Cortical Plasticity in Amyotrophic Lateral Sclerosis: Motor Imagery and Function,” Neurorehabilitation and Neural Repair, 21(6), pp. 518–526.

[269]S TANTON,B.R.,V.C.WILLIAMS, P. N. LEIGH, ETAL. (2007) “Cortical activation during motor imagery is reduced in Amyotrophic Lateral Sclerosis,” Brain Research, 1172(0), pp. 145 – 151.

[270]A STOLFI, L., F. DE VICO FALLANI, F. CINCOTTI, ETAL. (2007) “Imaging functional brain connectivity patterns from high-resolution EEG and fMRI via graph theory,” Psychophysiology, 44(6), pp. 880–893.

[271]G YSELS, E. and P. CELKA (2004) “Phase synchronization for the recognition of mental tasks in a brain-computer interface,” Neural Systems and Rehabilitation Engineering, IEEE Transactions on, 12(4), pp. 406 –415.

[272]S OE, N. N. and M. NAKAGAWA (2008) “Chaos and fractal analysis of electroencephalogram signals during different imaginary motor movement tasks,” Journal of the Physical Society of Japan, 77(4), p. 044801.

[273]S MITH,S.M.,K.L.MILLER,G.SALIMI-KHORSHIDI, ETAL. (2011) “Network modelling methods for FMRI,” NeuroImage, 54(2), pp. 875 – 891.

[274]W INTERHALDER,M.,B.SCHELTER, W. HESSE, ETAL. (2005) “Comparison of linear signal processing techniques to infer directed interactions in multivariate neural systems,” Signal Pro- cess., 85, pp. 2137–2160.

[275]S TAM, C. (2005) “Nonlinear dynamical analysis of EEG and MEG: Review of an emerging ﬁeld,” Clinical Neurophysiology, 116(10), pp. 2266 – 2301.

[276]D AVID,O.,D.COSMELLI, and K. J. FRISTON (2004) “Evaluation of different measures of functional connectivity using a neural mass model,” NeuroImage, 21(2), pp. 659 – 673. 164

[277]W ANG, Y., B. HONG,X.GAO, and S. GAO (2006) “Phase Synchrony Measurement in Motor Cortex for Classifying Single-trial EEG during Motor Imagery,” in Engineering in Medicine and Biology Society, 2006. EMBS ’06. 28th Annual International Conference of the IEEE, pp. 75 –78.

[278]G YSELS, E., P. RENEVEY, and P. CELKA (2005) “SVM-based recursive feature elimination to compare phase synchronization computed from broadband and narrowband EEG signals in Brain- Computer Interfaces,” Signal Processing, 85(11), pp. 2178–2189.

[279]C RONE,N.E.,D.L.MIGLIORETTI,B.GORDON, and R. P. LESSER (1998) “Functional mapping of human sensorimotor cortex with electrocorticographic spectral analysis. II. Event-related synchronization in the gamma band.” Brain, 121(12), pp. 2301–2315.

[280]L OPESDA SILVA, F. H., A. HOEKS,H.SMITS, and L. H. ZETTERBERG (1974) “Model of brain rhythmic activity,” Kybernetik, 15(1), pp. 27–37.

[281]B HATTACHARYA,B.S.,D.COYLE, and L. P. MAGUIRE (2011) “A thalamo-cortico-thalamic neural mass model to study alpha rhythms in Alzheimer’s disease,” Neural Networks, 24(6), pp. 631–645.

[282]V ALDES, P. A., J. C. JIMENEZ,J.RIERA,R.BISCAY, and T. OZAKI (1999) “Nonlinear EEG analysis based on a neural mass model.” Biological Cybernetics, 81(5-6), pp. 415–424.

[283]T RAUB,R.D.,D.CONTRERAS,M.O.CUNNINGHAM, ETAL. (2005) “Single-column thalam- ocortical network model exhibiting gamma oscillations, sleep spindles, and epileptogenic bursts.” Journal of Neurophysiology, 93(4), pp. 2194–2232.

[284]Z AVAGLIA,M.,L.ASTOLFI, F. BABILONI, and M. URSINO (2008) “A model of rhythm generation and functional connectivity during a simple motor task: preliminary validation with real scalp EEG data,” International Journal of Bioelectromagnetism, 10(1), pp. 68–75.

[285]K IEBEL,S.J.,M.I.GARRIDO,R.J.MORAN, and K. J. FRISTON (2008) “Dynamic causal modelling for EEG and MEG,” Cognitive neurodynamics, 2(2), pp. 121–136.

[286]J ONES,S.R.,D.L.PRITCHETT,M.A.SIKORA, ETAL. (2009) “Quantitative analysis and biophysically realistic neural modeling of the MEG mu rhythm: rhythmogenesis and modulation of sensory-evoked responses.” Journal of Neurophysiology, 102(6), pp. 3554–3572.

[287]W ILSON, H. R. and J. D. COWAN (1973) “A mathematical theory of the functional dynamics of cortical and thalamic nervous tissue.” Kybernetik, 13(2), pp. 55–80.

[288]A MARI,S.-I. (1977) “Dynamics of pattern formation in lateral-inhibition type neural ﬁelds,” Bio- logical Cybernetics, 27(2), pp. 77–87.

[289]D AUNIZEAU,J.,S.J.KIEBEL, and K. J. FRISTON (2009) “Dynamic causal modelling of distributed electromagnetic responses,” NeuroImage, 47(2), pp. 590–601.

[290]F REESTONE, D., P. ARAM,M.DEWAR, and K. SCERRI (2011) “A data-driven framework for neural ﬁeld modeling,” NeuroImage, 56(3), pp. 1043–1058.

[291]A RAM, P., D. FREESTONE,M.DEWAR, ETAL. (2013) “Spatiotemporal multi-resolution approximation of the Amari type neural ﬁeld model,” NeuroImage, 66(0), pp. 88 – 102.

[292]G ALKA, A., T. OZAKI,H.MUHLE,U.STEPHANI, and M. SINIATCHKIN (2008) “A data-driven model of the generation of human EEG based on a spatially distributed stochastic wave equation,” Cognitive Neurodynamics, 2(2), pp. 101–113.

[293]A BRAHAMS,S.,J.NEWTON,E.NIVEN,J.FOLEY, and T. H. BAK (2014) “Screening for cognition and behaviour changes in ALS,” Amyotrophic Lateral Sclerosis and Frontotemporal Degen- eration, 15(1-2), pp. 9–14. 165

[294]K LEIH, S., F. NIJBOER,S.HALDER, and A. KUBLER¨ (2010) “Motivation modulates the P300 amplitude during brain-computer interface use,” Clinical Neurophysiology, 121(7), pp. 1023–1031.

[295]M ULDER, T., J. HOCHSTENBACH,M. VAN HEUVELEN, and A. DEN OTTER (2007) “Motor imagery: The relation between age and imagery capacity,” Human Movement Science, 26(2), pp. 203 – 211.

[296]T ENNISON, M. N. and J. D. MORENO (2012) “Neuroscience, Ethics, and National Security: The State of the Art,” PLoS Biol, 10(3), p. e1001289.

[297]L IN, P. (2010) “Ethical Blowback from Emerging Technologies,” Journal of Military Ethics, 9(4), pp. 313–331.

[298]A LHOFF, F., P. LIN,J.MOOR, and J. WECKERT (2010) “Ethics of Human Enhancement: 25 Questions & Answers,” Studies in Ethics, Law, and Technology, 4(1).

[299]W ORLD MEDICAL ASSOCIATION (2013) “World Medical Association Declaration of Helsinki: Ethical principles for medical research involving human subjects,” JAMA, 310(20), pp. 2191–2194.

[300]U NITED STATES,NATIONAL COMMISSIONFORTHE PROTECTIONOF HUMAN SUBJECTSOF BIOMEDICALAND BEHAVIORAL RESEARCH (1979), “The Belmont Report: Ethical principles and guidelines for the protection of human subjects of research,” .

[301]N IJBOER, F., J. CLAUSEN,B.Z.ALLISON, and P. HASELAGER (2011) “The Asilomar Survey: Stakeholders’ Opinions on Ethical Issues Related to Brain-Computer Interfacing,” Neuroethics, pp. 1–38.

[302]V LEK,R.,D.STEINES,D.SZIBBO, ETAL. (2012) “Ethical issues in brain-computer interface research, development, and dissemination,” J Neurol Phys Ther, 36(2), pp. 94–99.

[303]F ENTON, A. and S. ALPERT (2008) “Extending Our View on Using BCIs for Locked-in Syn- drome,” Neuroethics, 1(2), pp. 119–132.

[304]W OLF, S. M., P. BOYLE,D.CALLAHAN, ETAL. (1991) “Sources of Concern about the Patient Self-Determination Act,” New England Journal of Medicine, 325(23), pp. 1666–1671.

[305]H ASELAGER, P., R. VLEK,J.HILL, and F. NIJBOER (2009) “A note on ethical aspects of BCI,” Neural Networks, 22(9), pp. 1352 – 1357.

[306]A LLISON, B. Z. (2010) “Toward Ubiquitous BCIs,” in Brain-Computer Interfaces (B. Graimann, G. Pfurtscheller, and B. Allison, eds.), The Frontiers Collection, Springer Berlin Heidelberg, pp. 357–387.

[307]C LAUSEN, J. (2009) “Man, machine and in between,” Nature, 457(7233), pp. 1080–1081.

[308]T AMBURRINI, G. (2009) “Brain to Computer Communication: Ethical Perspectives on Interaction Models,” Neuroethics, 2(3), pp. 137–149.

[309]G RUBLER¨ , G. (2011) “Beyond the responsibility gap. Discussion note on responsibility and liability in the use of brain-computer interfaces,” AI & Society, 26(4), pp. 377–382.

[310]F INS, J. J. (2003) “Constructing an ethical stereotaxy for severe brain injury: balancing risks, beneﬁts and access,” Nat Rev Neurosci, 4(4), pp. 323–327.

[311]L UCIVERO, F. and G. TAMBURRINI (2008) “Ethical monitoring of brain-machine interfaces,” AI & Society, 22(3), pp. 449–460. Andrew Geronimo

W301 Millennium Science Complex Phone: (201) 961-2289 CONTACT | The Pennsylvania State University E-mail: [email protected] INFORMATION | University Park, PA 16802 | EDUCATION The Pennsylvania State University, University Park, PA

PhD, Dept. Engineering Science and Mechanics, GPA: 3.94 2009 - Present Thesis: “Personalized brain-computer interfaces for amyotrophic lateral sclero- • sis”. The College of New Jersey, Ewing, NJ BS, Engineering Science, GPA: 3.83 2005 - 2009

RESEARCH The Pennsylvania State University, University Park, PA EXPERIENCE Research Assistant, Center for Neural Engineering 2009 - Present

REFEREED [1] A. Geronimo, H.E. Stephens, S.J. Schiff, Z. Simmons, “Acceptance of brain- JOURNAL computer interfaces in amyotrophic lateral sclerosis”. In: Amyotrophic Lateral PUBLICATIONS Sclerosis and Frontotemporal Degeneration, Early Online, Nov 2014.

[2] E.F. Teel, W.J. Ray, A.M. Geronimo, and S.M. Slobounov, “Residual alterations of brain electrical activity in clinically asymptomatic concussed individuals: an EEG study”, In: Clinical Neurophysiology, vol. 125, iss. 4, 2014.

REFEREE IEEE Transactions on Biomedical Engineering • SERVICE

TEACHING The Pennsylvania State University, University Park, PA EXPERIENCE Teaching Assistant August 2012 to December 2012 Teaching assistant for EMCH 211: Engineering Statics •

PROFESSIONAL Institute for Electrical and Electronics Engineers (IEEE), Member, 2011 – Present MEMBERSHIPS IEEE Engineering in Medicine and Biology Society (EMBS), Member, 2011 – Present Society for Neuroscience, Member 2012 – Present

AWARDS The Pennsylvania State University, University Park, PA Dr. Richard E. Llorens Graduate Award, 2013-2014. • Sabih and Guler Hayek Graduate Scholarship, 2013. • Other Awards IEEE EMBS Excellence in Neural Engineering Travel Award. $700 for travel to the • 5th International Conference on Neural Engineering, April 2011. BCI 2000 Student Scholarship. $1000 for travel to the Fourth International Brain- • Computer Interface Meeting, May 2010.

TECHNICAL Electroencephalographic recording techniques SKILLS Brain-computer interface design Programming Languages: MATLAB/Simulink, LabVIEW, Introductory C++, LaTeX

p. 1 of 1