Brain-Computer Interfacing for Intelligent Systems
Total Page:16
File Type:pdf, Size:1020Kb
Trends & Controversies Brain-Computer Interfacing for Intelligent Systems Anton Nijholt, University of Twente Desney Tan, Microsoft Research dvances in cognitive neurosci- interfaces. Additionally, knowing the José del R. Millán describes real-time, ence and brain-imaging tech- user’s state as well as the tasks they’re robust control of brain-actuated robots Anologies give us the unprec- performing might provide key informa- and neuroprostheses. He focuses on edented ability to interface directly with tion that would help us design context- how to optimally blend a human user’s brain activity. These technologies let us sensitive systems that adapt themselves mental capabilities with a robot’s in- monitor the physical processes in the for optimal user support. This could telligence to operate complex devices brain that correspond with certain forms prove useful to healthy users who might through a low-bit-rate BCI based on of thought. Driven by society’s growing be situationally disabled—that is, they electroencephalography. recognition of the needs of people with might lack full access to traditional, Brendan Allison and Bernhard Graimann physical disabilities, researchers have physically based communication modali- present specific situations in which BCI begun using these technologies to build ties. It also opens a whole new domain research aimed at the physically disabled brain-computer interfaces (BCIs)—com- of niche applications, carefully designed can apply to healthy users. munication systems that don’t depend to exploit this novel modality’s specific Finally, Florin Popescu, Benjamin on the brain’s normal output pathways affordances, perhaps in conjunction Blankertz, and Klaus-R. Müller ground of peripheral nerves and muscles. In with more traditional input devices. We the opportunities in the hardware, BCIs, users explicitly manipulate their believe that games might be an area of computational, and social challenges brain activity instead of motor move- early adoption—first, because games we face as we work to create BCIs ments to produce signals that control have traditionally pushed us to consider that work effectively in real-world computers or communication devices. completely new usage paradigms, and environments. This research has extremely high impact, second, because gamers tend to be especially for disabled individuals who fairly tolerant of new technologies. Edu- can’t otherwise physically communicate. cation could be another such domain. Although removing the need for mo- The four short articles in this issue’s Anton Nijholt is full professor of tor movements in computer interfaces is Trends & Controversies provide a quick computer science at the University of challenging and rewarding, we believe overview of the past, present, and fu- Twente and chair of its Human Media the full potential of brain imaging as ture of BCIs. They are written primarily Interaction subdepartment. Contact an input mechanism lies in the rich in- by European researchers working with him at [email protected]. formation it provides about the user’s noninvasive techniques, which repre- state. Having access to this state is im- sent a focused subset of the broader re- Desney Tan is a researcher at Microsoft portant to researchers because it might search and viewpoints in the field. Research, where he manages the Com- let us derive more direct measures of Gert Pfurtscheller and Clemens Brun- putational User Experiences group. He traditionally elusive phenomena such ner begin with a state-of-the-art survey. also holds an affiliate faculty appoint- as task engagement, cognitive work- They discuss brain signals that can be ment in the Computer Science and En- load, surprise, satisfaction, or frustra- measured with various devices, ways to gineering Department at the University tion. These measures could open new control these signals, and how to train of Washington. Contact him at desney@ avenues for evaluating systems and users to do this. microsoft.com. The State-of-the-Art in BCIs or attentional deficit hyperactivity disorders (ADHD); or Gert Pfurtscheller and Clemens Brunner, controlling computer games.1,2 Graz University of Technology Every mental activity—for example, decision making, intending to move, and mental arithmetic—is accom- A brain-computer interface (BCI) is a novel communica- panied by excitation and inhibition of distributed neural tion system that translates human thoughts or intentions structures or networks. With adequate sensors, we can re- into a control signal. In this way, a BCI provides a new, cord changes in electrical potentials, magnetic fields, and nonmuscular communication channel that system develop- (with a delay of some seconds) metabolic supply when the ers can use in a variety of applications, such as assisting activated neuron population exceeds some critical mass. people with severe motor disabilities; supporting biofeed- Consequently, we can base a BCI on electrical potentials, back training in people suffering from epilepsy, stroke, magnetic fields, or metabolic/hemodynamic recordings. 72 1541-1672/08/$25.00 © 2008 IEEE IEEE INTELLIGENT SYSTEMS Published by the IEEE Computer Society Figure 1 presents a schematic of the principal BCI components. The compo- Feature Preprocessing Classification nents involve signal acquisition, prepro- extraction cessing, feature extraction, classification, and an application interface together with the application. When we talk about a BCI, Signal Application acquisition interface we must consider several component op- tions. Signal recordings can be either inva- Brain signal Control signal sive or noninvasive. Signal features require Brain-computer interface (BCI) analysis and classification methods. Con- Thought trol functions require selecting a suitable mental strategy as well as operational and feedback mechanisms. Closed-loop system Application Suitable brain signals Invasive BCI methods place electrodes di- rectly on or inside the cortex. One method records electrical potentials for subsequent analysis of the electrocorticogram (ECoG). Feedback (visual, auditory, haptic) Another method places a multiunit elec- trode array in the cortex to record the neu- Figure 1. The brain-computer interface: (a) schematic of principal BCI components ral firing of a small population of neurons. and (b) three applications: playing table tennis (top), using a spelling system Both signal types have a superior signal- (middle), and restoring grasp functions (bottom). to-noise ratio, need little user training, and are suitable for replacing or restoring lost resolution, which can be as fine as 2 to 3 corded (and possibly preprocessed in motor functions in patients with damaged millimeters. Researchers have studied BCIs suitable ways), the next step is to extract parts of the neuronal system. using MEG data, but they haven’t been able prominent features that describe impor- Noninvasive BCIs, on the other hand, to demonstrate significant advantages over tant discriminative signal properties. This can use a variety of brain signals as input, EEG-based systems. processing stage aims simply to reduce such as electroencephalograms (EEG), Unlike EEG and MEG systems, which data and adequately transform it such that magnetoencephalograms (MEG), blood- detect the electromagnetical activity of cor- the subsequent classification process is oxygen-level-dependent (BOLD) signals, tical neurons, near-infrared spectroscopy optimal. Example features used in EEG and (de)oxyhemoglobin concentrations. (NIRS) measures the metabolic activity of processing are the power in a specific fre- The EEG, which is basically the sum of specific cortical regions. NIRS uses light quency band (band power), autoregressive many postsynaptic potentials in the cortex, in the near-IR spectrum (typically between parameters, and synchronization measures. is the most widely used brain signal for op- wavelengths of 630 to 1,350 nm) to deter- erating a BCI system. We can extract two mine the oxygenation of the tissue, and Choosing types of changes from the ongoing EEG researchers have recently applied it to BCI the mental strategy signals: one is time- and phase-locked research. The potential advantages of real- Operant conditioning is a learning pro- (evoked) to an externally or internally izing a BCI with this technique include its cess with the goal of self-regulating brain paced event, while the other is also time- insensitivity to typical EEG artifacts such potentials (such as slow cortical potential locked but not phase-locked (induced). To as the electrooculogram (EOG), electro- shifts) or brain waves (such as sensorimo- the former class belong the event-related myogram (EMG), and electrode failures. tor rhythms) with the help of suitable feed- potentials (ERPs), including the P300, However, the technique also requires sev- back. This process doesn’t require continu- steady-state visual evoked potentials eral seconds to pass before it can measure ous feedback, but it does require a reward (SSVEPs), and slow cortical negative shifts; the metabolic response, which is a long time for achieving the desired brain potential to the latter class belong the event-related compared to EEG and MEG. The spatial (wave). Researchers have used operant desynchronizations (ERDs) and event-re- resolution also lies in the centimeter range.3 conditioning to realize a communication lated synchronizations (ERSs). Like NIRS, functional magnetic reso- system for completely paralyzed (“locked- The MEG can measure brain activity