Brain-Computer Interfaces for Communication and Control

contributed articles

doi:10.1145/1941487.1941506 It has been known since the pio- The brain’s electrical signals enable people neering work of Hans Berger more than 80 years ago that the brain’s without muscle control to physically interact electrical activity can be recorded with the world. noninvasively through electrodes on the surface of the scalp.23 Berger ob- By Dennis J. McFarland and Jonathan R. Wolpaw served that a rhythm of about 10Hz was prominent on the posterior scalp and reactive to light. He called it the alpha rhythm. This and other obser- vations showed the electroencepha- Brain-Computer logram (EEG) could serve as an index of the gross state of the brain. Despite Berger’s careful work many scientists were initially skeptical, with some Interfaces for suggesting that the EEG might repre- sent some sort of artifact. However, subsequent work demonstrated con- clusively that the EEG is indeed pro- Communication duced by brain activity.23 Electrodes on the surface of the scalp are at some distance from brain and Control tissue, separated from it by the cover- ings of the brain, skull, subcutaneous tissue, and scalp. As a result, the signal is considerably degraded, and only the synchronized activity of large numbers of neural elements can be detected, limiting the resolution with which brain activity can be monitored. More- over, scalp electrodes pick up activ- Brain activity produces electrical signals detectable ity from sources other than the brain, on the scalp, on the cortical surface, or within the including environmental noise (such as 50Hz or 60Hz activity from power brain. Brain-computer interfaces (BCIs) translate lines) and biological noise (such as ac- these signals into outputs that allow users to tivity from the heart, skeletal muscles, communicate without participation of peripheral and eyes). Nevertheless, since the time 36 of Berger, many studies have used the nerves and muscles (see Figure 1). Because they do EEG to gain insight into brain function, not depend on neuromuscular control, BCIs provide with many of them using averaging to separate EEG from superimposed elec- options for communication and control for people trical noise. with devastating neuromuscular disorders (such as amyotrophic lateral sclerosis, or ALS, brainstem stroke, key insights cerebral palsy, and spinal cord injury). The central Brain-computer interfaces provide a new communication-and-control option purpose of BCI research and development is to enable for individuals for whom conventional these users to convey their wishes to caregivers, use methods are ineffective. Current BCI technology is slow, word-processing programs and other software, and even benefiting only those with the most control a robotic arm or neuroprosthesis. Speculation severe disabilities. has suggested that BCIs could be useful even to people Research may greatly expand the number of people who would benefit with lesser, or no, motor impairment. from the technology.

60 communications of the acm | may 2011 | vol. 54 | no. 5 BCIs are a direct communication pathway between the brain and external devices. EEG measurements at the Danish Master’s program in Medicine & Technology; http://www.medicin-ing.dk/kandidat/en.

EEG research reflects two major 1970s based on visual evoked-poten- negative and positive peaks, and the paradigms: evoked potentials and tials.34 His users viewed a diamond- numbers indicating the approximate oscillatory features. Evoked poten- shape red checkerboard illuminated latency in msec. tials are transient waveforms, or brief with a xenon flash. By attending to dif- Vidal’s achievement was an in- perturbations in the ongoing activ- ferent corners of the flashing checker- teresting demonstration of proof of ity, that are phase-locked to an event board, they could generate right, up, principle. In the early 1970s, it was far (such as a visual stimulus). They are left, and down commands, enabling from practical, given that it depended typically analyzed by averaging many them to move through a maze present- on a time-shared system with limited similar events in the time-domain. ed on a graphics terminal. An IBM360 processing capacity. Vidal34 also in- Although oscillatory features in an mainframe digitized the data, and cluded in his system online removal EEG may occur in response to specific an XDS Sigma 7 computer controlled of ocular artifacts to prevent them events, they are usually not phase- the experimental events. Users first from being used for control. A decade locked and typically studied through provided data to train a stepwise lin- earlier, Edmond Dewan6 of the Air spectral analysis. Historically, most ear discriminant function, then navi- Force Cambridge Research Lab, Bed- EEG studies have examined phase- gated the maze online in real time. ford MA, instructed users to explicitly locked evoked potentials. Both these Thus, Vidal34 used signal-processing use eye movements to modulate their l h major paradigms have been applied techniques to realize real-time analy- brain waves, showing that subjects

Ba 36 s in BCIs. sis of the EEG with minimal averag- could learn to transmit Morse code ar

L 34 y The term “brain-computer inter- ing. The waveforms showed by Vidal messages using EEG activity associ- b

ph face” can be traced to Jacques Vidal of suggested his BCI used EEG activity in ated with eye movement. ra g o the University of California, Los An- the timeframe of the N100-P200 com- The fact that both Vidal’s and De- t o h

P geles who devised a BCI system in the ponents, with the N and P indicating wan’s BCIs depended on eye move-

may 2011 | vol. 54 | no. 5 | communications of the acm 61 contributed articles

ment made them somewhat less in- use of a P300-based spelling device bet and several other symbols, focus- teresting from a scientific or clinical (see Figure 2b) in which a positive po- ing attention on the desired selection, point of view, since they required ac- tential around 300msec after an event as the rows and columns of the ma- tual muscle control or eye movement, significant to the subject is consid- trix were repeatedly flashed to elicit simply using EEG to reflect the result- ered a “cognitive” potential since it is visual evoked potentials. Farwell and ing gaze direction. generated in tasks where the subject Donchin7 found their users were able discriminates among stimuli. Far- to spell the word “brain” through the Varieties of BCI Signals well’s and Donchin’s7 users viewed a P300 spelling device; in addition, they Farwell and Donchin7 reported the first 6×6 matrix of the letters of the alpha- did an offline comparison of detection algorithms, finding the stepwise linear Figure 1. Basic design and operation of a BCI system. discriminant analysis was generally best. The fact that the P300 potential

Signal Acquisition Translation reflects attention rather than simply and Processing Signal Features Algorithm Device Commands gaze-direction implied this BCI did not depend on muscle, or eye-movement, control, thus representing a significant advance. Several groups have since fur- ther developed this BCI method.13 Wolpaw et al.38 reported the first use of sensorimotor rhythms (SMRs) for cursor control (see Figure 2a), or EEG rhythms that change with movement or imagination of movement and are spontaneous in the sense they do not require specific stimuli to occur. Peo- ple learned to vary their SMRs to move a cursor to hit one of two targets on the top or bottom edge of a video screen. Signals from the brain are acquired by electrodes on the scalp or head and processed to extract specific Cursor movement was controlled by signal features reflecting the user’s intent. These SMR amplitude (measured by spectral features are translated into commands to operate analysis). A distinctive feature of this a device. Users must develop and maintain good task is that it required users to rapidly correlation between their intent and the BCI’s signal features. The BCI must select and extract features switch between two states to select a the user can control, translating them into device particular target. The rapid bidirec- commands (adapted from Wolpaw et al.36). tional nature of the Wolpaw et al.38 paradigm made it distinct from prior studies that produced long-term uni-

Figure 2. Current human BCI systems.

A and B are noninvasive, and C is invasive. A. In a sensorimotor rhythm BCI, scalp EEG is recorded over sensorimotor cortex; users control the amplitude of rhythms to move a cursor to a target on the screen. B. In a P300 BCI, a matrix of choices is presented on screen, and scalp EEG is recorded as these choices flash in succession. C. In a cortical neuronal BCI, electrodes implanted in the cortex detect action potentials of single neurons; users learn to control the neuronal firing rate to move a cursor on screen (adapted from Wolpaw et al.36).

(a) Sensorimotor Rhythms (b) P300 Evoked Potential (c) Cortical Neuronal Activity Power Pz Induction Transmitter 4 –50 Cement other choices Skull Bone Gold wire 3 Top Target 0 desired choice Cortex 2 50 Glass Neurites cone 100 μV Row of Neurons 1 100 0.5 s A mplitude ( μ V) (a/d u) Voltage

Bottom Target 100 μV 0 150 0 5 10 15 20 25 30 –100 0 100 200 300 400 500 20 s

Frequency (Hz) Time (ms) On On Off On Off Off Top Target 10 μV 1 sec Bottom Target

62 communications of the acm | may 2011 | vol. 54 | no. 5 contributed articles directional changes in brain rhythms; encephalography (MEG),20 functional by the algorithm the BCI is using. It for example, users were required to magnetic resonance imaging (fMRI),28 is thus not possible to predict results maintain an increase in the size of an and near-infrared systems (fNIR).4 precisely from offline analyses that EEG rhythm for minutes at a time. In Current technology for recording cannot account for these effects. a series of subsequent studies, this MEG and fMRI is both expensive and Blankertz et al.3 identified several group showed that the signals control- bulky, making it unlikely for practical trends in the results of a BCI data- ling the cursor were actual EEG activity applications in the near term; fNIR is classification competition. Most win- and that covert muscle activity did not potentially cheaper and more com- ning entries used linear classifiers, mediate this EEG control.18,31 pact. However, both fMRI and fNIR are the most popular being Fisher’s dis- These initial SMR results were sub- based on changes in cerebral blood criminant and linear support vector sequently replicated by others21,24 and flow, an inherently slow response.11 machines (SVMs). The winning entries extended to multidimensional con- Electrophysiological features repre- for data sets with multichannel oscil- trol.37 These P300 and SMR BCI stud- sent the most practical signals for BCI latory features often used common ies together showed that noninvasive applications today. spatial patterns. In their review of the EEG recording of brain signals can literature on BCI classifiers, Lotte et serve as the basis for communication- System Design al.16 concluded that SVMs are particu- and-control devices. Communication-and-control applica- larly efficient, attributing the efficien- A number of laboratories have ex- tions are interactive processes requir- cy to their regularization property and plored the possibility of developing ing users observe the results of their immunity to the curse of dimensional- BCIs using single-neuron activity de- effort to maintain good performance ity. They also concluded that combina- tected by microelectrodes implanted and correct mistakes. For this reason, tions of classifiers seem efficient, not- in the cortex12,30 (see Figure 2c). Much BCIs must run in real time and provide ing a lack of comparison of classifiers of the related research has been done real-time feedback to users. While within the same study using otherwise in non-human primates, though trials many early BCI studies satisfied this identical parameters. have also been done with humans.12 requirement,24,38 later studies were Muller and Blankertz21 advocated a Other studies have shown that record- often based on offline analyses of pre- machine-learning approach to BCIs in ings of electrocorticographic (ECoG) recorded data1; for example, the Lotte which a statistical analysis of a calibra- activity from the surface of the brain et al.16 review of studies evaluating BCI tion measurement is used to train the can also provide signals for a BCI15; signal-classification algorithms found system. The goal is to develop a “zero- to date they indicate that invasive re- most used offline analyses. Indeed, training” method providing effective cording methods can also serve as the the current popularity of BCI research performance from the first session, basis for BCIs. Meanwhile, important is probably due in part to the ease of- contrasting it with one based on train- issues concerning their suitability for fline analyses are performed on pub- ing users to control specific features long-term human use have yet to be licly available data sets. While such of brain signals.38 A system that can resolved. offline studies may help guide actual be used without extensive training is Earlier studies demonstrating oper- online BCI investigations, there is no appealing since it requires less initial ant conditioning of single units in the guarantee that offline results will gen- effort on the part of both the BCI user motor cortex of primates,9 hippocam- eralize to online performance. Users’ and the system operator. Operation pal theta rhythm of dogs,2 and senso- brain signals are often affected by BCI of such a system is based on the as- rimotor rhythm in humans29 showed outputs that are in turn determined yet uncertain premise that users can brain activity could be trained with operant techniques. However, these Figure 3. Three approaches to BCI design. studies were not demonstrations of BCI systems for communication and Let the Machines Run Operant Conditioning Optimized Co-Adaptation control since they required subjects to increase brain activity for periods of many minutes, showing that brain User User User activity could be tonically altered in a single direction through training. However, communication-and-control devices require that users be able to select from at least two distinct alternatives; that is, there must be at least BCI System BCI System BCI System one bit of information per selection. Effective communication-and-control devices require users to rapidly switch between multiple alternatives. Arrows indicate the element that adapts; the BCI, the user, or both adapt to optimize and maintain BCI performance (adapted from McFarland et al.17). In addition to electrophysiological measures, researchers have also demonstrated the feasibility of magneto-

may 2011 | vol. 54 | no. 5 | communications of the acm 63 contributed articles

repeatedly and reliably maintain the suggested that cognitive tasks (such ger necessary to operate a sensorimo- specified correlations between brain as navigation and auditory imagery) tor rhythm-based BCI. As is typical signals and intent. Figure 3 outlines might be more useful in driving a BCI of many simple motor tasks, perfor- three different conceptualizations of than motor imagery. However, senso- mance becomes automatized through where adaptation might take place to rimotor rhythm-based BCIs may pro- extended practice. Automatized per- establish and maintain good BCI per- vide several advantages over systems formance may be less likely to inter- formance: In the first, the BCI adapts that depend on complex cognitive op- fere with mental operations users to the user; in the second, the user erations; for example, the structures might wish to engage in concurrent adapts to the BCI; and, in the third, involved in auditory imagery are also with their BCI use; for example, com- user and system adapt to each other. likely to be driven by auditory sen- posing a manuscript is much easier A number of BCI systems are de- sory input. Wolpaw and McFarland37 if the writer does not need to think signed to detect user performance of reported that with extended practice extensively about each individual key- specific cognitive tasks. Curran et al.3 users report motor imagery is no lon- stroke. As noted, EEG recording may be Figure 4. BCI2000 design consists of four modules: operator, source, signal processing, contaminated by non-brain activity and application. (such as line noise and muscle activity); see Fatourechi et al.8 for a review.

Operator Activity recorded from the scalp rep- resents the superposition of many

System Configuration Visualization signals, some originating in the brain, some elsewhere. These signals include potentials generated by retinal dipoles, or eye movement and blinks, and facial muscles. It is noteworthy that mental Source effort is often associated with changes 35 Brain Signals Signal Control Signals User in eye-blink rate and muscle activity. Processing Application BCI users might generate these arti- Event Markers Event Markers Storage facts without being aware of what they are doing simply by making facial ex- pressions associated with effort. Event Markers Facial muscles can generate sig- Operator deals with system configuration and online presentation of results to the nals with energy in the spectral bands investigator; during operation, information is communicated from source to signal processing used as features in an SMR-based to user application and back to source (adapted from Schalk et al.25). BCI18 Muscle activity can also modulate SMR activity; for example, users can move their right hands in order to desynchronize the mu rhythm over the left hemisphere. This sort of me- diation of the EEG through peripheral muscle movements was a concern in the early days of BCI development. As noted earlier, Dewan6 trained users to send Morse code messages using occipital alpha rhythms modu- lated by voluntary movements of eye muscles. For this reason, Vaughan et al.33 recorded EMG from 10 distal limb muscles, while BCI users used central mu or beta rhythms to move a cursor to targets on a video screen. EMG activity was very low in these well-trained users. Most important, the correlations between target position and EEG activity could not be accounted for through EMG activity. Similar studies have been done with BCI users moving a cursor in two di- mensions,37 showing that SMR modu- Figure 5. Hardware in the Wadsworth Center’s home BCI system, including 16-channel electrode cap for signal recording, solid-state amplifier, laptop, and additional monitor lation does not require actual move- as user display. ments or muscle activity.

64 communications of the acm | may 2011 | vol. 54 | no. 5 contributed articles

Applications options of practical value mainly for Several recent BCI spelling systems people severely limited in their motor are based on different EEG signals, skills and thus have few other options. including the mu rhythm22,26 and the Widespread use of BCI technology by P300.31 The Mu rhythm systems made individuals with little or no disability use of machine-learning paradigms Sensorimotor is unlikely in the short-term and would that minimized training, with users rhythm-based require much greater speed and accu- of both mu-based systems reportedly racy than has so far been demonstrat- averaging between 2.3–7 characters/ BCIs may provide ed in the scientific literature. minute22 and 2.85–3.38 characters/ several advantages Noninvasive and invasive methods minute.26 The P300 system averaged would both benefit from improved 3.66 selections/minute.31 Townsend et over systems recording methods. Current invasive al.24 noted the reported rate depends that depend on methods do not deal adequately with on how the figure is computed, but the need for long-term performance study authors do not always provide complex cognitive stability. The brain’s complex reac- details. Omitting time between trials tion to an implant is still imperfectly increased Townsend et al.31 results operations. understood and might impair long- from 3.66 to 5.92 characters/second. term performance. Noninvasive EEG In any case, these systems perform electrodes require some level of skill within a similar general range. At cur- in the person placing them, as well as rent BCI character rates, only users in periodic maintenance to ensure suf- with limited options could benefi. ficiently good contact with the skin; BCI systems have also been de- more convenient and stable electrodes veloped for control applications; for are under development. Improved example, several groups have shown methods for extracting key EEG fea- that human subjects can use their EEG tures and translating them into device activity to drive a simulated wheel- control, as well as user training, would chair.10,14 Bell et al.1 showed the P300 also help improve BCI performance. could be used to select among complex Recent developments in computer commands to a partially autonomous hardware provide compact portable humanoid robot; for a review of the systems that are extremely powerful. use of BCI for robotic and prosthetic Use of digital electronics has also led devices see McFarland and Wolpaw.19 to improved size and performance of Several commercial concerns re- EEG amplifiers. Thus it is no longer cently produced inexpensive devices necessary to use a large time-shared purported to measure EEG. Both Emo- mainframe, as it was with Vidal34; stan- tiv and Neurosky developed products dard laptops easily handle the vast ma- with a limited number of electrodes jority of real-time BCI protocols. Sig- that do not use conventional gel-based nal-processing and machine-learning recording technology27 and are intend- algorithms have also been improved. ed to provide input for video games. Coupled with discovery of new EEG Not clear is the extent to which they use features for BCI use and development actual EEG activity, as opposed to scalp- of new paradigms for user training, muscle activity or other non-brain such improvements are gradually in- signals. Given the well-established creasing the speed and reliability of prominence of EMG activity in activity BCI communication and control, de- recorded from the head, it seems likely velopments facilitated by the BCI2000 that such signals account for much of software platform.25 the control these devices provide.27 BCI2000 is a general-purpose research-and-development system in- Conclusion corporating any brain signal, signal- In a review of the use of BCI technology processing method, output device, for robotic and prosthetic devices, Mc- and operating protocol. BCI2000 Farland and Wolpaw19 concluded that consists of a general standard for cre- the major problem facing BCI applica- ating interchangeable modules de- tions is how to provide fast, accurate, signed according to object-oriented reliable control signals, as well as other principles (see Figure 4), including a uses of BCIs. Current BCI systems that source module for signal acquisition, operate using actual brain activity can signal-processing module, and user- provide communication-and-control application module. Configuration

may 2011 | vol. 54 | no. 5 | communications of the acm 65 contributed articles

and coordination of these modules is (Jan. 1970), 15–24. Applications, and Related Fields, Fifth Edition, E. 3. Blankertz, B., Muller, K-R, Krusienski, D.J., Schalk, G., Neidermeyer and F. Lopes da Silva, Eds. Lippincott accomplished through a fourth opera- Wolpaw, J.R., Schlogl, A., Pfurtscheller, G., Millan, J., Williams and Wilkins, Philadelphia, 2005, 1–15. tor module; several source modules, Schroder, M., and Birbaumer, N. The BCI competition 24. Pfurtscheller, G., Flotzinger, D., and Kalcher, J. Brain- III: Validating alternative approaches to actual BCI computer interface: A new communication device signal-processing modules, and user problems. IEEE Transactions on Neural Systems for handicapped persons. Journal of Microcomputer applications have been created for the and Rehabilitation Engineering 14, 2 (June 2006), Applications 16, 3 (July 1993), 293–299. 153–159. 25. Schalk, G., McFarland, D.J., Hinterberger, T., BCI2000 standard (see http://www. 4. Coyle, S.M., Ward, T.E., and Markham, C.M. Brain- Birbaumer, N., and Wolpaw, J.R. BCI2000: A general- bci2000.org/BCI2000/Home.html). computer interface using a simplified functional purpose brain-computer interface (BCI) system. near-infrared spectroscopy system. Journal of Neural IEEE Transactions on Biomedical Engineering 51 The Wadsworth Center recently be- Engineering 4, 3 (Sept. 2007), 219–226. (2004), 1034–1043. gan developing a system for home use 5. Curran, E., Sykacek, P., Stokes, M., Roberts, S.J., 26. Scherer, R., Muller, G.R., Neuper, C., Graimann, B., Penny, W., Johnsrude, I., and Owen, A. Cognitive Pfurtschheller, G. An asynchronously controlled EEG- by individuals with severe motor im- tasks for driving a brain-computer interface: A pilot based virtual keyboard: Improvement of the spelling 32 study. IEEE Transactions on Neural Systems and rate. IEEE Transactions on Biomedical Engineering 51, pairments. Its basic hardware (see Rehabilitation Engineering 12, 1 (Mar. 2003), 48–54. 6 (June 2004), 979–984. Figure 5) consists of a laptop comput- 6. Dewan, E.M. Occipital alpha rhythm eye position 27. Singer, E. Brain games. Technology Review 111, 4 and lens accommodation. Nature 214, 5092 (June 3, (July/Aug. 2008), 82–84. er with 16-channel EEG acquisition, a 1967), 975–977. 28. Sitaram, R., Caria, A., Veit, R., Gaber, T., Rota, G., second screen placed in front of the 7. Farwell, L.A. and Donchin, E. Talking off the top of Kuebler, A., and Birbaumer, N. fMRI brain-computer your head: Toward a mental prosthesis utilizing event- interface: A tool for neuroscientific research user, and an electrode cap; software is related brain potentials. Electroencephalography and and treatment. Computational Intelligence and provided by the BCI2000 general-pur- Clinical Neurophysiology 70, 6 (Dec. 1988), 510–523. Neuroscience (2007). 25 8. Fatourechi, M., Bashashati, A., Ward, R.K., and Birch, 29. Sterman, M.B., MacDonald, L.R., and Stone, R.K. pose system. The initial users had G.E. EMG and EOG artifacts in brain-computer Biofeedback training of sensorimotor EEG in man and late-stage ALS, with little or no volun- interface systems: A survey. Clinical Neurophysiology its effect on epilepsy. Epilepsia 15, 3 (Sept. 1974), 118, 3 (Mar. 2007), 480–494. 395–416. tary movement, and found conven- 9. Fetz, E.E. Operant conditioning of cortical unit activity. 30. Taylor D.A., Tillery S., and Schwartz, A.B. Direct tional assistive communication de- Science 163, 870 (Feb. 28, 1969), 955–958. cortical control of 3D neuroprosthetic devices. 10. Galan, F., Nuttin, M., Lew, E., Ferrez, P.W., Vanacker, Science 296, 5574 (June 7, 2002), 1829–1832. vices inadequate for their needs. The G., Philips, J., and Millan, J.d.R. A brain-actuated 31. Townsend, G., LaPallo, B.K., Boulay, C.B., Krusienski, P300-based matrix speller is used for wheelchair: Asynchronous and noninvasive brain- D.J., Frye, G.E., Hauser, C.K., Schwartz, N.E., Vaughan, computer interfaces for continuous control of T.M., Wolpaw, J.R., and Sellers, E.W. A novel P300- these applications due to its relatively robots. Clinical Neurophysiology 119, 9 (Sept. 2008), based brain-computer interface stimulus presentation high throughput for spelling and sim- 2159–2169. paradigm: Moving beyond rows and columns. Clinical 11. He, B. and Liu, Z. Multimodal functional neuroimaging: Neurophysiology 121, 7 (July 2010), 1109–1120. plicity of use. A 49-year-old scientist Integrating functional MRI and EEG/MEG. IEEE 32. Vaughan, T.M., McFarland, D.J., Schalk, G., Sarnacki, Reviews in Biomedical Engineering 1 (Nov.2008), W.A., Krusienski, D.J., Sellers, E.W., and Wolpaw, with ALS has used this BCI system on 23–40. J.R. The Wadsworth BCI research and development a daily basis for approximately three 12. Hochberg, L.R., Serruya, M.D., Friehs, G.M., Mukand, program: At home with BCI. IEEE Transactions J.A., Saleh, M., Caplan, A.H., Branner, A., Penn, D.R.D., on Rehabilitation Engineering 14, 2 (June 2006), years, finding it superior to his eye- and Donoghue, J.P. Neuronal ensemble control of 229–233. gaze system (a letter-selection device prosthetic devices by a human with tetraplegia. 33. Vaughan, T.M., Miner, L.A., McFarland, D.J., and Nature 442, 7099 (July 13, 2006), 164–171. Wolpaw, J.R. EEG-based communication: Analysis of based on eye-gaze direction) and us- 13. Krusienski, D.J., Sellers, E.W., McFarland, D.J., concurrent EMG activity. Electroencephalography and ing it from four to six hours per day Vaughan, T.M., and Wolpaw, J.R. Toward enhanced Clinical Neurophysiology 107, 6 (Dec. 1998), 428–433. P300 speller performance. Journal of Neuroscience 34. Vidal, J.J. Real-time detection of brain events in EEG. for email and other communication Methods 167, 1 (Jan. 15, 2008), 15–21. Proceedings of the IEEE 65, 5 (May 1977), 633–641. purposes.32 14. leeb, R., Friedman, D., Muller-Putz, G.R., Scherer, 35. Whitham, E.M., Lewis, T., Pope, K.J., Fitzbibbon, S.P., R., Slater, M., and Pfurtscheller, G. Self-paced Clark, C.R., Loveless, S., DeLosAngeles, D., Wallace, How far BCI technology will go and (asynchronous) BCI control of a wheelchair in virtual A.K., Broberg, M., and Willoughby, J.O. Thinking how useful it will be depend on future environments: A case study with a tetraplegic. activates EMG in scalp electrical recordings. Clinical Computational Intelligence and Neuroscience 79642 Neurophysiology 119, 5 (May 2008), 1166–1175. research developments. However, it is (2007). 36. Wolpaw, J.R., Birbaumer, N., McFarland, D.J., apparent that BCIs can serve the ba- 15. leuthardt, E.C., Schalk, G., Wolpaw, J.R., Ojemann, Pfurtscheller, G., and Vaughan, T.M. Brain-computer J.G., and Moran, D.W. A brain-computer interface interfaces for communication and control. Clinical sic communication needs of people using electrocorticographic signals in humans. Neurophysiology 113, 6 (June 2002), 767–791. whose severe motor disabilities pre- Journal of Neural Engineering 1, 2 (June 2004), 37. Wolpaw, J.R. and McFarland, D.J. Control of a 63–71. two-dimensional movement signal by a noninvasive vent them from using conventional 16. lotte, F., Congedo, M., Lecuyer, A., Lamarche, F., and brain-computer interface. Proceedings of the Arnaldi, B. A review of classification algorithms for National Academy of Science 101, 51 (Dec. 21, 2004), augmentive communications devices, EEG-based brain-computer interfaces. Journal of 17849–17854. all of which require muscle control. Neural Engineering 4, 2 (June 2007), 1–13. 38. Wolpaw, J.R., McFarland, D.J., Neat, G.W., and 17. McFarland, D.J., Krusienski, D.J., and Wolpaw, J.R. Forneris, C.A. An EEG-based brain-computer interface Brain-computer interface signal processing at the for cursor control. Electroencephalography and Acknowledgments Wadsworth Center: Mu and sensorimotor beta Clinical Neurophysiology 78, 3 (Mar. 1991), 252–259. rhythms. Progress in Brain Research 159 (2006), This work was supported in part by 411–419. grants from the National Institutes of 18. McFarland, D.J., Sarnacki, W.A., Vaughan, T.M., Health HD30146 (NCMRR, NICHD) and Wolpaw, J.R. Brain-computer interface (BCI) Dennis J. McFarland ([email protected]) is a operation: Signal and noise during early training research scientist in the Laboratory of Neural Injury and and EB00856 (NIBIB & NINDS) and the sessions. Clinical Neurophysiology 116 (2005), 56–62. Repair at the Wadsworth Center of the New York State James S. McDonnell Foundation. We 19. McFarland, D.J. and Wolpaw, J.R. Brain-computer Department of Health, Albany, NY. interface operation of robotic and prosthetic devices. thank Chad Boulay and Peter Brun- Computer 41, 10 (Oct. 2008), 48–52. ner for their comments on the manu- 20. Mellinger, J., Schalk, G., Braun, C., Preissl, H., Jonathan R. Wolpaw ([email protected]) is a Rosenstiel, W., Birbaumer, N., and Kubler, A. An MEG- research physician in the Laboratory of Neural Injury and script. based brain-computer interface (BCI). Neuroimage Repair in the Wadsworth Center of the New York State 36, 3 (July 1, 2007), 581–593. Department of Health, Albany, NY. 21. Muller, K.-R., and Blankertz, B. Towards noninvasive brain-computer interfaces. IEEE Signal Processing References Magazine 23, 1 (Sept. 2006), 125–128. 1. Bell, C.J., Shenoy, P., Chalodhorn, R., and Rao, R.P.N. 22. Muller, K.-R., Tangermann, M., Dornhege, G., Control of a humanoid robot by a noninvasive brain- Krauledat, M., Curio, GT., and Blankertz, B. Machine computer interface in humans. Journal of Neural learning for real-time single-trial EEG-analysis: Engineering 5, 2 (June 2008), 214–220. From brain-computer interfacing to mental-state 2. Black, A.H., Young, G.A., and Batenchuk, C. Avoidance monitoring. Journal of Neuroscience Methods 167, 1 training of hippocampal theta waves in flaxedilized (Jan. 15, 2008), 82–90. dogs and its relation to skeletal movement. Journal 23. neidermeyer,. E. Historical aspects. In of Comparative and Physiological Psychology 70, 1 Electroencephalography: Basic Principals, Clinical © 2011 ACM 0001-0782/11/05 $10.00

66 communications of the acm | may 2011 | vol. 54 | no. 5