Common Spatial Pattern for the Classification of Imagined

Common Spatial Pattern for the Classiﬁcation of Imagined Geometric Objects

Fabio R. Llorella Costa1, Gustavo Patow1 and Jose´ M. Azor´ın2 1VIRViG, Universitat de Girona, Girona, Spain 2BMI-LAB, Universidad Miguel Hernandez,´ Elche, Spain [email protected], [email protected], [email protected]

Keywords: Brain-Computer Interface, Common Spatial Pattern, Support Vector Machine, Visual Imagery.

Abstract: Electroencephalographic (EEG) signals contain cognitive information, which can be used by Brain-Computer Interface (BCI) systems to control devices through thought. In this work we study the possibility of detecting the visual imagination of seven different geometric objects (triangle, circle, square, pentagon, line, hexagon and parallelogram). The power spectral density in the α band were compared ofﬂine with using common spatial pattern (CSP) and the variance of each channel, obtaining as a best result the calculation of the CSP plus variance in the α band and classifying the vector of features with a support vector machine (SVM), obtaining an average result of 52% accuracy and a kappa value of 0.43 in the classiﬁcation of the seven geometrical shapes, reaching up to 83% and a kappa value of 0.78 for a single user.

1 INTRODUCTION use techniques widely used in motor imagery classification, such as the common spatial pattern, which Classification of electroencephalographic (EEG) sig- has offered good results (Wang et al., 2005; DaSalla nals is not a simple task as they show a great variabil- et al., 2009) and has been used to classify support vec- ity in time, that is, they are non-stationary signals (Lo tor machines, also widely used in the classification of et al., 2009). BCI are non-invasive systems that are EEG signals (Singla and Haseena, 2013; Mariko and ultimately based on the classification of EEG signals. Junichi, 2014; Lotte et al., 2018). To extract the char- There exist BCI devices that can properly classify acteristics of the signals, we chose to use the vari- EEG signals for the control of different devices (Chen ance of each channel, as it is a fast statistical tech- et al., 2015; Abiyev et al., 2016; Edlinger et al., 2011), nique. Currently, there are few works that offer light but the vast majority of works that can be found in the on how to build a BCI system that takes advantage of literature use motor imagery or related signals, as the the opportunities that visual imagination can offer in slow cortical potentials (SCP) (Mensh et al., 2004). the construction of BCI systems for people who have This limits applications of BCI systems to applica- artistic interests or who want to use BCI systems to tions of motor nature, i.e., where motor imagination carry out design or drawing applications. Here we can play an important role, as could be the movement demonstrate the possibility of classifying seven imag- of a wheelchair or a prosthesis (Ferreira et al., 2010; ined geometric objects with a high degree of accuracy. Galan´ et al., 2008; Guger et al., 2002). However, there Being simple geometric objects, these can be used for are applications that are difficult to implement and ul- the creation of a BCI system such as computer-aided timately are unnatural at the time of use. design. Another problem encountered by non-invasive BCI systems is that, as EEG signals are difficult to classify, these systems can only discriminate few 2 METHODS classes, usually between two and four classes, although it is true that There are studies where this number of classes is overcome using visual evoked 2.1 Subjects potentials (SSVEP) (Song et al., 2017). However, for imagined signals, the four-class limit cannot usu- Three subjects (two female and one male) between 24 ally be overcome. In this paper we have chosen to and 40 years of age, participated in this study. All subjects were right-handed and and they have not vision tors with the corresponding cognitive activity. In problems or neurological disorders. The experiment this work, we have used common spatial pattern, had the informed consent of the people. together with the variance of each component of the common spatial pattern in the feature extraction 2.2 Material phase (Ramoser et al., 2000).

In this study, the g.Nautilus device of g.tec was used. 3.1 Data Processing The device consists of eight wet electrodes at a sam- pling rate of 250 Hz. We used the international 10-10 While the signals were being recorded, they were fil- system to place the electrodes on the scalp. The elec- tered by the hardware of the g.Nautilus device be- trodes used were: Oz, P3, POz, P4, PO7, PO4, PO3 tween 0.01 and 60 Hz with a pass banda filter, the and PO8, with AFz used as reference. We have used order of the filter was six, and then a Notch filter these electrodes because they are located in the occip- between 48 and 52 Hz was also applied. Once the ital area, an area linked to vision and because other EEG signals were registered, they were partitioned studies identified electrodes P4, PO4, PO3 and PO7 into fragments of a second without overlapping, omit- as good to classify signals coming from visual imagi- ting the first second of the imagination task. Once nation (Bobrov et al., 2011). At the time of registering the trials were partitioned, they were filtered in the α the impedance of the electrodes was below 10kΩ. band, between 8 and 12 Hz (this band is important in visual imagery (Bobrov et al., 2011)) using a butter- 2.3 Experimental Paradigm worth band pass filter of order six. All the processing of the EEG signals was done offline using MATLAB The paradigm used lasts a total of eight seconds. Dur- R2016a (Matlab, 2016). ing the first second, a white cross is shown on a black background, indicating the start of the trial. The next 3.2 Common Spatial Pattern two seconds show the geometric figure that the person must imagine, and in the last five the black screen ap- The CSP is a special filter that aims to maximize the pears and is left. This is when the person must imag- variance between two different classes. Although this ine the figure that has been previously shown. The or- algorithm is usually applied between two classes, it der the figures appear on the screen is random, so the can be extended to an arbitrary number N of classes. user does not know what figure will appear and thus The objective of the CSP filter is to return a transfor- cannot anticipate it. This procedure is performed 20 mation matrix W with respect to a group of EEG data times (14 trials each). Once the trials have been regis- matrices Xi. The first step is to calculate the normal- tered, the person makes a short break and re-registers ized spatial covariance matrices of Xi where i is the other 14 trials, making a total of 280 trials (40 trials type of class being analyzed. per geometric figure). The OpenVibe software was used to program of the paradigm, because it offers a X XT R = i i (1) simple and quick way to create these paradigms. i T trace(XiXi ) where T is the transpose operator and trace is the sum of the diagonal elements. 3 PROCESSING PIPELINE Next, it is necessary create the covariance matrix. In all BCI systems, there are differentiated phases, N the first phase always being the pre-processing of the Rc = ∑ Ri (2) EEG signals, where the EEG signals are conditioned i=1 for the following phases. In this pre-processing, EEG where N is the number of classes. Here, we used N = signals are usually partitioned into pieces to be fil- 7. The corresponding matrices of eigenvectors are: tered in the frequency range and spatial filters are ap- T plied to eliminate the artifacts that they may contain. Ri = VλV (3) The next phase is feature extraction. Features where V is the diagonal matrix of eigenvalues. Fi- must contain the necessary information to be able nally, W is constructed as: to differentiate between the different classes that we √ want to classify. Q = V λ−1 (4) The last phase is the classification, which will be responsible for associating the different feature vec- W = (V T ∗ Q)T (5) To create the feature vector, we must transform the not applied, we calculated only the time variance of signals in the following way: the EEG signal channels.

Zi = W ∗ Xi (6) where xi is the set that contains the signals EEG that belong to class i. Finally is necessary calculate the variance for each component of Zi: fi = log(var(Zi( j)) (7) where j is the number of channels. With this method we build feature vectors of di- mension equal to the number of channels of the EEG signals. The advantage that this procedure has is that it creates small vectors and therefore does not need many trials to train the classiﬁer, because if the size Figure 1: Result of applying CSP (orange) and without ap- of feature vectors is very large, more trials will be plying CSP (blue), the dotted line indicates value for a ran- needed (Trunk, 1979). dom selection.

3.3 Classification It can be seen that applying the CSP algorithm results in more significant results than without applying The last stage of the BCI systems is usually the clas- it, obtaining a mean accuracy of 52% with CSP and sification of the vector of features that we have gener- 18% without applying CSP. The subject S3 is the best ated. Nowadays there are several types of classifiers one, with an 83% of accuracy and a 0.78 kappa value. that have been proven to be useful when classifying The accuracy of the subject S2 is the 30% and 0.20 EEG signals, but one of the most used are the support kappa value, which is the lowest accuracy. It should vector machines (SVM). This type of classifier tries be noted that in all three users the results are higher to identify the optimal hyperplane to separate the dif- than we would expect if selections were random. ferent classes, although currently we can find linear or non-linear SVMs: the difference lies in that, in the non-linear SVMs case, the data is mapped into higher dimensionality using a kernel function. This process is known as ”implicit mapping” (DaSalla et al., 2007). In this work, we had used a Radial Basis Functions (RBF) (Teukolsky and Vetterling, 2007), the form of this kernel is show below: 0 2 K(x,x0) = e−γ||x−x || (8) 1 γ = (9) 2σ2 where K is a kernel function of support vectors x and σ is a free parameter. The classification has been done offline, using the k-fold cross validation algorithm. Data is partitioned Figure 2: Distribution of the features depending on the geometric object. Dark blue: Triangle, Orange: Circle, Yellow: into k groups, where k − 1 groups are used to create Square, Lilac: Trapeze, Green: Line, Light blue: Hexagon, the model (called train) and one group to test it (called Red: Parallelogram. test). Finally, all the results are averaged and the fi- nal accuracy is calculated. This process is intended to In Figure 2 we can see the distribution of the fea- eliminate overfitting when training the classifier. In tures according to each user, where the y axis is the this work we used k = 5. P3 channel and the x axis is the PO4. These two channels seem to be the most appropriate to differentiate the geometric objects imagined. Subject 3 (S3) is 4 RESULTS where the accuracy of the classification is greater: in this subject it can be observed that the Pentagon, line, The results in Figure 1 were obtained by applying and hexagon and parallelogram objects have a high sepa- without applying the CSP algorithm before calculat- rability rate, hence the accuracy is much higher than ing the variance of the channels. When the CSP was that of the rest of users. This can be attributed to the level of concentration of the user at the time of doing Ferreira, A., Bastos-Filho, T., Sarcinelli-Filho, M., Mart´ın- the test, or that the user has more facility to visualize Sanchez,´ J., Garc´ıa-Garc´ıa, J., and (2010), M. M.-Q. imagined objects. (2010). Improvements of a brain–computer interface applied to a robotic wheelchair. Communications in Computer and Information Science 52 (2010) 64–73. Galan,´ F., Nuttin, M., Lew, E., P.W. Ferrez, G. V., 5 CONCLUSIONS Philips, J., and (2008), J. M. (2008). A brain- actuated wheelchair: asynchronous and non-invasive In this work we have demonstrated the possibility of brain–computer interfaces for continuous control of robots. Clinical Neurophysiology 119 (2008) using visual imagination for the construction of BCI 2159–2169. systems, specifically using the imagination of simple Guger, C., Harkam, W., Hertnaes, C., and Pfurtscheller, G. geometrical figures. A methodology for this task has (2002). Prosthetic control by an eeg-based brain- com- also been presented using the CSP filter together with puter interface (bci). AAATE 5th European Confer- the calculation of the variance of the transformations ence for the Advancement of Assistive Technology. made with the CSP filter. Lo, M., P.Huang, Tsai, Tsai, P. H., Lin, P., and Yue-Loong, As future work it would be interesting to register H. (2009). The nonlinear and nonstationary properties more people, study what frequency range and which in eeg signals: Probing the complex fluctuations by electrodes are better to perform the classification of hilbert-huang transform. Advances in Adaptive Data Analysis 1(03):461-482. the geometrical figures, and study the impact of the Lotte, F., Bougrain, L., Cichocki, A., Clerc, M., Congedo, different geometrical shapes in the classification. M., Rakotomamonjy, A., and (2018), F. Y. (2018). A review of classification algorithms for EEG-based brain–computer interfaces: a 10 year update. Jour- nal of Neural Engineering, 15(3), 031005, (November ACKNOWLEDGMENTS 2012). Mariko, M. and Junichi, H. (2014). Classification of silent The authors would like to thank the people who have speech using support vector machine and relevance volunteered to make the EEG records, without them this vector machine. Applied Soft Computing Volume 20, study would not have been possible. This work was par- pp 95-102. tially funded by the project TIN2017-88515-C2-2-R from Matlab (2016). version 9.0.0 (R2016a). The MathWorks Ministerio de Ciencia, Inovacion´ y Universidades, Spain. Inc., Natick, Massachusetts. Mensh, B., Werfel, J., and Seung, H. (2004). Bci com- petition 2003-data set ia: Combining gamma-band power with slow cortical potentials to improve single- REFERENCES trial classification of electroencephalographic signals. IEEE Trans. Biomed. Eng., vol. 51, no. 6, pp. 1052– Abiyev, R., Akkaya, N., Aytac, E., Gunsel, I., and Çagman, 1056. A. (2016). Brain-computer interface for control of Ramoser, H., Muller-Gerking,¨ J., and Pfurtscheller, G. wheelchair using fuzzy neural networks. BioMed Re- (2000). Optimal spatial filtering of single trial eeg dur- search International, 2016, 1–9. ing imagined hand movement. EEE Transactions on Bobrov, P., Frolov, A., Cantor, C., Fedulova, I., Bakhnyan, Rehabilitation Engineering. M., and Zhavoronkov, A. (2011). Brain-computer in- Singla, R. and Haseena, B. (2013). Bci based wheelchair terface based on generation of visual images. PLoS control using steady state visual evoked potentials and ONE, 6(6). support vector machines. International Journal of Soft Chen, X., Wang, Y., Nakanishi, M., Gao, X., Jung, and Computing and Engineering (IJSCE). (2015), S. G. (2015). High-speed spelling with Song, X., Xie, S., and Meng, W. (2017). Signal process- a noninvasive brain–computer interface. Proceed- ing methods for ssvep-based bcis. Biomechatronics in ings of the National Academy of Sciences, 112(44), Medical Rehabilitation pp 53-70. E6058–E6067. Teukolsky, S. and Vetterling, W. (2007). Radial Basis Func- DaSalla, C., Kambara, H., Sato, M., and Koike, Y. (2009). tion Interpolation. Single-trial classification of vowel speech imagery using common spatial patterns. Neural Networks 22 Trunk (1979). A problem of dimensionality: A simple ex- (2009) 1334–1339. ample. IEEE Transactions on Pattern Analysis and Machine Intelligence. DaSalla, C., Sato, M., and Koike, Y. (2007). Emg vowel recognition using a support vector machine. Interna- Wang, Y., Gao, S., and Gao, X. (2005). Common spatial tional symposium on measurement, analysis and mod- pattern method for channel selelction in motor im- eling of human functions (pp. 227–232). agery based brain-computer interface. IEEE Engi- neering in Medicine and Biology 27th Annual Con- Edlinger, G., Holzner, C., and (2011), C. G. (2011). A ference. hybrid brain-computer interface for smart home control. Lecture Notes in Computer Science, vol 6762. Springer, Berlin, Heidelberg.