Study of kernel machines towards Brain-computer Interfaces Tian Xilan To cite this version: Tian Xilan. Study of kernel machines towards Brain-computer Interfaces. Artificial Intelligence [cs.AI]. INSA de Rouen, 2012. English. tel-00699659 HAL Id: tel-00699659 https://tel.archives-ouvertes.fr/tel-00699659 Submitted on 21 May 2012 HAL is a multi-disciplinary open access L’archive ouverte pluridisciplinaire HAL, est archive for the deposit and dissemination of sci- destinée au dépôt et à la diffusion de documents entific research documents, whether they are pub- scientifiques de niveau recherche, publiés ou non, lished or not. The documents may come from émanant des établissements d’enseignement et de teaching and research institutions in France or recherche français ou étrangers, des laboratoires abroad, or from public or private research centers. publics ou privés. Th`ese Pr´esent´ee devant l’Institut National des Sciences Appliqu´ees de Rouen pour obtenir Docteur en Sciences Mention Informatique Apprentissage et Noyau pour les Interfaces Cerveau-machine par Xilan TIAN Equipe: LITIS Ecole doctorale: SPMII soutenue le 7 Mai 2012 devant la commission d’examen Rapporteurs : H´el`ene Paugam-Moisy - Universit´ede Lyon Val´erie Louis-Dorr - INPL Directeur : St´ephane Canu - INSA de Rouen Encadrant : Gilles Gasso - INSA de Rouen Examinateurs : Marie-F Lucas - Ecole Centrale de Nantes Alain Rakotomamonjy - Universit´ede Rouen R´esum´e Les Interface Cerveau-Machine (ICM) ont ´et´eappliqu´ees avec succ`es aussi bien dans le do- maine clinique que pour l’am´elioration de la vie quotidienne de patients avec des handicaps. En tant que composante essentielle, le module de traitement du signal d´etermine nettement la performance d’un syst`eme ICM. Nous nous consacrons `aam´eliorer les strat´egies de traitement du signal du point de vue de l’apprentissage de la machine. Tout d’abord, nous avons d´evelopp´e un algorithme bas´esur les SVM transductifs coupl´es aux noyaux multiples afin d’int´egrer diff´erentes vues des donn´ees (vue statistique ou vue g´eom´etrique) dans le processus d’apprentissage. Deuxi`emement, nous avons propos´eune version en ligne de l’apprentissage multi-noyaux dans le cas supervis´e. Les r´esultats exp´erimentaux montrent de meilleures performances par rapport aux approches classiques. De plus, l’algorithme propos´epermet de s´electionner automatiquement les canaux de signaux EEG utiles grˆace `al’apprentissage multi-noyaux. Dans la derni`ere partie, nous nous sommes attaqu´es `al’am´elioration du module de traitement du sig- nal au-del`ades algorithmes d’apprentissage automatique eux-mˆemes. En analysant les donn´ees ICM hors-ligne, nous avons d’abord confirm´equ’un mod`ele de classification simple peut ´egalement obtenir des performances satisfaisantes en effectuant une s´election de caract´eristiques (et/ou de canaux). Nous avons ensuite con¸cu un syst`eme ´emotionnel ICM par en tenant compte de l’´etat ´emotionnel de l’utilisateur. Sur la base des donn´ees de l’EEG obtenus avec diff´erents ´etats ´emotionnels, c’est-`a-dire, positives, n´egatives et neutre ´emotions, nous avons finalement prouv´eque l’´emotion affecter les performances ICM en utilisant des tests statisques. Cette partie de la th`ese propose des bases pour r´ealiser des ICM plus adapt´ees aux utilisateurs. Mot cl´es: Interface Cerveau-machine, apprentissage multi-noyaux, apprentissage semi-supervis´e, TSVM- MKL, LaMKL, ´emotionnel ICM. Abstract Brain-computer Interface (BCI) has achieved numerous successful applications in both clini- cal domain and daily life amelioration. As an essential component, signal processing determines markedly the performance of a BCI system. In this thesis, we dedicate to improve the signal processing strategy from perspective of machine learning strategy. Firstly, we proposed TSVM-MKL to explore the inputs from multiple views, namely, from statistical view and geometrical view; Secondly, we proposed an online MKL to reduce the computational burden involved in most MKL algorithm. The proposed algorithms achieve a better classification performance compared with the classical signal kernel machines, and realize an automatical channel selection due to the advantages of MKL algorithm. In the last part, we attempt to improve the signal processing beyond the machine learning algorithms themselves. We first confirmed that simple classifier model can also achieve satisfying performance by careful feature (and/or channel) selection in off-line BCI data analysis. We then implement another approach to improve the BCI signal processing by taking account for the user’s emotional state during the signal acquisition procedure. Based on the reliable EEG data obtained from different emotional states, namely, positive, negative and neutral emotions, we perform strict evaluation using statistical tests to confirm that the emotion does affect BCI performance. This part of work provides important basis for realizing user-friendly BCIs. Keywords: Brain-computer Interface, multiple kernel learning, semi-supervised learning, TSVM-MKL, LaMKL, emotional BCI. Contents 1 Review of Brain Computer Interfaces 15 1.1 Overview .............................................. 15 1.1.1 Signal acquisition ..................................... 16 1.1.2 Signal processing ..................................... 17 1.1.3 Applications and feedback ................................ 20 1.1.4 Summary ......................................... 21 1.2 Present-day EEG-based BCIs ................................... 22 1.2.1 EEG based BCI paradigms ................................ 22 1.2.2 Representative BCI systems ................................ 23 1.2.3 Summary ......................................... 26 1.3 Machine learning for BCIs ..................................... 26 1.3.1 General framework .................................... 27 1.3.2 Learning problems and their criterions: V and Ω .................... 28 1.3.3 Family of hypothesis .................................. 30 H 1.3.4 Associative learning algorithms .............................. 33 1.3.5 Summary ......................................... 36 1.4 Current limitations and challenges ................................ 37 1.4.1 Sensory interfacing problem ................................ 37 1.4.2 Limited knowledge of neuromechanism .......................... 37 1.4.3 Signal processing issues .................................. 38 1.4.4 Summary ......................................... 38 1.5 Some solutions: from challenges to current PhD study ..................... 38 1.5.1 A multi-kernel framework for inductive semi-supervised learning ............ 39 1.5.2 An online multiple kernel learning: LaMKL ........................ 39 1.5.3 Improving BCI performance beyond machine learning algorithms ............ 39 1.5.4 Summary ......................................... 40 1.6 Conclusions ............................................ 40 2 Semi-supervised learning in BCI 41 2.1 Semi-supervised SVMs ...................................... 42 2.1.1 Problem setting: preliminaries .............................. 43 2.1.2 Transductive SVM ..................................... 43 2.1.3 Laplacian SVM ...................................... 45 2.2 Multiple kernel learning for Transductive SVM .......................... 46 2.2.1 Balancing constraint ................................... 47 2.2.2 Loss functions ....................................... 47 2.2.3 New formulation of TSVM-MKL ............................. 48 2.3 Solving the multiple kernel TSVM problem ............................ 48 2.3.1 Principle of DC programming ............................... 49 2.3.2 Application to TSVM-MKL problem ........................... 49 2.4 Related work ............................................ 53 2.5 Numerical evaluation ....................................... 54 2.5.1 Evaluation under transductive and inductive settings .................. 54 2.5.2 Evaluation under semi-supervised style cross validation setting ............. 58 2.6 Application in BCI data analysis ................................. 59 2.6.1 Application on µ and β based BCI system ........................ 59 2.6.2 Application on motor imagery based BCI system ..................... 61 2.7 Conclusions ............................................ 65 iv Contents 3 Online multi-kernel learning: LaMKL 69 3.1 Multiple Kernel Learning Framework ............................... 70 3.1.1 Linear combination based MKL .............................. 71 3.1.2 Non-linear combination MKL ............................... 74 3.2 ℓp-norm MKL ........................................... 74 3.2.1 ℓp-norm squared MKL formulation ............................ 74 3.2.2 MKL solver: SMO-MKL ................................. 77 3.3 Online MKL: LaMKL ....................................... 79 3.3.1 LaMKL PROCESS .................................... 80 3.3.2 LaMKL REPROCESS ................................... 80 3.3.3 Online LaMKL ....................................... 81 3.4 Numeric evaluation ........................................ 82 3.5 Conclusions and discussions .................................... 84 4 Beyond complex classifier: how to improve signal processing in BCI? 89 4.1 Feature selection VS Classification model ............................ 90 4.1.1 Experimental setting ................................... 90 4.1.2 Signal preprocessing and feature extraction ....................... 91 4.1.3 Experimental analysis ................................... 92 4.1.4 Summary ........................................