Recommendations for Integrating a P300-Based Brain Computer
Total Page:16
File Type:pdf, Size:1020Kb
Recommendations for Integrating a P300-Based Brain Computer Interface in Virtual Reality Environments for Gaming Grégoire Cattan, Cesar Mendoza, Anton Andreev, Marco Congedo To cite this version: Grégoire Cattan, Cesar Mendoza, Anton Andreev, Marco Congedo. Recommendations for Integrating a P300-Based Brain Computer Interface in Virtual Reality Environments for Gaming. Computers, MDPI, 2018, Special Issue Advances in Mobile Augmented Reality, 7 (2), pp.34. 10.3390/comput- ers7020034. hal-01801151 HAL Id: hal-01801151 https://hal.archives-ouvertes.fr/hal-01801151 Submitted on 28 May 2018 HAL is a multi-disciplinary open access L’archive ouverte pluridisciplinaire HAL, est archive for the deposit and dissemination of sci- destinée au dépôt et à la diffusion de documents entific research documents, whether they are pub- scientifiques de niveau recherche, publiés ou non, lished or not. The documents may come from émanant des établissements d’enseignement et de teaching and research institutions in France or recherche français ou étrangers, des laboratoires abroad, or from public or private research centers. publics ou privés. computers Review Recommendations for Integrating a P300-Based Brain Computer Interface in Virtual Reality Environments for Gaming Grégoire Cattan 1,2,*, Cesar Mendoza 1, Anton Andreev 2 and Marco Congedo 2 1 IHMTEK (Interface Homme-Machine Technologie) Company, 38200 Vienne, France; [email protected] 2 Department of Image and Signal, University Grenoble Alpes, CNRS, Grenoble INP, GIPSA-lab, 38000 Grenoble, France; [email protected] (A.A.); [email protected] (M.C.) * Correspondence: [email protected]; Tel.: +33-(0)-474-782-229 Received: 8 March 2018; Accepted: 18 May 2018; Published: 28 May 2018 Abstract: The integration of a P300-based brain–computer interface (BCI) into virtual reality (VR) environments is promising for the video games industry. However, it faces several limitations, mainly due to hardware constraints and constraints engendered by the stimulation needed by the BCI. The main limitation is still the low transfer rate that can be achieved by current BCI technology. The goal of this paper is to review current limitations and to provide application creators with design recommendations in order to overcome them. We also overview current VR and BCI commercial products in relation to the design of video games. An essential recommendation is to use the BCI only for non-complex and non-critical tasks in the game. Also, the BCI should be used to control actions that are naturally integrated into the virtual world. Finally, adventure and simulation games, especially if cooperative (multi-user) appear the best candidates for designing an effective VR game enriched by BCI technology. Keywords: brain computer interface (BCI); virtual reality (VR); design; game design; brain computer interface; virtual reality 1. Introduction A video game can be defined as “a mental contest, played with a computer according to certain rules, for amusement, recreation, or winning a stake”. It has also been defined briefly as “story, art, and software” [1]. In some cases, for example in serious games, amusement is not the main goal, however, to date amusement still plays a major role in video game industry. Although by completely different means, virtual reality (VR) and the brain–computer interface (BCI) are both excellent candidates for enhancing the possibilities of entertainment and satisfaction in video games. Indeed, both enhance immersion and it is a common belief that this encourages the feeling of amusement. The concept of immersion was defined in [2], observing that everybody may enjoy a game with immersion, even if the gaming control seems to play the main role in the user’s enjoyment. According to [1] this immersion feeling is created by computer graphics, sound, haptics, affective computing and advanced user interfaces that increase the sense of presence. Virtual reality is a collection of devices and technologies enabling the end user to interact in three dimensions (3D) [3], e.g., spatialized sounds and haptic gloves (for example Dexmo, Dexta Robotics, Shenzhen, China). The particular type of experience that is created by VR is emphasized by [4]. Such experience is named telepresence, defined as the experience of presence in an environment by means of a communication medium [4], joining the concept of presence of [1]. A BCI can also enhance the feeling of presence in the virtual world since it can replace or enhance mechanical inputs. According to [2], immersive games are played using three Computers 2018, 7, 34; doi:10.3390/computers7020034 www.mdpi.com/journal/computers Computers 2018, 7, 34 2 of 21 different kinds of inputs: visual, auditory and mental; since a BCI may transform ‘mental’ signals into input commands, such an interface may play an unique role in the mentalization process involved in the feeling of immersion. However, considering the limitations of a BCI system (to be analysed later), it is stillComputers not clear 2018, to7, x what extent current BCI technology may improve immersion. As2 pointed of 21 out in [2], “engagement,games are played and using therefore three different enjoyment kinds throughof inputs: immersion,visual, auditory is notand possiblemental; since if there a BCI are may usability and controltransform problems”. ‘mental’ signals into input commands, such an interface may play an unique role in the Anmentalization element of process amusement involved derives in the feeling from theof immersion. originality However, and futuristic considering aspect the limitations of BCI technology of as compareda BCI system to other (to traditional be analysed inputs,later), it likeis still a mouse,not clear a to joystick what extent or a current keyboard. BCI technology Nonetheless, mayas often happensimprove in the technologicalimmersion. As industry,pointed out BCI intechnology [2], “engagement, risks beingand therefore dropped enjoyment by the general through public if immersion, is not possible if there are usability and control problems”. the improvementAn element it brings of amusement is not worth derives as from compared the origin toality the and effort futuristic needed aspect for of its BCI use. technology Virtual reality has alreadyas compared enjoyed tothe other “wow-factor” traditional inputs, and like VR asystems mouse, a tendjoystick to or be aemployed keyboard. Nonetheless, nowadays as in often commercial events especiallyhappens in forthe raisingtechnological this effectindustry, (Feel BCI Wimbledon technology risks by Jaguar,being dropped Coca Cola’s by the Santa’sgeneral public Virtual if Reality Sleigh Ride,the improvement McDonald’s it Happybrings is Meal not worth VR Headset as compared and to Ski the App, effort Michelle needed for Obama’s its use. VRVirtual Video, reality XC90 Test Drive byhas Volvo, already etc.) enjoyed (http://mbryonic.com/best-vr/ the “wow-factor” and VR system).s tend The to “wow-factor” be employed nowadays is defined in commercial in the Cambridge events especially for raising this effect (Feel Wimbledon by Jaguar, Coca Cola’s Santa’s Virtual Reality dictionary as “a quality or feature of something that makes people feel great excitement or admiration”, Sleigh Ride, McDonald’s Happy Meal VR Headset and Ski App, Michelle Obama’s VR Video, XC90 and wasTest previously Drive by Volvo, studied etc.) in (http://mbryonic.com/best-vr/). the domain of marketing The and “wow-factor” education (e.g.,is defined [5,6]). in Thethe recent developmentCambridge of dedicateddictionary as VR “a quality headsets, or feature that of is, something head-mounted that makes devices people (HMDs,feel great excitement e.g., the Oculus, Facebook,or admiration”, Menlo Park, and CA, was USA;previously HTC studied Vive, in HTC, the domain Taoyuan, of marketing Taiwan; and Google education Cardboard, (e.g., [5,6]). Google, MountainThe View,recent development CA, USA) of has dedicated paved VR the headsets, way to that the is, commercializationhead-mounted devices of (HMDs, combined e.g., the BCI+VR technology.Oculus, Indeed Facebook, HMDs Menlo provide Park, US; an HTC already Vive, HTC, built-in Taoyuan, structure Taiwan; that Goog canle Cardboard, support the Google, embedding Mountain View, US) has paved the way to the commercialization of combined BCI+VR technology. of EEG (electroencephalography) electrodes, which are needed for the BCI. The Neurable Company Indeed HMDs provide an already built-in structure that can support the embedding of EEG (Cambridge,(electroencephalography) MA, USA) has recently electrodes, announced which ar ae product needed combiningfor the BCI. an The HTC Neurable Vive (Taoyuan, Company Taiwan) with an(Cambridge, EEG cap. The MA, HTC USA) Vive has (Taoyuan,recently announce Taiwan),d a as product well as combining other HMDs an HTC such Vive as the (Taoyuan, SamsungGear (Samsung,Taiwan) Seoul, with Korea) an EEG usecap. inboardThe HTC electronics,Vive (Taoyuan thus, Taiwan), herein as well we referas other to HMDs them such as active as thedevices. On the contrary,SamsungGearpassive (Samsung,HMDs Seoul, consist Korea) of ause simple inboard mask electronics, with lenses thus herein in which we refer a smartphone to them as active is inserted devices. On the contrary, passive HMDs consist of a simple mask with lenses in which a smartphone (Figure1). Passive HMDs are particularly promising for the BCI+VR field since they are very affordable, is inserted (Figure 1). Passive HMDs are particularly