LNU-THESIS-2016.Pdf (2.995Mb)
Total Page:16
File Type:pdf, Size:1020Kb
REAL TIME CLASSIFICATION OF EMOTIONS TO CONTROL STAGE LIGHTING DURING DANCE PERFORMANCE A Thesis Presented to The Faculty of the Department of Biomedical Engineering University of Houston In Partial Fulfillment of the Requirements for the Degree Master of Science In Biomedical Engineering By Shruti Ray August 2016 REAL TIME CLASSIFICATION OF EMOTIONS TO CONTROL STAGE LIGHTING DURING DANCE PERFORMANCE _____________________________ Shruti Ray Approved: ________________________________ Chair of The Committee Dr. Jose Luis Contreras – Vidal, Professor, Department of Electrical and Computer Engineering Committee Members: ________________________________ Dr. Ahmet Omurtag, Associate Professor, Department of Biomedical Engineering _______________________________ Dr. Saurabh Prasad, Assistant Professor, Department of Electrical and Computer Engineering, _______________________ ____________________________ Dr. Suresh K. Khator, Dr. Metin Akay, Founding Chair, Associate Dean John S. Dunn Cullen Endowed Professor, Cullen College of Engineering Department of Biomedical Engineering Acknowledgement I would like to show my deepest gratitude for my advisor, Dr. Jose Luis Contreras - Vidal, for his continuous guidance, encouragement and support throughout this research project. I would also like to thank my colleagues from Laboratory for Noninvasive Brain- Machine Interface Systems, for their immense support and encouragement and help in data collection for analysis. I would like to thank Ms. Rebecca B. Valls and Ms. Anastasiya Kopteva for their dancer performances with EEG caps to help me with the data collection. Additionally, I would like to thank all my friends Su Liu, Thomas Potter, Dr. Kinjal Dhar Gupta and my sister Shreya Ray who have supported me in both happy and adverse conditions. Last, but not the least, I would like to thank my parents and family to believe in my dreams and supporting my quest for higher education. Without their love and support, it wouldn’t have been possible to experience the amount of success that I have. iv REAL TIME CLASSIFICATION OF EMOTIONS TO CONTROL STAGE LIGHTING DURING DANCE PERFORMANCE An Abstract of a Thesis Presented to The Faculty of the Department of Biomedical Engineering University of Houston In Partial Fulfillment of the Requirements for the Degree Master of Science In Biomedical Engineering By Shruti Ray August 201 Abstract Recently, there has been a growing research in the field of Electroencephalography (EEG) based recognition of emotions known as affective computing, where the subjects are either shown pictures to elicit the necessary emotional response or made to imagine a particular situation to produce the desired emotion. Research has shown that different emotions affect the brain waves differently thus leading to further research in computerized recognition of human emotions [1] [2] [3]. In my current master’s thesis, I have analyzed the neural (EEG) data recordings during emotional dance performance from 2 trained dancers. This processed data was used to control the stage lighting color (with changing emotions). Data from subject 1 and subject 2 was used to train the classifier offline. The classification was done by use of Artificial Neural Network. Four musical pieces (details in the method section) were selected by the dancers, each representing a particular emotion – “Anger”, “Fear”, “Neutral” and “Happy”. These emotions were so selected to cover the emotional range of positive, negative and neutral emotions. The feature type of ASM12 [4] with temporal resolution of one second and 50% overlapping hamming window was used. The sub band frequency range - delta (1-3 Hz), theta (4-7 Hz), alpha (8-12 Hz) and beta (14-30 Hz) were used for each of the symmetric electrode pair. The results showed a high level of accuracy of 72.1% was obtained for subject 1 and an accuracy of 75.7% was obtained for subject 2 obtained during offline model training and testing of model using multilayer neural network with 1 hidden layer and 32 hidden layer units. The real-time accuracy was low, and could majorly classify two emotional classes. vi Table of Contents Acknowledgement .............................................................................................................. iv Abstract .............................................................................................................................. vi Table of Contents…………………………………………………………………………………………………vii List of Figures .................................................................................................................... ix List of Tables ..................................................................................................................... xii Introduction ......................................................................................................................... 1 1.1.1 Problem Statement ....................................................................................................... 1 1.2 Contribution ................................................................................................................... 4 1.3 Thesis Organization ...................................................................................................... 4 Background and Related Work ........................................................................................... 6 2.1 Introduction to Affective Computing ......................................................................... 6 2.1.2 Applications of Affective Computing ................................................................. 7 2.2 Emotions ........................................................................................................................ 8 2.2.1 How do we define emotion?................................................................................. 8 2.2.2 Brain and Emotions .............................................................................................. 9 2.3 Machine Learning Background ................................................................................. 10 2.3.1 Methods in machine learning ............................................................................ 10 2.3.2 Algorithms – ........................................................................................................ 11 2.4 Current Technology for emotion classification ...................................................... 13 2.4.1 My Contribution ...................................................................................................... 15 vii Brain Computer Interface .................................................................................................. 16 3.1 What is Brain Computer Interface? ........................................................................ 16 3.2 Brain Machine Learning ......................................................................................... 18 Experimental setup ............................................................................................................ 19 4.1 Subjects ............................................................................................................................... 20 4.2 Equipment used........................................................................................................... 20 4.3 Experimental Protocol ................................................................................................ 20 Data Processing and Analysis ........................................................................................... 26 5.1 Data Preprocessing ..................................................................................................... 26 5.2 Data Analysis ............................................................................................................... 27 5.3 Feature Matrix –.......................................................................................................... 28 5.4 Feature matrix classification – .................................................................................. 28 5.5 Mapping of classified data to stage lights – ............................................................. 29 Results and Discussion ...................................................................................................... 31 6.1 Power spectral density of the four emotions ........................................................... 31 6.2 Classification results ................................................................................................... 38 6.3 Confusion Matrix - ...................................................................................................... 42 6.4 Discussion .................................................................................................................... 43 Future Work and Conclusion ............................................................................................ 46 7.1 Limitations and Future work ..................................................................................... 46 7.2 Conclusion .................................................................................................................... 47 viii References ......................................................................................................................... 48 ix List of Figures Figure 1: Wheel of Emotions created by Robert Plutchik.………………………………………….…2 Figure 2: Classification of emotions in a 2 dimensional scale ……………………….…………..…3 Figure 3: Components