PHYSIOLOGICAL SELF REGULATION WITH BIOFEEDBACK GAMES A Dissertation by AVINASH RAO PARNANDI Submitted to the Office of Graduate and Professional Studies of Texas A&M University in partial fulfillment of the requirements for the degree of DOCTOR OF PHILOSOPHY Chair of Committee, Ricardo Gutierrez-Osuna Committee Members, Tracy Hammond Dylan Shell Thomas Ferris Head of Department, Dilma Da Silva May 2017 Major Subject: Computer Engineering Copyright 2017 Avinash Parnandi ABSTRACT Mental stress is a global epidemic that can have serious health consequences including cardiovascular diseases and diabetes. Several techniques are available to teach stress self-regulation skills including therapy, meditation, deep breathing, and biofeedback. While effective, these methods suffer from high drop-outs due to the monotonic nature of the exercises and are generally practiced in quiet relaxed environment, which may not transfer to real-world scenarios. To address these issues, this dissertation presents a novel intervention for stress training using games and wearable sensors. The approach consists of monitoring the user’s physiological signals during gameplay, mapping them into estimates of stress levels, and adapting the game in a way that promotes states of low arousal. This approach offers two key advantages. First, it allows users to focus on the gameplay rather than on monitoring their physiological signals, which makes the training far more engaging. More importantly, it teaches users to self-regulate their stress response, while performing a task designed to increase arousal. Within this broad framework, this dissertation studies three specific problems. First, the dissertation evaluates three physiological signals (breathing rate, heart rate variability, and electrodermal activity) that span across the dimensions of degrees of selectivity in measuring arousal and voluntary control in their effectiveness in lowering arousal. This will identify the signal appropriate for game based stress training and the associated bio-signal processing techniques for real-time arousal estimation. Second, this dissertation investigates different methods of biofeedback presentation e.g. ii visual feedback and game adaptation during gameplay. Selection of appropriate biofeedback mechanism is critical since it provides the necessary information to improve the perception of visceral states (e.g. stress) to the user. Furthermore, these modalities facilitate skill acquisition in distinct ways (i.e., top-down and bottom-up learning) and influence retention of skills. Third, this dissertation studies reinforcement scheduling in a game and its effect on skill learning and retention. A reinforcement schedule determines which occurrences of the target response are reinforced. This study focuses on continuous and partial reinforcement schedules in GBF and their effect on resistance to extinction (i.e. ability to retain learned skills) after the biofeedback is removed. The main contribution of this dissertation is in demonstrating that stress self-regulation training can be embedded in videogames and help individuals develop more adaptive responses to reduce physiological stress encountered both at home and work. iii DEDICATION Dedicated to my family. iv ACKNOWLEDGEMENTS I would like to thank my advisor Dr. Ricardo Gutierrez Osuna for giving me an opportunity to pursue a PhD and teaching me the ropes of research. His insights were crucial to the progress of my research and my own personal development as a researcher. He also provided me numerous opportunities to expand my horizon and encouraged me to pursue a career in research/academia for which I would be forever grateful. Thanks to Dr. Tracy Hammond and Dr. Dylan Shell for agreeing to be on my PhD committee and their valuable suggestions. I would also like to thank Dr. Thomas Ferris for being on my committee, for the guidance and support, and for regularly taking time for the research and teaching related discussions. In addition to my committee, I would like to acknowledge Dr. Beena Ahmed, Dr. Eva Shipp, Dr. Kirrie Ballard, Mohona Bhowmick, Dr. Maja Mataric, and Dr. Eric Wade for their guidance over the years. I would also like to acknowledge Christopher Blanchard, James Burian, Mary Thompson, Kim Truong, and Youngpyo Son for developing the initial version of the ChillOut game, which was used in dissertation. This dissertation would not have been possible without the help and support of my lab mates and friends who led the way in many instances and generously invested their time. I would like to recognize the following individuals (in no particular order): Rhushabh Bhandari, Anshul Gupta, Manisha Srivastava, Dr. Sandesh Aryal, Dr. Jin Huang, Virendra Karappa, Guanlong Zhao, Christopher Liberatore, Dennis Silva, Zelun v Wang, Adam Hair, Purvesh Karkamkar, Roger Solis, Shaojin Ding, Tian Lan, Difan Chen, Dr. Rakesh Gosangi, Dr. Joseph Lee, Amol Joshi, Abhipsha Kikani, Nitya Singh, Ishan Bajaj, Akshay Jain, Karthik Sridhara, Dr. Michael Chang, Eve Chang, Vatsal Shah, Dr. Manish Bansal, and Payel Banerjee. Finally, I would like to thank my parents, sister and brother-in-law for their incredible support, patience, and understanding throughout the process. vi CONTRIBUTORS AND FUNDING SOURCES Contributors This work was supervised by a dissertation committee consisting of Professor Ricardo Gutierrez-Osuna, Professor Dylan Shell, and Professor Tracy Hammond of the Department of Computer Science and Engineering and Professor Thomas Ferris of the Department of Industrial and Systems Engineering. The data presented in Chapter 3 was completed by the student in collaboration with Professor Ricardo Gutierrez-Osuna of the Department of Computer Science and Engineering and Professor Beena Ahmed of the Department of Electrical and Computer Engineering at the Texas A&M University Qatar and Dr. Eva Shipp of the Texas A&M Transportation Institute. All other work for the dissertation was completed independently by the student. Funding This work was supported by the Qatar National Research Fund (a member of Qatar Foundation) under NPRP Grant #5-678-2-282. Its contents are solely the responsibility of the authors and do not necessarily represent the official views of the funding agency. vii TABLE OF CONTENTS Page ABSTRACT .......................................................................................................................ii DEDICATION .................................................................................................................. iv ACKNOWLEDGEMENTS ............................................................................................... v CONTRIBUTORS AND FUNDING SOURCES ............................................................vii TABLE OF CONTENTS ............................................................................................... viii LIST OF FIGURES ........................................................................................................... xi LIST OF TABLES ........................................................................................................... xv 1. INTRODUCTION ...................................................................................................... 1 1.1 Research challenges ........................................................................................ 5 1.2 Specific research goals.................................................................................... 7 1.3 Summary of findings ...................................................................................... 8 1.4 Dissertation outline ......................................................................................... 9 2. RELATED WORK ................................................................................................... 10 2.1 Stress and the human body ........................................................................... 10 2.1.1 Effects of stress on health ................................................................. 11 2.1.2 Effects of stress on mental health ..................................................... 12 2.1.3 Effects of stress on performance ....................................................... 13 2.2 Traditional intervention for stress management ........................................... 14 2.2.1 Cognitive behavioral therapy ............................................................ 15 2.2.2 Self-guided methods ......................................................................... 17 2.3 Technology based methods ........................................................................... 18 2.3.1 Videogames for stress recovery and health and wellness applications ....................................................................................... 20 2.3.2 Biofeedback games for stress self-regulation ................................... 22 2.3.3 Biofeedback games and relaxation skill transfer .............................. 25 3. DESIGN AND VALIDATION OF THE GAME BIOFEEDBACK APPROACH ............................................................................................................. 29 viii 3.1 System overview ........................................................................................... 29 3.2 Biofeedback game......................................................................................... 32 3.3 Game biofeedback and instrumental conditioning ....................................... 35 3.3.1 Game adaptation ..............................................................................