Modeling User Emotion in Interactive Play Environments: a Fuzzy Physiological Approach EXAMINING COMMITTEE: Torsten Moller, Chair

Regan Lee Mandryk B.Sc. University of Winnipeg 1997 M.Sc. Simon Fraser University 2000 A DISSERTATIONSUBMITTED IN PARTIALFULFILLMENT OF THE REQUIREMENTSFOR THE DEGREEOF in the School of Computing Science 0 Regan Lee Mandryk 2005 SIMONFRASER UNIVERSITY FALL2005 All rights reserved. This work may not be reproduced in whole or in part, by photocopy or other means, without the permission of the author. APPROVAL NAME: Regan L. Mandryk DEGREE: Doctor of Philosophy TITLE OF THESIS: Modeling User Emotion in Interactive Play Environments: A Fuzzy Physiological Approach EXAMINING COMMITTEE: Torsten Moller, chair Dr. Kori M. Inkpen, Senior Supervisor Associate Professor, Faculty of Computer Science Dalhousie University Dr. Thomas W. Calvert, Senior Supervisor Professor, School of Computing Science Simon Fraser University Dr. M. Stella Atkins, Supervisor Professor, School of Computing Science Simon Fraser University Dr. Lyn Bartram, Supervisor Assistant Professor, School of Interactive Arts and Technology Simon Fraser University, Surrey Campus Dr. Kellogg S. Booth, Supervisor Professor, Department of Computer Science University of British Columbia Dr. John Dill, SFU Examiner Professor, School of Engineering Science Simon Fraser University Dr. Scott Hudson, External Examiner Professor, Human-Computer Interaction Institute School of Computer Science, Carnegie Mellon University DATE APPROVED: SIMON FRASER ~Riv~~dibrary DECLARATION OF PARTIAL COPYRIGHT LICENCE The author, whose copyright is declared on the title page of this work, has granted to Simon Fraser University the right to lend this thesis, project or extended essay to users of the Simon Fraser University Library, and to make partial or single copies only for such users or in response to a request from the library of any other university, or other educational institution, on its own behalf or for one of its users. The author has further granted permission to Simon Fraser University to keep or make a digital copy for use in its circulating collection, and, without changing the content, to translate the thesislproject or extended essays, if technically possible, to any medium or format for the purpose of preservation of the digital work. The author has further agreed that permission for multiple copying of this work for scholarly purposes may be granted by either the author or the Dean of Graduate Studies. It is understood that copying or publication of this work for financial gain shall not be allowed without the author's written permission. Permission for public performance, or limited permission for private scholarly use, of any multimedia materials forming part of this work, may have been granted by the author. This information may be found on the separately catalogued multimedia material and in the signed Partial Copyright Licence. The original Partial Copyright Licence attesting to these terms, and signed by this author, may be found in the original bound copy of this work, retained in the Simon Fraser University Archive. Simon Fraser University Library Burnaby, BC, Canada STATEMENT OF ETHICS APPROVAL The author, whose name appears on the title page of this work, has obtained, for the research described in this work, either: (a) Human research ethics approval from the Simon Fraser University Office of Research Ethics, (b) Advance approval of the animal care protocol from the University Animal Care Committee of Simon Fraser University; or has conducted the research (c) as a co-investigator, in a research project approved in advance, (d) as a member of a course approved in advance for minimal risk human research, by the Office of Research Ethics. A copy of the approval letter has been filed at the Theses Office of the University Library at the time of submission of this thesis or project. The original application for approval and letter of approval are filed with the relevant offices. Inquiries may be directed to those authorities. Simon Fraser University Library Burnaby, BC, Canada ABSTRACT Researchers are integrating emerging technologies into interactive play environments, and established game markets continue to expand, yet evaluating play environments is challenging. While task performance metrics are commonly used to objectively and quantitatively analyse productivity systems; with play systems, the quality of the experience, not the performance of the participant is important. This research presents three experiments that examine users' physiological signals to continuously model user emotion during interaction with play technologies. Modeled emotions are powerful because they capture usability and playability, account for user emotion, are quantitative and objective, and can be represented continuously. In Experiment One we explored how physiological signals respond to interaction with play technologies. We collected a variety of physiological measures while observing participants playing a computer game in four difficulty conditions, providing a basis for experimental exploration of this domain. In Experiment Two we investigated how physiological signals differ between play conditions, and how physiological signals co-vary with subjective reports. A different physiological response was observed when playing a computer game against a co- located friend versus a computer. When normalized, the physiological results mirrored subjective reports. In Experiment Three we developed a method for modeling emotion using physiological data. A fuzzy logic model transformed four physiological signals into arousal and valence. A second fuzzy logic model transformed arousal and valence into five emotions: boredom, challenge, excitement, frustration, and fun. The modeled emotions' means were evaluated with test data, and exhibited the same trends as the reported emotions for fun, boredom, and excitement, but modeled emotions revealed differences between three play conditions, while differences between reported emotions were not significant. Mean emotion modeled from physiological data fills a knowledge gap for objective and quantitative evaluation of entertainment technologies. Using our technique, user emotion can be analyzed over an entire experience, revealing variance within and between conditions. This continuous representation has a high evaluative bandwidth, and is important because the process, not the outcome of playing determines success. The continuous representation of modeled emotion is a powerful evaluative tool, that when combined with other approaches, forms a robust method for evaluating user interaction with play technologies. Keywords: User Interfaces, human-computer interaction, emotion, play, computer games, fun, evaluation methodology, physiology, GSR, EMG, HR, fuzzy logic, affective computing DEDICATION For Kevin For pushing me, pulling me, and walking beside me For supporting my hockey habit and keeping me supplied with pink pens and black sweats I am fortunate beyond wordsto have you as my partner and friend ACKNOWLEDGEMENTS There are so many people to thank for helping me along this journey. I can't possibly name all of the people that supported me, but there are some special thanks that need to go out to those who have profoundly impacted me and my research. For generous financial support, I would like to thank NSERC, NECTAR, NewMIC, and EA Canada. My experiments would not have been possible without the assistance of many people. Thanks to Thecla Schiphorst at SFU Surrey, John Buchanan at EA, the GrUVi Lab at SFU, the Irnager Lab at UBC, and the NRC for lending equipment and providing the participant incentives. Many groups have provided me with a home during the course of my research. For making me welcome, providing me with desk space, and bunging up my inbox with departmental e-mail, I would like to thank the GrUVi Lab at SFU, the Affectionate Computing Group at SFU Surrey, NewMIC, the Imager Lab at UBC, and the EDGE Lab at Dalhousie University. There are far too many people that I've met at each of these institutions to thank them all personally, but I would like to mention Dr. Torsten Moller, and Dr. Ted Kirkpatrick at SFU; Sang Mah, Gordon Pritchard, Glenn Davies, Alan Boykiw, Dr. David Darvill, Julia Rylands, Todd Zimmerman, Brian Corrie, and Dr. Rodger Lea from my time at NewMIC and SFU Surrey; and Dr. Joanna McGrenere and Dr. Jason Harrison from UBC. In addition, I've been fortunate to interact with numerous students from each of the research institutions that opened their doors to me. Thanks to all of the students that I met along the way, and specifically to Diego Maranan for his work on the False Prophets project, which defined my research direction. The members of the EDGE Lab, both past and present, provided friendship, advice, and encouragement during some painful all-nighters. Thanks also to all of the students from NewMIC and the IDRG at UBC. Everyone who commits to a Ph.D. should have a colleague with whom to take the journey. I am deeply fortunate to have had Dr. Stacey Scott, a fellow student and friend who I respect and admire, who went through every step with me, and supported me when I had 'the academic bends'. A special thanks has to go to Hiromi Matsui at SFU for providing endless support, great ideas, and the 'Hiromi Express'. The CS administrative staff, the Network Support Group and ~echnicalSupport Staff in CS at SFU also provided a lot of assistance throughout my time there. A special thanks goes to sumo Kindersley for always giving me answers, friendship, and Diet Pepsi. My committee has provided me with guidance, feedback

Load more