Measuring the experience of playing self-paced games Joe Timothy Cutting Doctor of Philosophy University of York Computer Science December 2018 Abstract Self-paced games are digital games which do not require players to make a move within a particular period of time, so the game can be played at whichever speed the player desires. Game experience measures can help game designers and increase understanding of how games create engagement. This thesis aimed to develop new measures of the experience of playing self-paced games. It investigated two possible measures; measuring cognitive load using pupil dilation and measuring attention using irrelevant distractor images. The first approach found a significant difference in pupil dilation between easy and hard variants of a task taken from the game Two Dots. In a subsequent study, participants played three different versions of Two Dots – one of which required no cognitive effort. There was no significant difference in pupil dilation due to cognitive load between the games. It seems likely that although players could use sustained cognitive effort to play the game, they chose not to, and use other strategies instead. I concluded that pupil dilation is unlikely to be an effective measure of game experience. The second approach developed a new measure known as the distractor recognition paradigm. This measure surrounds the game with constantly changing irrelevant images. After playing, participants are tested on how many of these images they recognise. An initial study found a significant difference in images recognised between two very different versions of the game Two Dots. There was also a significant difference in distractors recognised between three more similar games. This was found to be a stronger measure of game attention than using eye tracking and also found to be effective if the distractor images were placed inside the game graphics. This approach succeeded in the aim of the thesis which was to find a new measure of the experience of playing self-paced games. 2 Contents Abstract ................................................................................................................................................. 2 List of tables ......................................................................................................................................... 8 List of figures ..................................................................................................................................... 10 Acknowledgements .......................................................................................................................... 14 Author’s declaration ......................................................................................................................... 15 1. Introduction ................................................................................................................................... 16 1.1. Measuring game experience ................................................................................................. 17 1.1.1. Self-paced games are different ...................................................................................... 19 1.2. Research questions ................................................................................................................. 20 1.2.1. Pupil dilation ................................................................................................................... 20 1.2.2. Game attention ................................................................................................................ 20 1.3. Methodology .......................................................................................................................... 21 1.4. Outline of the research .......................................................................................................... 22 1.4.1. Outline of pupil dilation experiments ......................................................................... 22 1.4.2. Outline of attention experiments .................................................................................. 23 1.5. Scope ........................................................................................................................................ 25 1.6. Ethics statement ..................................................................................................................... 26 1.6.1. Participant welfare ......................................................................................................... 26 1.6.2. Anonymity and confidentiality .................................................................................... 27 1.6.3. Informed consent ............................................................................................................ 27 1.7. Contributions .......................................................................................................................... 27 1.7.1. Pupil dilation ................................................................................................................... 27 1.7.2. Game attention ................................................................................................................ 28 2. Literature review ........................................................................................................................... 30 2.1. Play and fun ............................................................................................................................ 31 2.2. Game design and other non-empirical approaches .......................................................... 32 2.3. Empirical models of game experience ................................................................................ 35 2.4. Physiological measures of game experience ...................................................................... 43 2.4.1. Overall game experience ............................................................................................... 44 2.4.2. Game events .................................................................................................................... 46 2.5. Approaches to developing a new measure of the experience of playing self-paced games .............................................................................................................................................. 48 2.6. Measuring cognitive load using pupil dilation.................................................................. 48 2.6.1. Pupil dilation experimental design .............................................................................. 50 2.6.2. Data analysis ................................................................................................................... 51 3 2.7. Measuring how well games hold our attention ................................................................. 52 2.7.1. Measuring attention ........................................................................................................ 55 2.8. Chapter conclusion................................................................................................................. 56 3. Experimental setup ........................................................................................................................ 58 3.1. Two Dots.................................................................................................................................. 58 3.1.1. How to play Two Dots ..................................................................................................... 60 3.1.2. Variants of the game ....................................................................................................... 60 3.1.3. In-game distractors ......................................................................................................... 64 3.2. Eye tracking equipment ......................................................................................................... 66 3.3. Game experience questionnaire ........................................................................................... 67 4. Measuring cognitive load using pupil dilation ......................................................................... 69 4.1. Experimental design and pilot studies ................................................................................ 70 4.1.1. Confounding factors ....................................................................................................... 70 4.1.2. Experimental design ....................................................................................................... 71 4.1.3. SMI Eye tracker pilot studies ......................................................................................... 71 4.2. Experiment Pupil 1: Audio stimulus ................................................................................... 73 4.2.2. Hypothesis ....................................................................................................................... 73 4.2.3. Procedure ......................................................................................................................... 75 4.2.4. Results ............................................................................................................................... 76 4.2.5. Discussion ........................................................................................................................ 80 4.3. Experiment Pupil 2: Visual stimulus ..................................................................................