The Effects of Immersion and Increased Cognitive Load on Time Estimation in a Virtual Reality Environment

Mehrdad Ghomi

B.Sc, The University of British Columbia, 2015

A THESIS SUBMITTED IN PARTIAL FULFILLMENT OFTHEREQUIREMENTSFORTHEDEGREEOF

MASTER OF APPLIED SCIENCE

THE FACULTY OF GRADUATE AND POSTDOCTORAL STUDIES (Electrical and Computer Engineering)

The University of British Columbia (Vancouver)

October 2018

c Mehrdad Ghomi, 2018 The following individuals certify that they have read, and recommend to the Faculty of Graduate and Postdoctoral Studies for acceptance, the thesis entitled:

”The Effects of Immersion and Increased Cognitive Load on Time Estimation in a Virtual Reality Environment”

Submitted by Mehrdad Ghomi in partial fulﬁllment of the requirements for the degree of Master of Applied Science in Electrical and Computer Engineer- ing.

Examining Committee: Dr. Sid Fels (Co-supervisor, Electrical and Computer Engineering) Dr. Bernie Garrett (co-supervisor, Nursing) Dr. Mathew Yedlin (Committee Chair, Electrical and Computer Engineering)

ii Abstract

The perceived duration of a time interval can seem shorter or longer relative to real time (i.e., solar time or clock time) depending on what fills that time interval. Re- search has suggested that increased immersion alters a users ability to reproduce a given duration whilst doing a simple task or playing a game in an Immersive Virtual Environment (IVE). Virtual Reality (VR) allows users to experience virtual environments similar to the real world. The contribution of this experimental research is to explore the effects of undertaking a cognitive spatial task and immersion within a VR environment on a persons perception of time. A VR experience using a cognitive task (maze navigation) was compared with a non VR (control) experience of the same task to explore if the effects exist and if the effects are more significant in an IVE compared to a screen-based simple multimedia experience. Also, a VR experience of the environment without any task was compared to the same environment with the cognitive task to establish the effect of a spatial cognitive task on temporal perception. More specifically, this study measured how much temporal distortion is achievable utilizing cognitive tasks in a VR experience. In this thesis the use of cognitive tasks and VR are the independent variables and the perceived duration of the experiment (time) is the dependent variable. Obtained data suggest that being immersed in a VR experience results in 16.10% underestimation of time, while a non-VR experience results in 7.5% overestimation of time. Moreover, navigating mazes that involve a high cognitive load results in 6.45% underestimation of time. Finally, the combination of VR and high cognitive load (navigating the mazes without guiding lines in a VR experience) result in 22.18% underestimation of time. Finally, the implications of this research are discussed at the end of this thesis.

iii Lay Summary

Virtual reality (VR) technologies have granted users the possibility of experiencing virtual environments in similar ways as experiencing the real world. One area of interest in the development of VR is the way in which people perceive time within a VR experience. The goal of this study is to identify whether immersion within VR (using head mounted devices) and cognition load (performing spatial cognitive tasks) result in the underestimation of the passage of time by the user. The contribution of this research is the investigation of the effects of cognitive load and immersion on our perception of time. The results suggest that increased immersion in VR and cognitive activity (undertaking spatial tasks, such as navigation) reduces the perceived duration of time.

iv Preface

This thesis is original, unpublished, independent work by the author. In the second chapter (Background Knowledge), sections 2.1 (Factors that inﬂuence temporal perception) and 2.2 (VR and related work on temporal perception) contain descriptions of other research, which are cited. Dr. Sidney Fels and Dr. Bernie Garrett were the supervisors of the research. The Statistical Opportunity for Students (SOS) program experts from University of British Columbia (UBC) Department of Statistics helped me with my data analysis in chapter 3.6 of this thesis. The necessary ethical review was requested from the UBC Behavioral Research Ethics Board (BREB) and approval was obtained (H17-00106) (See Appendix C) before recruitment for the experiment.

v Table of Contents

Abstract ...... iii

Lay Summary ...... iv

Preface ...... v

Table of Contents ...... vi

List of Tables ...... ix

List of Figures ...... x

Glossary ...... xi

Acknowledgments ...... xii

Dedication ...... xiii

1 Introduction ...... 1 1.1 Problem Identiﬁcation ...... 1 1.2 Statement of the Purpose ...... 3 1.3 Research Questions ...... 4 1.4 Investigation ...... 5 1.5 Signiﬁcance of the Study ...... 5

2 Background Knowledge ...... 7 2.1 Factors That Inﬂuence Temporal Perception ...... 9

vi 2.2 VR and Related Work on Temporal Perception ...... 14

3 Methods ...... 18 3.1 Experiment ...... 19 3.1.1 Experimental Design ...... 20 3.1.2 Ethical Review ...... 20 3.2 Participant Recruitment ...... 21 3.2.1 Sampling Plan ...... 21 3.2.2 Inclusion and Exclusion Criteria ...... 21 3.2.3 Recruitment Methods ...... 22 3.3 Materials ...... 22 3.3.1 Location ...... 22 3.3.2 Hardware ...... 22 3.3.3 Software ...... 23 3.4 Instruments ...... 24 3.4.1 Demographic Questionnaire ...... 24 3.4.2 Post-experience Questionnaires ...... 25 3.4.3 Terminal Interview ...... 25 3.4.4 Data Log ...... 26 3.5 Procedures ...... 26 3.6 Analysis ...... 29 3.6.1 Quantitative Analysis ...... 29 3.6.2 Linear Mixed Effects Model ...... 31 3.6.3 Qualitative Analysis ...... 34

4 Results ...... 35 4.1 Quantitative Data ...... 35 4.1.1 Descriptive Univariate Statistics ...... 35 4.1.2 Inferential Statistics ...... 36 4.2 Qualitative Data ...... 39 4.2.1 Demographic Questionnaires ...... 39 4.2.2 Post-Experience Questionnaire Data ...... 40 4.2.3 Terminal Interview Data ...... 40

vii 5 Discussion ...... 42 5.1 Limitations ...... 47

6 Conclusion ...... 49 6.1 Implications and Future Work ...... 50

Bibliography ...... 52

Appendix A ...... 57

Appendix B ...... 58

Appendix C ...... 61

Appendix D ...... 65

Appendix E ...... 67

viii List of Tables

Table 3.1 Four Experiment Conditions ...... 19 Table 3.2 Latin Square Design ...... 27

Table 4.1 Results from the four runs of the experiment for participant 1 . 38 Table 4.2 Results of the R Code for Various Experiment Conditions (A Bolded P-value indicates Signiﬁcant Effect) ...... 38 Table 4.3 Age Distribution of Participants ...... 38 Table 4.4 Gender Distribution of Participants ...... 39

Table D.1 Factors That Effect Perception of Time ...... 66

Table E.1 User Study Data ...... 72

ix List of Figures

Figure 3.1 HTC Vive Setup and Equipment ...... 23 Figure 3.2 Top View of the Tutorial Maze with Guiding Lines ...... 28 Figure 3.3 Top View of the Lobby (four Portals to the four main Mazes) . 29 Figure 3.4 View of a Maze With Guiding Lines, Pictures and Voice Overs 30

Figure 4.1 Error Distribution of all Trials (The Horizontal Axis Shows the Absolute Error in Seconds, Vertical Axis Shows the Participant Count) ...... 37 Figure 4.2 Actual and Perceived Time on Various Experiment Conditions 40

Figure 5.1 Actual, Perceived and Mean Error for all Four Mazes . . . . . 43 Figure 5.2 Histogram of Perceived Over Actual Ratio Distribution . . . . 48

Figure B.1 Top View of All 4 Identical Main Mazes (2 with guiding lines and pictures, 2 without lines and pictures) ...... 59 Figure B.2 Players View of a Maze ...... 60

Figure C.1 Demographic Questionnaire ...... 62 Figure C.2 Post Exposure VR Experience Questionnaire ...... 63 Figure C.3 Post Exposure Semi-Structured Interview Questions . . . . . 64

x Glossary

3D Three-Dimensional

ADHD Attention Deﬁcit Hyperactivity Disorder

ANOVA Analysis of Variance

BREB Behavioral Research Ethics Board

DF Degree of Freedom

ECE Electrical and Computer Engineering

GHZ Gigahertz

GVR Game VR Environments

HMD Head Mounted Display

IVE Immersive Virtual Environment

LME Linear Mixed Effects

RDLPFC Right Dorsolateral Prefrontal Cortex

RTMS Repetitive Transcranial Magnetic Stimulation

SOS Statistical Opportunity for Students

UBC University of British Columbia

VR Virtual Reality

xi Acknowledgments

I want to thank my parents for making it possible for me to be able to pursue my academic career up to this point. I would like to show my gratitude to Dr. Sidney Fels and Dr. Bernie Garrett for sharing their pearls of wisdom with me during the course of this research, and I thank all of the 34 participants who participated in my user study and gave me very useful insights on the research. I want to thank Dr. Jim Little and Dr. David Poole for helping me with participant recruitment. I am also immensely grateful to Dr. Joseph Anthony for his comments on earlier versions of this research. I would like to mention that any errors are my own and should not tarnish the reputations of these esteemed persons. I would also like to thank my colleagues from the Uni- versity of British Columbia, departments of Electrical and Computer Engineering and School of Nursing who provided insight and expertise that greatly assisted this research. Finally, I thank the Department of Statistics for assistance with data analysis and suggestion of the Linear Mixed Effects Model, and also for comments that greatly improved this research.

xii Dedication

I dedicate all my work to my parents, who have supported me through life, without whom none of my success would be possible.

xiii Chapter 1

Introduction

When a man sits with a pretty girl for an hour, it seems like a minute. Let him sit on a hot stove for a minute and it’s longer than any hour. — Albert Einstein

1.1 Problem Identification The experience of time pervades every aspect of our lives. Duration is an important, basic aspect of the temporal experience [16]. Although there are some indications that being immersed within a virtual environment does alter a user’s perception of time, the nature of time perception within virtual environments represents a field with little published research, to date. The perceived duration of a time interval can seem shorter or longer relative to real time (i.e., solar time or clock time) depending on what fills that time interval. As an example, auditory impulses may interact with our bodys temporal oscillator that works like a pacemaker, and result in perturbations in our estimation of time. Higher frequencies are known to be associated with the overestimation of temporal duration (thinking an event has taken longer than it actually has) while lower frequencies are associated with the underestimation of temporal duration (thinking an event has been shorter than it actually has) [39]. There are two paradigms to consider when researching the perception of time: Prospective, where a person is aware that they needed to make a time estimate before experiencing an event, and retrospective, where a person is unaware of the

1 need for a time estimate until after the event has passed. The prospective paradigm is known as experienced duration, while the retrospective paradigm is known as remembered duration. Researchers [34] illustrate that for prospective time estimation attention is necessary to monitor time passing. In some sense, the person is mentally counting the ticking of some internal clock. However, the more a persons attention is required elsewhere, the more ticks they miss, and hence, they underestimate time [34]. For retrospective time estimates, a person looks back over their memory of a specific duration and essentially counts the memories. The fewer contextual changes requiring distinct memories, the lower the estimated time is. One consistent finding across both paradigms is that if users are engaged in immersive environments and are given two tasks to do, one temporal, and one not, then there is a strong inference effect with the secondary task causing the estimation of time to become shorter, more variable, or more inaccurate. Research has also suggested that increased immersion alters a players ability to reproduce a given duration whilst doing a simple task or playing a game in an immersive virtual environment [14]. Virtual Reality (VR) allows users to experience virtual environments similar to the real world. Nowadays different fields of applications such as training programs, immersive walk-throughs, architectural and industrial designs, and video games benefit from VR technologies by creating virtual scenes. Head Mounted Display (HMD)s have allowed users to experience a sense of presence in virtual scenes by combining motion-tracking and visual stimuli together, rendering graphical scenes in real time. A trend of the growing industry and market is observable due to relatively cheap hardware such as the Oculus Rift, HTC Vive or Samsung Gear VR. With the rapid growth of VR and its ever-increasing number of applications in various fields, altering a user’s perception of time is a useful area for exploration. The alteration of perceived time could, for example, be beneficial during long flights where by distracting the brain, a person may underestimate the temporal duration of the flight. In general, shortening the perceived duration may be beneficial for a variety of unpleasant circumstance. One area of interest in the development of VR is the way in which people

2 perceive time within VR itself. This research focuses on the implementation of time in virtual environments and discusses how, by altering the implementation of the environment, it may be possible to affect a users perception of time and manipulate it.

1.2 Statement of the Purpose The goal of this research is to explore if it is possible to alter users perception of time while they are present in an immersive virtual environment (IVE). The time perception distortion that we chose was underestimation of passage of time, mean- ing, users underestimated the amount of time they spent in the IVE. We decided to explore the speciﬁc contribution of: 1) Increased immersion within a virtual experience 2) Increased cognitive load by undertaking a cognitive spatial task A controlled experiment was designed and conducted in order to verify the level of contribution of immersion and cognitive spatial task on temporal underestimation. Four virtual mazes were developed and 34 participants navigated the mazes in order to ﬁnd the exits of each maze. They solved the mazes with either the presence or absence of a VR experience and spatial task (four runs were done by each participant). The outcome of these 136 runs was analyzed using linear mixed effects model and the quantitative results are reported in the fourth chapter of this thesis. More- over, demographic questionnaires, post-experiment questionnaires and terminal interviews were designed in order to obtain qualitative data as well. These results are also reported in the fourth chapter of this thesis. Although overestimation of time has its own applications in areas such as video game content development or general situations where it is preferable to stretch out the duration of a pleasant experience with minimal resources, the main focus of this thesis is to identify methods that can practically make VR technology users underestimate the passage of time while they are undergoing a virtual experience. Applications of time underestimation can include long travels on airplanes equipped with VR tools or shortening of any unpleasant experience. Implement- ing VR in a way that makes the user underestimate the passage of time can be

3 highly beneficial for uncomfortable medical treatments, such as for post-traumatic stress disorder or exposure therapies (fear of flying, claustrophobia, etc.). We can improve the experience of VR pain treatments (such as burn victims) by reducing the perceived passage of time in treatment periods. This can also help to shorten the perceived duration of a workout on a treadmill or other exercising equipment. Further, this can be used through physical therapy sessions, in which the users can have an enjoyable experience if the perceived duration of the session is reduced. The contribution of this experimental research is to explore the effects of immersion in a VR experience on a user’s perception of time while performing a cognitive task in the environment. A VR experience using a spatial cognitive task was compared with a non-VR experience (a simple screen-based multimedia experience using the same task, as a control) to explore if an effect on temporal perception existed. In addition, a VR experience of the environment without any task was compared to the same environment with a spatial cognitive task, to establish any effect of the task on temporal perception. More specifically, the study was designed to measure how much temporal distortion was achievable utilizing a cognitive task in a VR experience. In this research, the use of VR and a cognitive spatial task were independent variables and the perceived duration of the experiment (perceived time) was the dependent variable.

1.3 Research Questions This work sought to address the following questions: 1) To what degree does immersion within a VR experience decrease the perceived passage of time (underestimation) compared to a non-VR multimedia screen- based experience? 2) To what degree does undertaking a cognitive spatial task decrease the perceived passage of time (underestimation) in a VR experience? To address the above questions, the effects on temporal perception in a VR experience with a simple screen-based multimedia version of the same experience (the control) were compared to test whether VR immersion indeed makes a dif-

4 ference. The effects on temporal perception of introducing a cognitive spatial task into the VR experience was also tested.

1.4 Investigation The two main factors investigated in this study were the effects of VR immersion and cognitive load on temporal underestimation in virtual environments. In order to investigate whether immersion and cognitive load had a significant effect on temporal underestimation in virtual environments, an experimental study was conducted with 34 participants. The participants were asked to navigate four mazes. Two of the mazes provided solution guidelines on the floor, showing the correct path (requiring a minimal cognitive load), while the other two were typical mazes that the participants needed to navigate and solve (using a high spatial cognitive load). The participants were asked to solve one maze with the solution and one without the solution in a VR experience, and another two mazes (one with solution and one without solution) in a simple multimedia screen-based experience. After each maze the participants were asked to fill a questionnaire in which one of the questions asked for their estimated duration for completing the maze. This number was compared with the actual logged duration. The Linear Mixed Effects Model method was used to test whether immersion and cognition load had any significant effects on temporal underestimation.

1.5 Significance of the Study Our gathered and analyzed data suggest that being immersed within a VR experience results in 16.10% underestimation of time, while a non-VR experience results in 7.50% overestimation of time. Also, our data suggests that navigating mazes involving a high cognitive load results in 6.45% underestimation of time. Finally, the combination of VR and high cognitive load (navigating the mazes without guiding lines in a VR experience) resulted in 22.18% underestimation of time in our study. The contribution of this thesis indicates that by using a combination of increased immersion and increased cognitive load, a significant underestimation of time (18.11%) is achievable. These two factors are not only significantly effective, but also practical and feasible to implement.

5 To summarize the contributions of this thesis: 1) It was found that being immersed within a virtual experience significantly affects perception of time and results in users’ underestimation of the passage of time. 2) It was found that undertaking a cognitive spatial task did not result in significant underestimation of time on its own. 3) The most underestimation occurred when both immersion within a VR experience and a cognitive spatial task were present. 4) Although undertaking a cognitive spatial task did not result in significant underestimation of time on its own, when the task was performed in a VR environment, users perception of time was significantly affected and underestimation was the result. 5) It was found that factors such as user frustration can contribute to user overestimation of time.

6 Chapter 2

Background Knowledge

Psychologists and neuroscientists believe that humans have several complementary systems involving the cerebral cortex, cerebellum and basal ganglia that govern our perception of time [31]. Some cell clusters appear to be capable of short-range timekeeping (ultradian rhythm), while the suprachiasmatic nucleus is responsible for the circadian rhythm (daily) [15]. On the other hand, in physics time is unam- biguous and is defined as what a clock reads [24]. Time can also be introduced using an operational definition; where observing a certain number of repetitions of one cyclical event, such as the passage of a free-swinging pendulum, creates a standard unit such as a second. The term specious present refers to the time duration wherein one’s perceptions are considered to be in the present. This term was introduced by E.R Clay (E. Robert Kelly) in 1882 and further developed by William James in 1890 [2]. In relation, time perception refers to the sense of time, which differs from other senses as time cannot be directly perceived but must be reconstructed by the brain. Time is understood by one’s own perception of the duration of events unfold- ing. The perceived time interval between two successive events is referred to as ”perceived duration”. Another person’s perception of time cannot be directly experienced or understood, but it can be objectively studied and inferred through scientific experiments. Time perception is, therefore, a construction of the brain that is manipulable and distortable under certain circumstances. Such temporal illusions help to expose the underlying neural mechanisms working together in order

7 to let us perceive the passage of time. In the field of psychology, experimental studies of time perception have well established that estimates of the duration of a stimulus do not always match its objective time interval, and can be affected by a variety of factors. Since time cannot be directly measured at a given moment in the brain, the mind is often assumed to estimate time based on internal biological or psychological events, or external sig- nals. The effect of exogenous cues (i.e. Zeitgebers or time symbols) from the local environment on endogenous biological clocks (e.g. circadian rhythms) is studied in the field of chronobiology. It is possible that differences in exogenous time cues between those occurring in real world phenomena and those in virtual environments have an effect on internal human time perception. In particular, system latency is known to change the perception of sensory synchronicity and can degrade the perceptual stability of the environment. Space and time are interdependent phenomena not only in physics, but also in human perception [5]. Further essential background knowledge for this research is how VR, immersion and presence are defined in the context of this research. In literature, univer- sally accepted definitions for these phenomenons are yet to emerge, but here are the definitions we found suitable for the purpose of this research from a review article [11]: VR involves an artificial 3-dimensional (3D) environment that is experienced by a person through sensory stimuli (usually visual, aural, and often touch) delivered by a computer and in which ones actions partially determine what happens in the environment [11]. The sense of immersion in an immersive virtual reality (IVR) environment is achieved through visual and auditory stimuli that simulate 3D visual and auditory cues available in the real world. Visually, this is delivered to the user with a head- mounted display (HMD), which presents the computer generated imagery (CGI) of the VR scene from the perspective of each of the users eyes. The HMD usually displays stereoptic (3D) imagery and tracks head motion so that the user seems to move naturally around the virtual space and observe it in a natural manner. Thus, stereoscopic imagery is presented with the 3D visual depth cues of occlusion, perspective, motion parallax, and natural surface textures, all updated interactively

8 in real time. Audio is also simulated in 3D with a head-related transfer function, which enables the HMD wearer to locate simulated sound at a real location in space. Together, this enables the user to gain a sense of immersion inside the 3D virtual world, by presenting the illusion of a 3D scene everywhere the user looks [11]. Presence refers to the sense of being within an environment that is generated by technically mediated means. VR involves human experience in which 2 tech- nological dimensions are considered to contribute to a sense of presence. The ﬁrst dimension is vividness, or the production of a sensorially richmediated environment. The second is interactivity, deﬁned as a users ability to engage with the environment and modify its form or alter events through interaction with it. An immersive environment is considered to be a computed environment that elicits a users sense of presence or ”being there” [11].

2.1 Factors That Influence Temporal Perception Researchers have identified various environmental and non-environmental factors that may alter ones perception of time. Psychoactive Drugs: These drugs can alter the judgement of time. Stimulants are known to produce overestimates of time duration, whereas depressants and anesthetics produce underestimates of time duration [46]. Psychoactive substances include traditional psychedelics such as LSD, psilocybin, and mescaline as well as the dissociative class of psychedelics such as PCP, ketamine and dextromethor- phan. Researchers found psilocybin significantly impaired the ability to reproduce interval durations longer than 2.5 seconds, as well as significantly impairing syn- chronization of motor actions (taps on a computer keyboard) to regularly occurring tones, and impaired the ability to keep tempo when asked to tap on a key at a self-paced but consistent interval [45]. Clinical Disorders such as Parkinson’s disease, schizophrenia, and Attention Deficit Hyperactivity Disorder (ADHD) have also been linked to noticeable impairments in time perception. Neuropharmacological research indicates that the internal clock, used to time durations in the seconds-to-minutes range, is linked to dopamine function in the basal ganglia [7].

9 Repetitive Transcranial Magnetic Stimulation: The Right Dorsolateral Pre- frontal Cortex (RDLPFC) may be important in time perception in humans. In the present studies, a virtual lesion of the rDLPFC created by Repetitive Transcranial Magnetic Stimulation (RTMS) has led to underestimation of time perception for brief intervals (lasting a few seconds) in working memory [20]. Aging: Psychologists have found that the subjective perception of the passing of time tends to speed up with increasing age in humans. Aging may cause people to increasingly underestimate a given interval of time in older people. This fact can likely be attributed to a variety of age-related changes in the aging brain, such as the lowering in dopaminergic levels with older age [8]. Music: The perceived duration of a time period may be influenced by proper- ties of environmental stimuli that fill the period. Findings suggest that perceptions of duration are influenced by music in a way that contradicts conventional wisdom (i.e., the ”time flies when you’re having fun” hypothesis). Time did not pass quickly when an interval was filled with affectively positive stimulation and participants overestimated the interval. Perceived duration was longest for subjects exposed to positively valenced (major key) music, and shortest for negatively valenced (atonal) music. Music pitched in a major key produced the longest average duration estimates and the greatest disparity between actual (i.e., clock) time and perceived time. Music pitched in minor key produced a significantly shorter average duration estimate. Atonal music produced the shortest and the most accurate estimations [19]. Emotional States: Awe: Research has suggested that the feeling of awe has the ability to ex- pand one’s perceptions of time availability [33]. Awe can be characterized as an experience of immense perceptual vastness that coincides with an increase in focus. Consequently, it is conceivable that one’s temporal perception may slow down when experiencing awe [33]. Fear: Research suggests that perceived time seems to slow down when a person sky-dives or bungee jumps [42], or when a person suddenly and unexpectedly senses the presence of a potential predator or mate. This reported slowing in temporal perception may have been evolutionarily advantageous because it may have enhanced our ability to intelligibly make quick decisions in moments that were of

10 critical importance to our survival [38]. Depression: Depression may increase one’s ability to perceive time accurately. One study assessed this concept by asking subjects to estimate the amount of time that passed during intervals ranging from 3 seconds to 65 seconds [21]. Results indicated that depressed subjects more accurately estimated the amount of time that had passed than non-depressed patients; non-depressed subjects overestimated the passing of time. This difference was hypothesized to be because depressed subjects focused less on external factors that may skew their judgment of time. Temporal Illusions: Vierordts Law: This law indicates that short time intervals tend to be overestimated and long ones underestimated, the indifference interval being the inter- mediate length that is neither overestimated nor underestimated, usually found by experiment to be in the region of 0.7 seconds [1]. Kappa Effect: The Kappa effect (also known as perceptual time dilation) is a form of temporal illusion verified by experiment [43], wherein the temporal duration between a sequence of consecutive stimuli is thought to be relatively longer or shorter than its actual elapsed time, due to the spatial/auditory/tactile separation between each consecutive stimuli. Brain biases perception in favor of expectation. Specifically, the results suggest that the brain automatically incorporates prior expectation for speed in order to overcome spatial and temporal imprecision inherent in the sensorineural signal [12]. The kappa effect can be displayed by considering a journey made in two parts that take an equal amount of time. Between these two parts, the journey that covers more distance may appear to take longer than the journey covering less distance, even though they take an equal amount of time (also closely related to the Tau Effect) [18, 43]. Chronostasis: A type of temporal illusion in which the first impression of a new event or task to the brain appears to be extended in time. For example, Chronostasis temporarily occurs when fixating on a target stimulus, immediately following a saccade (e.g., quick eye movement). It results in an overestimation in the temporal duration for which that target stimulus (i.e., post saccadic stimulus) was perceived. This effect can extend apparent durations by up to 500 ms and is consistent with the idea that the visual system models events prior to perception [47].

11 The most well-known version of this illusion is known as the stopped-clock illusion, wherein a subject’s ﬁrst impression of the second-hand movement of an analog clock, subsequent to one’s directed attention (i.e., saccade) to the clock, is the perception of a slower-than-normal second-hand movement rate (the seconds hand of the clock may seemingly temporarily freeze in place after initially looking at it) [47]. The occurrence of chronostasis also extends beyond the visual domain into the auditory and tactile domains. In the auditory domain, chronostasis and duration overestimation occur when observing auditory stimuli. One common example is a frequent occurrence when making telephone calls. If, while listening to the phone’s dial tone, research subjects move the phone from one ear to the other, the length of time between rings appears longer [17]. In the tactile domain, chronostasis has persisted in research subjects as they reach for and grasp objects. After grasping a new object, subjects overestimate the time in which their hand has been in contact with this object. In other experiments, subjects turning a light on with a button were conditioned to experience the light before the button press [28]. The Oddball Effect: Humans typically overestimate the perceived duration of the initial event in a stream of identical events and unexpected oddball stimuli seem to be perceived as longer in duration, relative to expected or frequently presented standard stimuli [32]. The oddball effect may serve an evolutionarily adapted alerting function and is consistent with reports of time slowing down in threatening situations, which is similar to the chronostasis phenomenon. The effect seems to be strongest for images that are expanding in size on the retina (images that are looming or approaching the viewer), and the effect can be eradicated for oddballs that are contracting or perceived to be receding from the viewer [41]. Cognitive Task (Spatial Cognition): One of the most effective means of altering perception of time is to increase spatial cognitive load [35]. Spatial ability or visuo-spatial ability is the capacity to understand, reason and remember the spatial relationships amongst objects or space. Visual-spatial abilities are used everyday from navigation, understanding or ﬁxing equipment, understanding or estimating distance and measurement, and psy- chomotor job performance. Spatial working memory is the ability to temporarily store visual-spatial memories under attentional control, in order to complete a task

12 [9]. This cognitive ability mediates individual differences in the capacity for higher level spatial abilities, such as mental rotation. Spatial working memory involves storing large amounts of short-term spatial memories in the form of a visuo-spatial sketchpad. It is used in the temporary storage and manipulation of visual-spatial information, such as memorizing shapes, colors, location or motion of objects in space. It is also involved in tasks which consist of planning spatial movements, like determining one’s route through a complex building or maze. There exists a group of brain regions engaged in both time perception tasks and during tasks requiring spatial cognitive effort. Thus, brain regions associated with working memory and executive functions have been found to be engaged during time estimation tasks, and regions associated with time perception were found to be engaged by an increase in the difficulty of non-temporal spatial cognitive tasks. The implication is that temporal perception and cognitive processes demanding cognitive control become interlinked when there is an increase in the level of cognitive effort demanded [3]. Regarding the relationship between the neural mechanisms of time perception and other functions, studies on the prefrontal cortex evidence the implication of the same dorsolateral prefrontal cells for both cognitive timing and working memory [37]. Also, the neural mechanisms of timing are recruited in a manner that is modulated by degree of attention [30]. Therefore, there is reason to postulate that correct executive functioning and cognitive control requires participation of both functional, and neuroanatomical components of time perception. In fact, time perception and other executive components such as interference control, seem to share a common neuroanatomical basis in early developmental stages [6]. According to Block and Zakay [3], it was concluded that activation of several cortical (supplementary motor area, insula/operculum, rDLPFC) and subcortical regions (thalamus and striatum) during timing tasks is load-dependent. Addition- ally, activation of the dorsolateral prefrontal cortex under conditions of minimal working memory involvement was observed. These findings support the specific involvement of this region in temporal processing rather than a more general involvement in working memory. Moreover, the overlap between regions participating in both time perception and executive functions (performing spatial cognitive

13 tasks) could also indicate that both functions require similar cognitive abilities, such as sustained attention over time, maintaining information in working memory, and making decisions and preparing motor responses. These findings are also consistent with results from two previous meta-analyses carried out independently to explore the neuroanatomical basis of time perception and cognitive load [26, 29]. During time perception tasks there is participation of various cognitive processes (such as working memory or executive functions). In a parallel manner, during non-temporal cognitive tasks with various levels of cognitive effort, some level of temporal processing is also needed and engaged, therefore, brain regions traditionally associated with working memory and executive functions. Additionally, specific regions traditionally associated with time perception (such as the insula and the putamen) would be engaged during non-temporal cognitive tasks in response to increases in difficulty level. Other Factors Identified as Influencing Temporal Perception: Stimuli Intensity: Time durations may appear longer with greater stimulus intensity (e.g., auditory loudness or pitch) [23]. Higher Change Rates: Time intervals associated with more changes may be perceived as longer than intervals with fewer changes [4]. Body Temperature: The chemical clock hypothesis implies a causal link between body temperature and the perception of duration [44]. Immersion: Studies [14] suggest that increased immersion alters users ability to reproduce a given duration whilst doing a simple task or playing a game. Auditory Stimuli: Auditory stimuli may appear to last longer than visual stimuli [22, 27]. Table-E.1 (See Appendix E) will illustrate various factors discussed above, split into underestimation and overestimation of temporal duration columns.

2.2 VR and Related Work on Temporal Perception As a relatively new technical innovation, the perception of temporal duration in VR has not been extensively researched to date. However, environmental and cognitive factors on time perception have been explored in other contexts. In research undertaken by a group of researchers at the University of Hamburg

14 in Germany, results showed that duration was underestimated when people engaged in performing cognitive tasks. They also showed that the use of time symbols (such as the sun) in an environment could alter a persons perception of time. [35] They created a virtual tropical island for both stereoscopic VR headsets and a regular PC screen with sun as its time symbol in 3 cases, which were: 1. A sun with no motion 2. A sun that moved in a realistic 24-hour cycle 3. A sun that progressed at double the speed of the real 24-hour sun The results showed that when a user was just sitting without performing task, the speed at which the sun moved greatly affected perception of time, and the user tended to overestimate how much time was immersed in the VR for all time symbol conditions. Once cognitive tasks were introduced, the user tended to underestimate time, and the sun manipulation made less of an impact since the user wasnt processing the setting as vividly. This was especially true for the spatial cognition test, likely because the user needed to draw resources from the same area of the brain processing the spatial passage of the sun. At the University of Oxford researchers have worked on the human bodys internal clock and have found evidence for a temporal oscillator underlying time perception with some estimates of its characteristic frequency. They suggested that auditory pulses interact with our bodys temporal oscillator, which works like a pacemaker and can result in perturbations in time estimation. [40] -Higher frequencies were associated with overestimation of temporal duration while lower frequencies were associated with underestimation. In another study, researchers from the Institute of Neuroinformatics, in Zurich, Switzerland, found that increased immersion altered players ability to reproduce a given duration while performing a simple task or playing a game in a virtual environment [14]. At the University of Duke, School of Nursing, researchers evaluated the Effect of VR on time perception in patients receiving chemotherapy. [36] In a trial using VR as a distraction intervention, women with breast cancer were found more likely to experience altered time perception (underestimation) and lung cancer patients were found less likely. They concluded that VR is a non-invasive intervention that can make chemotherapy treatments more tolerable. Understanding factors that

15 predict responses to interventions can help clinicians tailor coping strategies to meet each patient’s needs. Another study at Eindhoven University of Technology aimed to investigate how effective VR was in manipulating and eventually training time perception for children with learning and/or behavior disorders [13]. Children with attention deficit hyperactivity disorder (ADHD) appear to have dysfunction in time orientation (overestimation of passage of time). As the interconnectivity of multiple brain regions is involved in time perception, it was hypothesized that small dys- functions in these brain regions could be causing time perception problems, and could be improved with training. The researchers at Eindhoven presented a the- oretical and empirical framework that used a VR time simulation game for time perception training for children with ADHD. They concluded that children with time-perception problems could be cured in their early years by such techniques. They believe that using Game VR Environments (GVR) for children with ADHD could have some benefits in learning time perception [13]. In another study published in the European Journal of Special Needs Educa- tion, researchers examined whether deaf and hard-of-hearing children perceived a temporal sequence differently under different representational modes. [30] They compared the effect of Three-Dimensional (3D) VR representations on sequential time perception among deaf and hard-of-hearing children with pictorial, textual, spoken and signed representations. They studied 69 participants aged 410, who were divided into two age groups: kindergarten and school age. Using different modes of representation, they examined the childrens ability to arrange episodes of a script in which a temporal order existed. They included six scripts that were adapted to the different modes of representation and thus created a sum of 30 scripts. Their findings demonstrated that the VR 3D representation and the signed representation enabled the most accurate perception of sequential time. The poor- est results were for the textual representation. An interesting finding was that the two dimensional pictorial representation scored low, indicating that this form of representation was not as easy as expected. After exploring the factors that can alter time perception and the existing studies in the relevant literature, combining immersion in VR with increased cognitive activity appears to be a useful area to explore in terms of assessing impacts on

16 inﬂuencing a person’s perception of time.

17 Chapter 3

Methods

Immersion in VR is thought to affect temporal perception [35, 36]. Additionally, spatial cognitive tasks are thought to add to the cognitive load and therefore pro- mote an underestimation of time for the user [13]. Therefore, alongside immersion in VR, a cognitive task (spatial cognition) was selected as a suitable technique to study in a control experiment to explore if temporal perception can be manipulated in a computer-generated environment. The spatial task that was chosen for the experiment was navigating and solving a maze. Maze navigation is a classic example of spatial cognitive tasks as it is a branch of cognitive psychology, which focuses on how people acquire knowledge about their spatial surroundings to determine their location and to navigate towards the goal (maze exit in this case). This particular cognitive task was chosen because task difficulty and task duration could be easily adjusted by the mathe- matical design of the mazes, which added significant flexibility to the experiment. Performing spatial cognitive tasks requires resources from the same areas in the brain as required by duration estimation, and therefore, spatial cognitive tasks can theoretically distract the brain from accurately estimating the passage of time. In the preliminary iterations of this experiment which were conducted to establish the best method and choose the time perception altering factor, the presence of an auditory time cue (tick of a clock with a tempo of 1-tick/1.2 seconds) was used. Although the auditory time cue did seem effective, it did not look as promising as adding cognitive tasks, hence cognitive spatial tasks were used. It appears that

18 immersion and cognitive load can interact with each other to affect estimation of temporal duration. In order to test the inﬂuence of immersion in a VR experience and cognitive load on duration estimation, an experiment was conducted in which participants navigated through four mazes. Their subjective perceived durations of the experiences were collected immediately following the experience, and were compared to the actual recorded durations (See Appendix F).

3.1 Experiment The four null hypotheses that we intended to test were: 1) There is no significant difference in the perception of time experienced between participants using a VR maze and a non-VR maze. 2) There is no significant difference in the perception of time experienced between participants solving a maze with a cognitive task and a maze with no cognitive task. 3) There is no significant difference in the perception of time experienced between participants using a VR maze with a cognitive task and a VR maze with no cognitive task. 4) There is no significant difference in the perception of time experienced between participants using a non-VR maze with a cognitive task and a non-VR maze with no cognitive task. Table 3.1 illustrates all four experimental conditions. Progression through the four maze conditions was designed to take roughly five minutes on average for each maze, followed by a short post-exposure survey (See Appendix D). Upon the completion of the last maze and questionnaire, a semi- structured interview was conducted. Afterwards, the participants were thanked for

Experiment Condition VR Experience Cognitive Task 1 Yes Yes 2 No Yes 3 Yes No 4 No No

Table 3.1: Four Experiment Conditions

19 their time and their incentive gift cards were provided. On average, the whole procedure took around 40 minutes. Each participant was asked for 45 minutes of their time to make sure that there was enough time for all steps to be completed.

3.1.1 Experimental Design The experiment had a retrospective design, in that the participants were not aware that they had to judge the elapsed time until afterwards. Each trial ended as soon as the participant reached the goal, or after 600 seconds had passed. The actual passed time was measured by the systems clock and was recorded in a data log. Maze navigation was chosen as the cognitive task for the user study. The reason behind this design decision was that spatial cognitive tasks share the most resources with time tracking mechanisms in our body among all the cognitive task types. Also, complicated maze navigation is considered a top demanding task in this family of cognitive tasks. In addition, maze navigation was a suitable and compatible task to perform both in VR and on a 2D screen.

3.1.2 Ethical Review The necessary ethical review was requested from the UBC Behavioral Research Ethics Board (BREB) and approval was obtained (H17-00106) (See Appendix C) before recruitment for the experiment. The experiment was assisted by the Elec- trical and Computer Engineering Electrical and Computer Engineering (ECE) department at UBC. which, upon obtaining BREB approval, aided the participant recruitment process by sending invitation emails to ECE graduate students. The signed consent forms were collected and it was acknowledged that all data collected during the experiments would be confidential and would be kept in a secure location . All participant names were removed and replaced with subject numbers. The participants were also informed that their voice would be recorded and would be later transcribed for further analysis and kept safe and confidential at all times. Data collected from the users are completely confidential and kept in a secure area in the School of Nursing Media Room (T302) at UBC Hospital.

20 3.2 Participant Recruitment

3.2.1 Sampling Plan A convenience sample of 30 participants was recruited for the study. This sample size was calculated using the UBC Statistics Departments online tool for sample size/power analysis; the calculation was developed by Dr. Rollin Brant. The mean values we used were Mean1 = 340 ± 60 and Mean2 = 270. We set the sigma value to 60, alpha to 0.05 and power to 80%. In addition, we used the www.clincalc.com website to make sure that the obtained sample size was consistent with other available tools. The desired power elected was 80% for the test and 0.05 as the alpha. We used mean values (Mean1 = 340 ± 60 and Mean2 = 270) and the Cohen’s d effect size was calculated to be 0.433 (21% effect size) based on our results. The generated sample sizes were 28 from UBCs tool and 27 from the clincalc online tool. In addition to all the above calculations, the effect of adding a new participant gradually started to become insigniﬁcant after we gathered data from 30 participants, and therefore we stopped the user study after successfully running the experiment with 34 participants.

3.2.2 Inclusion and Exclusion Criteria The sample included both female and male adults. The inclusion criteria for participants were: 1) Participants must have been over 19 years old by the date they signed the consent form; 2) Participants should have been able to wear a VR head mounted display (HMD) for 5-10 minutes at a time; 3) Participants should be able to perform simple cognitive tasks; 4) Participants should have had no mobility problems when walking short dis- tances. The exclusion criteria were: 1) Participants with hearing or visual impairments (eyeglass visual correction was acceptable);

21 2) Participants with a susceptibility to claustrophobia or motion sickness; 3) In addition, in order to have some standardization of ability, participants had to successfully pass a tutorial maze (with guiding lines on the ﬂoor) within a set time before starting the experiment.

Participants must have successfully ﬁnished the tutorial maze in less than two minutes. If they failed to solve the tutorial maze in time, they were given another chance, where upon a second failure they were thanked for participating in the experiment, received their incentives, and discontinued.

3.2.3 Recruitment Methods The recruitment process used email invitation via the ECE department mailing lists, and ﬂyers were placed across different departments and buildings on the UBC cam- pus. UBC professors (Dr. Little (CPSC 425 and CPSC 505), Dr. Garrett (NURS 540), and Dr. Poole (CPSC 532)) invited students in their classes to participate with direct in-class invitations before a session of their classes started. A consent form was supplied to the participants upon initial recruitment and a signed copy was obtained at the start of the experiment. The participants got the chance to use our VR equipment in lab and to play VR videogames, and they received a 10$ Starbucks gift card as an incentive for their participation.

3.3 Materials

3.3.1 Location The experiment was conducted in a UBC Hospital room that had sufﬁcient area for walking around and exploring whilst participants were wearing HMDs.

3.3.2 Hardware The HTC Vive was used as the VR equipment due to its high graphic quality and ease of movement using its hand controllers. It has motion tracking, two hand controllers that simulate participants hands positions in the IVE, a stereoscopic HMD and stereo headphones, and two location sensors (Figure-3.1). The resolution of

22 its display is 2160 1200 (1080 1200 per eye). The refresh rate is 90 Hz and it covers approximately a 110 degree ﬁeld of view. For rendering, system control and logging an Intel computer was used with a 3.6 Gigahertz (GHZ) quad Core i7 processor, 16GB of main memory and an NVIDIA GeForce GTX 970 graphic card with 8GB of RAM.

3.3.3 Software In order to develop the software for the experiences in the experiment, several multimedia VR and 3D mazes and a lobby environment were required. Unity3D 5.4.1 engine was used to create them. Unity3D is the worlds largest gaming development platform and is widely used among various game/environment developers. It also has one of the largest development communities, which eased the development process. Moreover, it is among the top game engines in terms of rendering quality and optimization. The SteamVR plugin was used for integrating the HTC Vive facilities into this study’s environment. SteamVR produces hand controllers in the game which mirror physical location and orientation. In addition, for the VR experiences the SteamVR and the Vive tracked the physical boundaries of the real world room and displayed safety boundaries using blue dotted lines so the participants did not approach physical walls and obstacles. Most of the code was implemented in C# and JavaScript programming languages. Some public coding libraries were also used for the game engine, physics engine and rendering, all of which were provided in the basic Unity3D development environment and are free to use.

Figure 3.1: HTC Vive Setup and Equipment

23 The materials and shaders were taken from the Unity asset store and were all free to use. For all of the physical interactions in the environment and lighting, the standard Unity libraries of 3D objects was used and any necessary modiﬁcations to construct the maze IVEs was implemented. We recorded audio voiceovers for all instructions and descriptions. During the experiment participants walked around the VR mazes naturally whilst wearing HMDs on their heads. They also walked through the non-VR mazes displayed on a multimedia PC screen using hand controllers to control their movement. Cyber sickness (also known as virtual reality motion sickness) symptoms in the VR experiences can possibly result in general discomfort, headache, stomach awareness, nausea, vomiting, pallor, sweating, fatigue, drowsiness, disorientation, and apathy. In order to reduce this sickness caused by VR, a movement functional- ity was implemented allowing the user to look in a direction and then move in that direction by pressing the trigger button on the hand controllers.

3.4 Instruments The tools used to gather and analyze the data from the user studies were: 1) A Demographic questionnaire 2) A maze post-experiment questionnaire (completed at the end of each maze experience) 3) A semi-structured terminal interview (voice recording and transcript) 4) Data logs of the participants’ progress through the mazes recorded during the experiment

3.4.1 Demographic Questionnaire A brief questionnaire was created containing questions regarding participants basic demographic information such as age and gender. It also asked whether the participants had heard about this experiment and its purpose beforehand, and whether the participants suspected to have cyber sickness or claustrophobia. This questionnaire served as a basic ﬁlter for excluding participants that did not meet the inclusion requirements. The age and gender of the participants were obtained but were not

24 used in this research, as most participants had similar ages; this data could be used for further exploration in the future (See Appendix D).

3.4.2 Post-experience Questionnaires A questionnaire was designed to be administered immediately after each maze experience and contained questions regarding the color of the portal they took as a distracting question. It asked about whether they saw any moving creatures in the maze (which did not exist) and again functioned as another distracting question. The next question which was the goal of this survey was how long they thought they were in the maze from the entering portal to the exit portal. The next question was another distraction question, which asked participants how enjoyable their experience in the maze was. The ﬁnal question regarded the level of interactivity of the maze, which also functioned as a distraction question. The purpose of these distraction questions was to hide the main question regarding time estimation, so that in the next mazes the participants did not focus on the time taken to complete the maze. Most of these questions resulted in quantitative data. The participants’ answers to the question asking how long they thought they were in the maze was the main input of the conducted analysis (See Appendix D).

3.4.3 Terminal Interview A short semi-structured individual interview script was developed to be undertaken with each participant following the experiment. (See Appendix D). Participants would be asked if they experienced any time distortion and if so, they would be asked to explain in which experiences it was the most present. They would also be asked what factors they thought might have resulted in the time distortion. The purpose of this interview was to explore the participants’ experiences in more depth. Questions were designed to provide more details of participant perceptions regarding the passage of time in the experience, and help explain their actions and thought processes with more detail, which otherwise may have been lost. Partici- pants were informed before the interview that the main purpose of the experiment was to assess if any time distortion had occurred for them during the trials, and that the interview would be recorded and transcribed for analysis. See Appendix D for

25 the interview questions.

3.4.4 Data Log A data log was also implemented that would be automatically recorded by the computer during the experiment. Data was recorded to calculate the objective time taken to complete each maze. These records included: the exact position (x, y coordinates) of the player 40 times per second with their timestamps, the exact direction they were looking at for each rendered frame with their timestamps, the timestamps of each enter/exit portals in order to log the duration of each maze and the color of the portal they took, and the time that participants spent to look at pictures on the wall of the mazes and hear their descriptions. This latter element was performed by measuring the amount of time the players spent standing still in the areas in front of pictures that occurred at intervals in the non-cognitive task mazes to slow their progression (see section 3.5 below). Using these logs, the total distance they walked was calculated, as well as their average speed, the cumulative time they spent looking at the pictures, and the percentage of time they spent looking at the pictures versus the total time. These gathered data were sufﬁcient to reproduce the whole experiment, but some of them were not used in the analysis. These logs were dynamically written on a text ﬁle and were stored on the hard disk.

3.5 Procedures The four different maze experiences mentioned at the beginning of this chapter were performed in varied predeﬁned sequential orders using a Latin Square design for each participant to prevent any possible sequencing effects and additional bias (Table-3.2). The order of trials was not randomized due to the fact that the number of participants was not large enough to ensure randomness (34 as the number of participants was not large enough in order to choose the mazes randomly and be sure that all sequence of mazes would be covered). Before starting each maze, the researcher instructed the participant which maze (portal) they should go to ﬁrst. This was followed by a 5 minute VR orientation and acclimatization (taking about 10 minutes in total).

26 Red (maze #1) Green (maze #2) White (maze #3) Yellow (maze #4) Green (maze #2) White (maze #3) Yellow (maze #4) Red (maze #1) White (maze #3) Yellow (maze #4) Red (maze #1) Green (maze #2) Yellow (maze #4) Red (maze #1) Green (maze #2) White (maze #3)

Table 3.2: Latin Square Design

The steps that the participants went through were: 1) The HMD was handed to participants to put on and was made sure to be adjusted and calibrated in a way that suited them. They started the experiment by putting on the HMD and earphones and holding the hand controller (for VR mazes). Al- ternatively for non-VR mazes, participants began the experiment by being guided towards their seat in front of the PC and were handed the controllers. 2) Participants initially started in a small tutorial maze that had a guideline on the floor. They heard instructions on how to move around and navigate towards the exit portal, which took them to the main lobby. If they did not pass on the first attempt, they were given another chance to successfully finish in time. If they were unable to do so, they did not meet the standard, and did not continue to the other mazes. Their participation ended at this point, but they still received the incentive gift card for participating.

3) Upon successfully completing the tutorial maze in two minutes, the success- ful participants were teleported to the main lobby (repositioned from the exit of the actual mazes to a specific coordinates in the lobby environment). At this point, they had a chance of removing the HMD and relaxing their eyes. The main lobby had four colorful portals that each directed them to undertake one of the four different mazes of the experiment. They then were directed to go through one specific portal. 4) If they had started in a maze without the guidelines, they had to navigate the maze and find the exit portal that took them back to the main lobby. After completing the maze, they removed the HMD and undertook a short questionnaire about their experience.

27 Figure 3.2: Top View of the Tutorial Maze with Guiding Lines

5) If they had been teleported to a maze with the guideline, they could follow the green line towards the exit portal that took them back to the main lobby. These mazes contained 20 pictures of interesting landmarks or universities around the world with short audio narrations. The participants heard a 10 second description of the pictures when they approved them, which were hung on the walls at speciﬁc points in the maze. The reason for the use of these pictures was to increase the average time participants took to complete the mazes with guidelines in order to make them comparable with the other two mazes.

6) The participants looped four times into the main lobby and the mazes (2 VR and 2 non-VR) until they ﬁnished all of the mazes. Moreover, one of the two VR mazes and one of the non-VR mazes were with guidelines. In brief, each partic-

28 Figure 3.3: Top View of the Lobby (four Portals to the four main Mazes) ipant navigated one maze with guidelines in a VR experience, one maze without guidelines in a VR experience, one maze with guidelines in a non-VR experience, and one maze without guidelines in a non-VR experience. The experiment was then complete and the participants were asked to remove the HMD and participate in a semi-structured interview answering a few questions in which their voices were recorded.

3.6 Analysis Quantitative and qualitative data were obtained and analyzed from the experiment. These data consisted of objective measurements from timings, and logs, and subjective data from the participants perceptions of the experience.

3.6.1 Quantitative Analysis The participants perceived duration for solving each maze were recorded after each maze experience and then compared to the actual recorded duration of the expe-

29 Figure 3.4: View of a Maze With Guiding Lines, Pictures and Voice Overs rience (See Appendix F). These quantitative data were collected from the surveys and timings recorded during the experiments. Inferential statistics utilizing a Lin- ear Mixed Effects (LME) model was used to compare time perceptions versus actual time for both the cognitive task and immersion conditions. In this experiment the dependent variable was time and independent variables were the two treatments (VR and Cognitive task).

30 3.6.2 Linear Mixed Effects Model

Model and Assumptions Linear mixed models extend simple linear models. These models allow both fixed and random effects. They are particularly beneficial when there is non independence in the data, arising from a hierarchical structure for instance. For example, students could be sampled from within classrooms, patients from within hospitals, or in our case, participants within different maze types. When dealing with hierarchical data, there are multiple approaches that we can take. One simple approach is to aggregate. As an example, imagine 10 participants are sampled from each maze type. Rather than using the individual participants data, which is not independent, we could take the average of all participants within a maze type. This aggregated data would then be independent. Although aggregate data analysis yields consistent and effective estimates and standard errors, it does not really take advantage of all the data, because participants’ data are simply averaged. Another approach to hierarchical data is analyzing data from one unit at a time. Again in our example, we could run four separate linear regressions, one for each maze type in the sample. Although this does work, there are many models and each model does not take advantage of the information in data from other maze types. This can also introduce more noise in our results, such that the estimates from each model are not based on sufficient amounts of data. Linear mixed models (also called multilevel models) can be thought of as a trade off between the above two alternatives. Individual regressions has many estimates and lots of data, but is a noisy method. Aggregation is less noisy, but may lose important differences by averaging all samples within each maze type. LMEs are somewhere in between [10]. Beyond simply caring about correcting standard errors for non independence in the data, there can also be important reasons to explore the difference between effects within and between groups. LMEs allow us to explore and understand these important effects. The core of mixed models is that they incorporate fixed and random effects.

31 A ﬁxed effect is a parameter that does not vary. In contrast, random effects are parameters that are themselves random variables. In equation form:

β ∼ N(µ,σ) (3.1)

This equation is similar to linear regression, in which we assume the data are random variables, but the parameters are fixed effects. Now the data are random variables, and the parameters are random variables (at one level), but fixed at the highest level (for example, we still assume some overall population mean). Independence, being the most important assumption, is one of the main reasons we use mixed models rather than just working with linear models, and allows us to resolve non-independence in our data. However, mixed models can still violate independence if missing important fixed or random effects. A random effect is generally something that can be expected to have a non- systematic, idiosyncratic, unpredictable, or random influence on your data [10]. In experiments, this is often the participant, and we normally want to generalize over the idiosyncrasies of individual participants. Fixed effects, on the other hand, are expected to have a systematic and predictable influence on data. The type of mixed model above (LME) is suitable for systems that contain both fixed effects and random effects. The linear mixed effect model, which is an extension of linear regression, is preferred over Analysis of Variance (ANOVA) in settings where the gathered data contains repeated measures of the same nature [35]. The collected data should describe the relationship between a response variable and independent variables, with coefficients that can vary with respect to one or more grouping variables. Unlike most machine learning problems, here we are interested in using a model of inference of correlating data, rather than a model for the prediction of events. The error of time estimations was distributed as a Gaussian distribution and the exploratory variables were assumed to be related linearly based on [35]. Therefore, the results were analyzed using Linear Mixed Effects (LME) model using the R programming language (suggested by The Statistical Opportunity for Students (SOS) statisticians from UBCs Department of Statistics). The main difference between the LME model and linear regression is that the

32 linear mixed effects model treats observations obtained from the same participant as correlated, whilst a linear regression model treats them as independent. In this experiment, the former was more appropriate since there are likely similarities in perceived time within individuals. For example, some individuals may consistently underestimate the amount of time spent in the maze, and some may consistently overestimate. In order to use an LME model in R, the nlme package and the lme( ) function within that package was used (See appendix A).

LME in This Experiment When there are multiple levels, such as participants navigating the same maze type, the variability in the outcome can be thought of as being either within group or between group (no maze effect was observed from our investigations, and this has been discussed in more depth in chapter 5 of this thesis). Participant level observations are not independent, as within a given maze type participants act more similarly. Units sampled at the highest level (in our example, maze types) are independent. The extra assumptions that we needed to worry about were collinearity (which has been checked based on [35]), influential data points (we had one, but decided to keep all the data), and normality (which has been checked by plotting the data - see chapter 5). In this study the time estimate is the fixed effect term of the model, while individuals habits and qualities represent random effects in the LME model. This model is particularly useful in our setting where the individual specific effect is correlated with the independent variables. The linear mixed effects model is similar to a linear regression model, and the goal of fitting such a model is to determine whether or not independent variables (e.g. VR, no VR, cognitive spatial task and no cognitive spatial task) had a significant effect on perceived time (i.e the difference between perceived time and actual time recorded). The estimated coefficients of either model give the estimated magnitude of the effect of each variable and whether or not they may be statistically significant. A linear mixed effects model with interaction was used for statistical analysis. By adding an interaction term, the model became more flexible and was able to detect differences in how the re-

33 sponse was affected by different combinations of VR/no-VR and cognitive spatial task/no-cognitive spatial task experiment settings. The interaction term estimates the difference of the VR effect in a setting with the solution. If the interaction term in the model is found to be statistically significant, then the effect of both of those factors would be the total of the effect of being in VR, the effect of having a solution, and the effect of the interaction term. This model should only be used if the interaction term is found to be significant; otherwise the additive model described above is sufficient.

3.6.3 Qualitative Analysis For analyzing the qualitative data from the semi-structured interview at the end of the experiment, the recorded script of the interview was subjected to a simple content analysis. Transcripts of the interview were read and reread to identify similar emerging thematic items. These were coded and synthesized into signiﬁcant thematic elements in order to shed light on the participants perceptions about their experiences in the mazes. The main goal of our thematic analysis was to pinpoint themes and patterns in the interview scripts that can help us answer our research questions. A six phase coding and reviewing of the themes were performed, through which the repeating patterns and themes were identiﬁed and coded in order to help understand the outcomes in more depth.

34 Chapter 4

Results

Thirty-four participants (15/34 females and 19/34 males) participated in the experiment (zero drop outs, no participant left the study due to cyber sickness, one participant suspected that she was feeling minor cyber sickness but wanted to continue the experiment, and two participants had prior experience using VR). Most of the participants were UBC graduate students and faculty members. Objective quantitative and qualitative data, as well as some subjective data were obtained. The results were analyzed using the Linear Mixed Effects Model, as mentioned in section 3.6.1 of this thesis. Two different fitting functions were used and compared, one with the VR-Cognitive task interaction term and one without the the interaction term. The interaction term was found not significant (p = 0.1511) (Degree of Freedom (DF) = 99, STD. Error = 0.112) and therefore we used the first model that did not include the interaction term. The interaction term estimates the difference of the VR effect in a setting with the solution. Here in this chapter, the following results are reported.

4.1 Quantitative Data

4.1.1 Descriptive Univariate Statistics Figure-4.1 shows the error distribution of 136 valid observations (and zero outliers) obtained through all trials that were analyzed. The histogram plots the frequency of

35 each absolute-error bucket. The vertical red line passing zero on the horizontal axis indicates an absolute accurate estimation. A negative error (on the left side of the red line) represents underestimation of time while a positive error (on the right side of the red line) corresponds to overestimation. The mean of all 136 data points is -4.3% (4.3% underestimation of time), and the median is -7%. In terms of absolute values, the mean error is found to be -13 seconds (median is -25 seconds). In terms of data distribution, this is a normal graph, which indicates that the time estimation error’s distribution is Gaussian. One last observation from the error distribution histogram is that overall, more people have underestimated than overestimated the durations, while the overestimations are larger in value.

4.1.2 Inferential Statistics Linear Mixed Effects Model: The model was implemented using R programming language (See Appendix A) and formatted in a long table with 34 groups and 136 observations (each row represents one of the four observations for each participant). The first four rows of the table are shown below (Table-4.1) as an example to indicate the first participant’s data. See Appendix F for the full table of all participant data. In this table, perceived time represents participants responses to the question on post-experiment questionnaires which asks, How long were you in the maze from the entrance portal to the exit portal?. The actual time represents the amount of time participants actually were in the maze, which was automatically logged and stored by the application. Ratio is defined as perceived time over actual time. VR column indicates whether the experiment was a VR experience (1) or a screen- based non-VR experience (0). The solution column specifies whether the maze had guidelines (1) or did not have guidelines (0). The maze column shows which maze the experiment was conducted in. Finally, the last column shows the ratio of the difference between perceived and actual times over actual time. All durations are in seconds. Table-4.2 shows the achieved results of the linear mixed effects model using R for the various experimental conditions. A comparison of All VR and All No-VR will show us the effect of immersion within a VR experience on time perception,

36 Figure 4.1: Error Distribution of all Trials (The Horizontal Axis Shows the Absolute Error in Seconds, Vertical Axis Shows the Participant Count) while comparing all the mazes without guidelines (Cog. Task) with all the mazes with guidelines will indicate the effect of increased spatial cognitive load on time perception. Finally, comparing the combination of VR and Cog. Task with control (no VR no Cog. Task) will indicate the effect of combining the two. In this table, VR maze refers to the runs in the experiment in which the participant navigated the maze using the HTC Vice headset. A No-VR maze refers to the maze navigated by the participant on the 2D computer screen (without the VIVE headset). A Cog. Task maze is one that did not have guidelines on the floor and where the participant had to use spatial cognition in order to navigate the maze. A No-Cog maze refers to runs in which there were guidelines on the floor which the participant could follow and reach the maze exit. The analysis of these achieved results and the related discussion around their significance is throughly discussed in the fifth chapter of this thesis.

37 Participant VR No-Task Maze Perceived Actual Ratio Diff/ ID Time (s) Time (s) Actual 1 1 1 R 180 351 0.51 49%(U) 1 1 0 G 300 572 0.52 48%(U) 1 0 1 W 180 383 0.47 43%(U) 1 0 0 Y 240 415 0.58 42%(U)

Table 4.1: Results from the four runs of the experiment for participant 1

Experiment # Under est. (U) P F R DF Std. Condition Obs. Over est. (O) Val. Val. sqr Err. All VR 68 16.10% (U) ≤0.01 26.32 0.28 99 0.09 All No-VR 68 7.50% (O) ≤0.01 31.14 0.32 99 0.07 All Cog. Task 68 6.45% (U) 0.36 22.68 0.26 99 0.08 All No-Cog.Task 68 1.65% (U) 0.36 28.54 0.30 99 0.07 VR and 34 22.18% (U) ≤0.01 21.44 0.32 99 0.15 Cog. Task No-VR and 34 4.98% (O) ≤0.01 15.79 0.33 99 0.10 No-Cog. Task

Table 4.2: Results of the R Code for Various Experiment Conditions (A Bolded P-value indicates Signiﬁcant Effect)

Figure-4.2 illustrates the actual time estimate means, as well as perceived time estimate means for all the experiment conditions. The Error in the graph is the difference between actual and perceived mean values. A negative error represents an underestimation, while a positive error is an indicator of overestimation. Table-4.3 and Table-4.4 indicate the demographic questionnaire results about age and gender distribution of the participants. The non-uniform bucketing of age brackets in Table 4.3 is due to the non-linear effect of aging on time perception, based on [8].

Demographic Age Age Age Age Age (19-24) (25-30) (31-40) (41-50) (50+) Portion of Participants (%) 41.18% 47.06% 8.82% 2.94% 0%

Table 4.3: Age Distribution of Participants

38 Again, the four null hypotheses that we intended to test were: 1) There is no significant difference in the perception of time experienced between participants using a VR maze and a non-VR maze. 2) There is no significant difference in the perception of time experienced between participants solving a maze with a cognitive task and a maze with no cognitive task. 3) There is no significant difference in the perception of time experienced between participants using a VR maze with a cognitive task and a VR maze with no cognitive task. 4) There is no significant difference in the perception of time experienced between participants using a non-VR maze with a cognitive task and and a non-VR maze with no cognitive task.

The obtained results provide sufﬁcient evidence to reject hypothesis 1 and 3, while they do not provide sufﬁcient evidence to reject hypothesis 2 and 4.

4.2 Qualitative Data

4.2.1 Demographic Questionnaires None of the participants had heard about the details of the experiment before participating in the experiment. None of the participants were suspected to suffer from cyber-sickness or motion-sickness (four of the participants were not sure whether they suffered from cyber-sickness or motion-sickness). None of the participants were suspected to have claustrophobia (one of the participants was not sure whether they suffered from claustrophobia).

Demographic Female Male Other Portion of Participants (%) 44.12% (15/34) 55.88% (19/34) 0% (0/34)

Table 4.4: Gender Distribution of Participants

39 Figure 4.2: Actual and Perceived Time on Various Experiment Conditions

4.2.2 Post-Experience Questionnaire Data Only time estimates that participants reported for their experience (See Appendix D) were used from the post-experiment questionnaires. The other results from distraction questions were discarded.

4.2.3 Terminal Interview Data After analyzing the terminal interview recorded scripts, it was found that frustration level, VR factors (factors related to VR and its effect on time estimation), cognitive factors (factors related to the effect of undertaking a cognitive task on time estimation), maze design, maneuver system, and engagement level are some of the pinpointed themes. Transcripts of the post experience terminal interviews were subjected to a conventional content analysis (no themes or keywords were predeﬁned; rather, they were generated through the analysis). The transcripts were read and reread a few times until recurring themes were identiﬁed and categorized into groups. The groups were given names using the terminologies used in this thesis. The main

40 themes that were easily identiﬁable were the following: most participants believed that the no-VR version of the experiment was not a satisfying experience and was exhausting. Participant 6 mentioned “Time was really slow and I was bored when I was ﬁnishing the maze using screen”. Another common theme was that some of the participants believed that the speed their avatar character was moving was slow and made them overestimate the passage of time. As participant 19 mentioned,“also movement speed very much affects your sense of how time passes because if you move faster time seems to be faster and if you are moving slower time seems slower”.

The last observation, which was clearly reﬂected in the results, was that those participants who thought the experiment was more fun and more interactive underestimated the passage of time. Participant 13 indicates “There were the signs or the pictures on the wall and the fact that when you got close to them there was a voice or history it was interactive. It made it a little bit more fun and interactive, and also distractive, which I think might contribute to perception of time.” This relation between how fun and interactive the experiment was and the perceived passage of time needs further analysis, which was not the focus of this thesis and can be a starting point for any future work of this research.

41 Chapter 5

Discussion

The experiment revealed a significant (16.1% underestimation, p ≤0.01, DF = 99, STD. Error = 0.09) underestimation of passage of time in the experiment runs conducted in VR compared to the control. Moreover, the effect of cognitive spatial tasks on time underestimation was not within the significance boundary (6.45% underestimation, p = 0.36, DF = 99, STD. Error = 0.08) and was below the 0.05 significance level. The most significant underestimation occurred when the experiment was in VR and when a cognitive spatial task was present (22.18% underestimation, p ≤0.01, DF = 99, STD. Error = 0.15). This reveals that being immersed in an experience results in a significant shortening of perceived duration and the effect is amplified when coupled with undertaking a cognitive spatial task. More- over, our findings illustrate that immersion within VR is a more effective method for shortening perceived duration in virtual environments.

Maze Design There was no signiﬁcant maze effect observed in the results. None of the four designed mazes (Red, Green, Yellow, White) had any signiﬁcant effect on the achieved results by participants. Moreover, the similar mean values for all VR and all non-VR mazes show that VR didn’t effect the mean time taken to solve a maze, and is a sign of good design as it makes the achieved statistics of both cases very comparable. Figure-5.1 shows the achieved mean time to solve the mazes in all four mazes.

42 Figure 5.1: Actual, Perceived and Mean Error for all Four Mazes

It is worth mentioning that the effect of cognitive tasks on perception of time has been found significant in other studies (Schatzschneider et. al., 2016). This may be due to relatively small sample sizes of 34 or less, or the pleasing experience of simple screen based non-immersive versions of the experiment. It is worth mentioning that cognitive tasks had a much more significant effect on time underestimation when the experiment was conducted in VR. In fact, the underestimation becomes significant when the cognitive task is performed while being immersed in a VR experience (22.18% underestimation, p ≤0.01, DF = 99, STD. Error = 0.15), which is the most significant achieved underestimation. Finally, an interesting observation was that adding a cognitive spatial task to the experience significantly increases time estimation errors (both underestimation and overestimation). This shows that undertaking the task of navigating mazes significantly affected participants’ time estimation abilities, but our findings do not support that the task only resulted in an underestimation of passage of time. The most accurate time esti-

43 mates were related to the trials in which there was no cognitive task present and where participants could follow the guiding lines.

Effects of VR and Cognitive Spatial Task The combination of immersion in VR and performing a cognitive spatial task resulted in the largest underestimation of time. The effect of immersion on underestimation was the most frequent theme of the terminal interviews. 71% of participants indicated that immersion affected their temporal perception and resulted in an underestimation of time, while 6% of participants believed that immersion resulted in an overestimation of time. Our findings show that solving the mazes on a 2D screen resulted in a 7.5% overestimation of time (p ≤0.01, DF = 99, STD. Error = 0.07). Second to being immersed in a virtual experience, participants believed that the difficulty level of the maze was affected temporal perception. Indeed, difficulty level of the maze required a higher cognitive load level, and more than 56% of the participants indicated that they underestimated time in the mazes that did not have guidelines. However, our results from the post-experience questionnaires reveal that how “fun” and “difficult” the mazes were for participants had no significant effect on their judgment of passage of time and did not result in any significant underestimation of time.

Interactivity Another interesting finding from the questionnaires is that those participants who found the mazes more interactive, (based on their mean interactivity scores from the questionnaires) significantly underestimated the passage of time compared to those who did not find the mazes interactive (this was the initial finding from go- ing over the data briefly, but further investigation is necessary before making any strong claims). This also shows that interactivity, which can also result in further immersion, can be an effective way of shortening the duration of an experience. Further statistical analysis was not done on this matter as this was not the focus of this thesis.

44 Demographic Factors Participants’ gender was found not to be significant regarding time estimation, however, participants’ age was found to be significant. Among the age brackets defined on our demographic questionnaire, it was found that participants of age 19 to 30 significantly underestimated the passage of time, more than those who were in the 31 to 40 and 41 to 50 brackets. We suspect this may have been caused by how much participants were familiar with VR technologies and how experienced they were with playing video games. No further analysis was done on the matter of age and VR technology familiarity. This is a good area to begin further investigations with regard to this study.

Effect of Frustration The frustration level (a term that was assigned to a group of very similar comments given by participants during the interviews) of the experience was believed to affect the temporal perception with third most significance. Generally, this term was assigned to those who either directly used the word ”frustrated”, or gave feedback about their frustration created by movement difficulties on the 2D screen and the speed of the player avatar. Around 21% of the participants believed that the more frustrating mazes resulted in overestimation of time. Another interesting point related to frustration level was that 9% of the participants believed that the movement speed was too slow for the player avatar, which resulted in their overestimation of time. One factor that might have contributed to frustration levels was the implementation of the 2D controller. The challenge in this implementation was that we did not want to add a new variable to our user study by implementing a separate method of moving and interacting specifically for the mazes that were navigated on a 2D screen. Therefore, we utilized the HTC Vive controller’s touchpads to control the avatar both in VR and in 2D. However, those touchpads have been designed for VR movement and interactions, and might not have the best user experience for 2D movements on a PC compared to a more traditional controller that has been designed for 2D screens specifically. The finalized design decision was to not add the new variable due to the scope of the research. For any further work on this

45 research it seems essential to evaluate the effect of controllers on the mazes solved on a 2D traditional screen. Other factors were also discussed during the interviews, such as the background audio, how fun the mazes were, learning about the pictures, or even getting lost in the beautiful sky. Again, due to the focus of this thesis, no further statistical analysis was done on these factors. Further investigation is necessary before making any claims with regard to the effect of frustration on time perception.

Form of the Cognitive Task It is also necessary to pick a cognitive task that suits the purpose of the virtual environment. Tasks that involve substantial and significant spatial cognition in their performance are (ordered based on cognition level): 1) Way-finding as part of navigation, 2) Acquiring and using spatial knowledge from direct experience, 3) Using spatially iconic symbolic representations, 4) Using spatial language, 5) Imagining places and reasoning with mental models, 6) Location allocation [25]. Way-finding was chosen for the maze solving task, as navigation involves one of the highest levels of spatial cognition and hence, was a suitable task for this research. However, some of the participants lost their sense of navigation in the VR mazes and used a brute force approach to solving the mazes. This reduced the cognitive load of the experiment. In a way, the cognitive task that was picked (maze navigation), although generally a very engaging cognitive spatial task, in the VR domain it may result in losing sense of direction and contribute to less cognitive load. Other spatial tasks, such as playing an real time, action shooter video game might be a more suitable task for determining underestimation in VR. It is important to note that due to the way we have defined percentage error, 100% overestimation can be achieved if a participant’s perceived duration doubles the actual one, while 100% underestimation occurs only if the participant believed zero seconds had passed during the experiment, which is not possible. Moreover, there is no upper bound on overestimation, while the upper bound for underestimation is 100%. This was one of our reasons for using absolute errors in figure-4.1 to check error distribution rather than a percentage error. The absolute error distribution is normal and is the reason why we chose the linear mixed effects model

46 to analyze our gathered data. Figure 5.2 shows the distribution of perceived time over actual time ratio. In this graph a ratio of one indicates an accurate, perfect estimation of time. Ratios smaller than one indicate underestimation of time and ratios larger than one indicate overestimation of time.

5.1 Limitations There are a few limitations in this experiment that need to be addressed. The sample size was initially calculated to be 28 by a UBC-based sample size calculator and 27 by another online power calculator, however, we recruited 34 participants. This is still a very small sample size for a multiple intervention study. With such small samples the risk that each observation could have a considerable impact on the results is large. Although the observed effect size matches the expected initial effect size, recruiting more participants can be a point of improvement for any further investigations on this study. Another limitation was the sampling bias. Although the ordered sequence of mazes in the experiment followed a Latin Square style to avoid any possible sequencing effects, the convenience sample of participants were self-selecting and the majority had an interest in VR technologies and applications, which can be considered a sampling bias. This may affect the reliability and generalizability of the ﬁndings. Including randomly chosen participants can potentially increase the validity of the study. In terms of technical limitations, participants used the HTC Vive headset which is wired to a computer. Occasionally participants became tangled with the wires, which may have affected their sense of immersion. At the beginning of each trial the experimenter informed the participants of the risk of the wire and instructed them how to pass their feet above the wire while rotating to avoid such situations. A good solution for any future work on this research is to use a hanger from the ceiling to pass the wire through, so that the headset is connected from above, avoid- ing the tangling issue. Moreover, the physical boundaries of the room and the participants’ worries about collisions may have resulted in a lower sense of immersion, which may have affected our results. One of the reasons that we picked HTC Vive as our head

47 Figure 5.2: Histogram of Perceived Over Actual Ratio Distribution mounted display was the fact that Vive tracks the borders and boundaries of the room and any furniture inside it, and plots these boundaries as light blue dotted lines displayed to the user. However, it was obvious from participant body language that they were still concerned about colliding with the walls.

48 Chapter 6

Conclusion

In this thesis, the effects of being immersed in a virtual environment and undertaking a cognitive spatial task on time perception were explored. An experiment was designed and presented in which participants perception of time under the influence of immersion and cognitive load was analyzed. The results achieved indicate that immersion within a VR experience can significantly affect time perception and results in a significant underestimation of the passage of time. Moreover, our results show that performing a spatial cognitive task (maze navigation) also leads to an underestimation of time, but the impact is not significant. The highest underestimation occurred when both immersion within a VR experience and a cognitive task were present. On the other hand, the largest overestimation of time occurred when immersion within VR was not present but cognitive task was present. To address our main research questions explicitly, it was found that immersion within a virtual experience results in a significant underestimation of time (16.10% underestimation). When combined with undertaking a cognitive spatial task, it resulted in a 22.18% underestimation of time among 136 observations from 34 participants. Undertaking a cognitive spatial task did not result in significant underestimation of time on its own, however, it did result in significant underestimation of time when performed in a VR experience. In order to build VR experiences that maximize underestimation of time, the combination of increased immersion with cognitive spatial tasks is effective. It

49 is also possible to benefit from factors such as Zeitgebers [29], which are previously shown to be effective (table-E.1 shows effective factors from other published studies). Our qualitative results also indicate that a more interactive and engaging interface can also contribute to the effectiveness of the environment and can result in significant underestimations of time by participants. To summarize the contributions of this thesis: 1) It was found that being immersed within a virtual experience will significantly affect perception of time and can lead to an underestimation of the passage of time. 2) It was found that undertaking a cognitive spatial task did not result in significant underestimation of time on its own. 3) The most underestimation occurred when both immersion within a VR experience and a cognitive spatial task were present. 4) Although undertaking a cognitive spatial task did not result in significant underestimation of time on its own, when the task was performed in a VR environment, it significantly affected users perception of time and resulted in underestimation of time. 5) It was found that factors such as frustration level can contribute to overestimation of time by users. Moreover, participants’ age also had a significant effect on time estimations. A maze’s interactivity level also affected participant time estimations and contributed to underestimations of time. 6) It was found that participant gender did not have any significant effect on time estimations. It was also found that how ”fun” or ”difficult” the navigation experience was also did not result in any significant effect.

6.1 Implications and Future Work The relationship between how fun and interactive the experiment was and the perceived passage of time needs further analysis, as this was not the focus of this thesis and can be a starting point for future work. Adding other factors from the metadata, such as tech-savviness, into the linear mixed effects model and determining whether such parameters can also have a sig- niﬁcant effect on time perception could also be another interesting area for further

50 investigations. Plotting the gathered data using other visualization techniques and methods might also give us new insights. A good example of this for any further investigation on this research is to plot data in a way that possible clusters form. If any cluster existed, further investigation on the cause would be essential. These possible clusters can give us further insight on other factors in our metadata, such as tech-savviness or age, and can perhaps explain some of the achieved results in more depth. As previously stated in the discussion chapter, choosing another cognitive spatial task can also be an interesting area of exploration for further work. To conclude, the combination of immersion within VR and performing a cognitive spatial task seems like a very practical, and yet non-expensive, solution for many virtual environments in which underestimation of time has value.

51 Bibliography

[1] L. G. Allan. The perception of time. Perception & Psychophysics, 26(5): 340–354, 1979. → page 11

[2] H. K. Andersen and R. Grush. A brief history of time-consciousness: historical precursors to james and husserl. Journal of the History of Philosophy, 47(2):277–307, 2009. → page 7

[3] R. A. Block and D. Zakay. Prospective and retrospective duration judgments: A meta-analytic review. Psychonomic bulletin & review, 4(2): 184–197, 1997. → page 13

[4] S. W. Brown. Time, change, and motion: The effects of stimulus movement on temporal perception. Perception & Psychophysics, 57:105–116, 1995. → page 14

[5] G. Bruder and F. Steinicke. Threefolded motion perception during immersive walkthroughs. In Proceedings of the 20th ACM symposium on virtual reality software and technology, pages 177–185. ACM, 2014. → page 8

[6] C. Constantinidis, G. V. Williams, and P. S. Goldman-Rakic. A role for inhibition in shaping the temporal ﬂow of information in prefrontal cortex. Nature neuroscience, 5(2):175, 2002. → page 13

[7] D. B. Davalos, M. A. Kisley, and R. G. Ross. Deﬁcits in auditory and visual temporal perception in schizophrenia. Cognitive Neuropsychiatry, 7(4): 273–282, 2002. → page 9

[8] J.-C. Dreher, A. Meyer-Lindenberg, P. Kohn, and K. F. Berman. Age-related changes in midbrain dopaminergic regulation of the human reward system. Proceedings of the National Academy of Sciences, 105(39):15106–15111, 2008. → pages 10, 38

52 [9] S. Eden. The effect of 3d virtual reality on sequential time perception among deaf and hard-of-hearing children. European Journal of Special Needs Education, 23(4):349–363, 2008. → page 13 [10] J. Fox. Applied regression analysis and generalized linear models. Sage Publications, 2015. → pages 31, 32 [11] B. Garrett, T. Taverner, W. Masinde, D. Gromala, C. Shaw, and M. Negraeff. A rapid evidence assessment of immersive virtual reality as an adjunct therapy in acute pain management in clinical practice. The Clinical journal of pain, 30(12):1089–1098, 2014. → pages 8, 9 [12] D. Goldreich. A bayesian perceptual model replicates the cutaneous rabbit and other tactile spatiotemporal illusions. PloS one, 2(3):e333, 2007. → page 11 [13] P. Gongsook. Time simulator in virtual reality for children with attention deficit hyperactivity disorder. In International Conference on Entertainment Computing, pages 490–493. Springer, 2012. → pages 16, 18 [14]K.H agni,¨ K. Eng, M.-C. Hepp-Reymond, L. Holper, B. Keisker, E. Siekierka, and D. C. Kiper. The effect of task and ownership on time estimation in virtual environments. Barcelona, Spain, 2007. → pages 2, 14, 15 [15] J. Heron, C. Aaen-Stockdale, J. Hotchkiss, N. W. Roach, P. V. McGraw, and D. Whitaker. Duration channels mediate human time perception. In Proc. R. Soc. B, volume 279, pages 690–698. The Royal Society, 2012. → page 7 [16] R. E. Hicks, G. W. Miller, and M. Kinsbourne. Prospective and retrospective judgments of time as a function of amount of information processed. The American journal of psychology, pages 719–730, 1976. → page 1 [17] I. Hodinott-Hill, K. V. Thilo, A. Cowey, and V. Walsh. Auditory chronostasis: hanging on the telephone. Current Biology, 12(20): 1779–1781, 2002. → page 12 [18] B. Jones and Y. L. Huang. Space-time dependencies in psychophysical judgment of extent and duration: Algebraic models of the tau and kappa effects. Psychological bulletin, 91(1):128, 1982. → page 11 [19] J. J. Kellaris and R. J. Kent. The influence of music on consumers’ temporal perceptions: does time fly when you’re having fun? Journal of consumer psychology, 1(4):365–376, 1992. → page 10

53 [20] G. Koch, M. Oliveri, S. Torriero, and C. Caltagirone. Underestimation of time perception after repetitive transcranial magnetic stimulation. Neurology, 60(11):1844–1846, 2003. → page 10

[21] D. E. Kornbrot, R. M. Msetﬁ, and M. J. Grimwood. Time perception and depressive realism: judgment type, psychophysical functions and bias. PLOS one, 8(8):e71585, 2013. → page 11

[22] P. A. Lewis and R. C. Miall. Distinct systems for automatic and cognitively controlled time measurement: evidence from neuroimaging. Current opinion in neurobiology, 13(2):250–255, 2003. → page 14

[23] W. J. Matthews, N. Stewart, and J. H. Wearden. Stimulus intensity and the perception of duration. Journal of Experimental Psychology: Human perception and performance, 37(1):303, 2011. → page 14

[24] G. K. McMillan, D. M. Considine, et al. Process/industrial instruments and controls handbook, volume 7. McGraw Hill, 1999. → page 7

[25] D. R. Montello and M. Raubal. Functions and applications of spatial cogniticdn˜. 2013. → page 46

[26] S. Neufang, G. R. Fink, B. Herpertz-Dahlmann, K. Willmes, and K. Konrad. Developmental changes in neural activation and psychophysiological interaction patterns of brain regions associated with interference control and time perception. Neuroimage, 43(2):399–409, 2008. → page 14

[27] T. A. Niendam, A. R. Laird, K. L. Ray, Y. M. Dean, D. C. Glahn, and C. S. Carter. Meta-analytic evidence for a superordinate cognitive control network subserving diverse executive functions. Cognitive, Affective, & Behavioral Neuroscience, 12(2):241–268, 2012. → page 14

[28] R. Nijhawan and B. Khurana. Space and time in perception and action. Cambridge University Press, 2010. → page 12

[29] A. C. Nobre and J. O’Reilly. Time is of the essence. Trends in cognitive sciences, 8(9):387–389, 2004. → page 14

[30] J. Radua, N. O. Del Pozo, J. Gomez,´ F. Guillen-Grima, and F. Ortuno.˜ Meta-analysis of functional neuroimaging studies indicates that an increase of cognitive difﬁculty during executive tasks engages brain regions associated with time perception. Neuropsychologia, 58:14–22, 2014. → page 13

54 [31] S. M. Rao, A. R. Mayer, and D. L. Harrington. The evolution of brain activation during temporal processing. Nature neuroscience, 4(3):317, 2001. → page 7 [32] D. Rose and J. Summers. Duration illusions in a train of visual stimuli. Perception, 24(10):1177–1187, 1995. → page 12 [33] M. Rudd, K. D. Vohs, and J. Aaker. Awe expands peoples perception of time, alters decision making, and enhances well-being. Psychological science, 23(10):1130–1136, 2012. → page 10 [34] T. Sanders and P. Cairns. Time perception, immersion and music in videogames. In Proceedings of the 24th BCS interaction specialist group conference, pages 160–167. British Computer Society, 2010. → page 2 [35] C. Schatzschneider, G. Bruder, and F. Steinicke. Who turned the clock? effects of manipulated zeitgebers, cognitive load and immersion on time estimation. IEEE transactions on visualization and computer graphics, 22 (4):1387–1395, 2016. → pages 12, 15, 18, 32, 33 [36] S. M. Schneider, C. K. Kisby, and E. P. Flint. Effect of virtual reality on time perception in patients receiving chemotherapy. Supportive Care in Cancer, 19(4):555–564, 2011. → pages 15, 18 [37] J. T. Shelton, E. M. Elliott, B. Hill, M. R. Calamia, and W. D. Gouvier. A comparison of laboratory and clinical working memory tests and their prediction of ﬂuid intelligence. Intelligence, 37(3):283–293, 2009. → page 13 [38] C. Stetson, M. P. Fiesta, and D. M. Eagleman. Does time really slow down during a frightening event? PloS one, 2(12):e1295, 2007. → page 11 [39] M. Treisman. Temporal discrimination and the indifference interval: Implications for a model of the” internal clock”. Psychological Monographs: General and Applied, 77(13):1, 1963. → page 1 [40] M. Treisman, A. Faulkner, P. L. Naish, and D. Brogan. The internal clock: Evidence for a temporal oscillator underlying time perception with some estimates of its characteristic frequency. Perception, 19(6):705–742, 1990. → page 15 [41] P. U. Tse, J. Intriligator, J. Rivest, and P. Cavanagh. Attention and the subjective expansion of time. Perception & psychophysics, 66(7): 1171–1189, 2004. → page 12

55 [42] V. Van Wassenhove, M. Wittmann, A. Craig, and M. P. Paulus. Psychological and neural mechanisms of subjective time dilation. Frontiers in neuroscience, 5:56, 2011. → page 10

[43] Y. Wada, T. Masuda, and K. Noguchi. Temporal illusion called’kappa effect’in event perception. Perception ECVP abstract, 34:0–0, 2005. → page 11

[44] J. H. Wearden and I. S. Penton-Voak. Feeling the heat: Body temperature and the rate of subjective time, revisited. The Quarterly Journal of Experimental Psychology, 48(2):129–141, 1995. → page 14

[45] M. Wittmann, O. Carter, F. Hasler, B. R. Cahn, U. Grimberg, P. Spring, D. Hell, H. Flohr, and F. X. Vollenweider. Effects of psilocybin on time perception and temporal control of behaviour in humans. Journal of Psychopharmacology, 21(1):50–64, 2007. → page 9

[46] M. Wittmann, D. S. Leland, J. Churan, and M. P. Paulus. Impaired time perception and motor timing in stimulant-dependent subjects. Drug & Alcohol Dependence, 90(2):183–192, 2007. → page 9

[47] K. Yarrow, P. Haggard, R. Heal, P. Brown, and J. C. Rothwell. Illusory perceptions of space and time preserve cross-saccadic perceptual continuity. Nature, 414(6861):302, 2001. → pages 11, 12

56 Appendix A

This appendix shows the R code used in order to analyze the obtained data. This is the code for the Linear Mixed Effects Model

R Code: install.packages(nlme) library(nlme)

dat < −read.csv(C : /Users/Derek/Desktop/data f ile.csv,header = T) fit1 < −lme(Di f f erence VR + Solution,random = 1|ID,data = dat) summary(fit1) summary(fit1) fit2 < −lme(Di f f erence VR ∗ Solution,random = 1|ID,data = dat) summary(fit2)

57 Appendix B

The next two pages (Figure B.1 and B.2) show the top view of all the four main mazes as well as participant’s view of the tutorial maze.

Figure B.1 shows two mazes with green guidelines and two mazes without the guidelines. In this picture it is obvious that all four mazes have identical sizes (and also identical difﬁculties).

Figure B.2 gives a closer look at the design of the maze environment and shows how exactly participants perceived the maze environments.

58 Figure B.1: Top View of All 4 Identical Main Mazes (2 with guiding lines and pictures, 2 without lines and pictures)

59 Figure B.2: Players View of a Maze

60 Appendix C

Figures D.1, D.2, and D.3 show the Demographic Questionnaire, the Post-Exposure VR Experience Questionnaire, and the Post Experience Semi-structured Interview Questions that participants received during the user study.

Each participant completed: 1) The Demographic Questionnaire at the beginning of the user study (1 time)

2) The Post-Exposure VR Experience Questionnaire at the end of each of the VR mazes (4 times)

3) The Post Experience Semi-structured Interview Questions at the end of the user study (1 time)

61 Figure C.1: Demographic Questionnaire

62 Figure C.2: Post Exposure VR Experience Questionnaire

63 Figure C.3: Post Exposure Semi-Structured Interview Questions

64 Appendix D

Table E.1 (next page) shows identiﬁed factors that can possibly affect one’s perception of time. The effects have been divided into two columns (Overestimation and Underestimation).

65 Overestimation Underestimation Psychoactive Drugs Psychoactive Drugs Music (Positively Valenced) Music (Negatively Valenced, Casino Ambient Sound) Emotional States (Awe, Fear, ) Clinical Disorders Telescoping Effect Aging Vierordts Law Vierordts Law Kappa Effect Kappa Effect Chronostasis Cognitive Tasks (especially Spatial cog. Tasks) The Oddball Effect Repetitive Transcranial Magnetic Stimulation High Stimulus Intensity Low Stimulus Intensity High Change Rates in Scenes Low Change Rates in Scenes Body Temperature Body Temperature Interrupted Scenes Uninterrupted Scenes Visual Time Symbols (Zeitgebers) Visual Time Symbols (Zeitgebers) Performing temporal tasks while Performing non-temporal tasks/ in immersive environments playing games while experiencing immersion High Frequency Auditory Time Cues Low Frequency Auditory Time Cues

Table D.1: Factors That Effect Perception of Time

66 Appendix E

Table F.1 (next 5 pages) shows all the recorded data. The columns show the participant ID, whether the experience was VR or non-VR, whether it had a cognitive spatial task or not, which maze it was, the perceived time (from the Post-experience questionnaire), the logged actual time taken to solve each maze, the ratio of perceived time over actual time, and ﬁnally the underestimation/overestimation percentage.

67 User Study Data Participant VR No-Task Maze Perceived Actual Ratio Diff/ ID Time Time Actual 1 1 1 R 180 351 0.51 49%(U) 1 1 0 G 300 572 0.52 48%(U) 1 0 1 W 180 383 0.47 43%(U) 1 0 0 Y 240 415 0.58 42%(U) 2 1 1 R 360 327 1.10 -10%(O) 2 1 0 Y 510 532 0.96 4%(U) 2 0 1 W 210 201 1.04 4%(O) 2 0 0 G 300 247 1.21 21%(O) 3 1 1 W 180 141 1.28 28%(O) 3 1 0 Y 420 826 0.51 49%(U) 3 0 1 R 120 125 0.96 4%(U) 3 0 0 G 180 208 0.87 13%(U) 4 1 1 W 300 300 1.47 47%(O) 4 1 0 G 420 216 1.94 94%(O) 4 0 1 R 900 475 1.89 89%(O) 4 0 0 Y 600 362 1.66 66%(O) 5 1 0 Y 180 162 1.11 11%(O) 5 1 1 R 180 137 1.31 31%(O) 5 0 0 G 420 469 0.90 10%(U) 5 0 1 W 240 176 1.36 36%(O) 6 1 0 Y 135 317 0.43 57%(U) 6 1 1 W 60 156 0.38 62%(U) 6 0 0 G 120 239 0.50 50%(U) 6 0 1 R 90 112 0.80 20%(U) 7 1 0 G 600 321 1.87 87%(O) 7 1 1 R 240 149 1.61 61%(O) 7 0 0 Y 240 159 1.51 51%(O) 7 0 1 W 240 111 1.08 8%(O) 8 1 0 G 300 322 0.93 7%(U) 8 1 1 W 45 172 0.26 74%(U) 8 0 0 Y 30 189 0.15 85%(U) 8 0 1 R 45 124 0.36 64%(U)

68 Participant VR No-Task Maze Perceived Actual Ratio Diff/ ID Time Time Actual 9 0 1 R 180 240 0.75 25%(U) 9 0 0 Y 300 344 0.87 13%(U) 9 1 1 W 180 189 0.95 5%(U) 9 1 0 G 420 408 1.02 2%(O) 10 0 1 R 90 171 0.53 47%(U) 10 0 0 G 150 217 0.69 31%(U) 10 1 1 W 120 344 0.35 65%(U) 10 1 0 Y 150 324 0.46 54%(U) 11 0 1 W 240 154 1.56 56%(O) 11 0 0 Y 600 287 2.09 109%(O) 11 1 1 R 180 121 1.49 49%(O) 11 1 0 G 300 337 0.89 11%(U) 12 0 1 W 180 229 0.79 21%(U) 12 0 0 G 360 327 1.10 10%(O) 12 1 1 R 240 202 1.19 19%(O) 12 1 0 Y 180 191 0.94 6%(U) 13 0 0 Y 225 416 0.54 46%(U) 13 0 1 R 230 380 0.60 40%(U) 13 1 0 G 130 316 0.41 59%(U) 13 1 1 W 105 179 0.59 41%(U) 14 0 0 Y 240 204 1.18 18%(U) 14 0 1 W 240 344 0.70 30%(U) 14 1 0 G 360 291 1.24 24%(O) 14 1 1 R 420 379 1.10 10%(O) 15 0 0 G 300 495 0.61 39%(U) 15 0 1 R 240 391 0.61 39%(U) 15 1 0 Y 180 200 0.90 10%(U) 15 1 1 W 240 344 0.70 30%(U) 16 0 0 G 193 441 0.44 56%(U) 16 0 1 W 120 377 0.32 68%(U) 16 1 0 Y 90 285 0.32 68%(U) 16 1 1 R 210 517 0.41 59%(O)

69 Participant VR No-Task Maze Perceived Actual Ratio Diff/ ID Time Time Actual 17 1 1 R 420 289 1.45 45%(O) 17 1 0 Y 600 838 0.72 28%(U) 17 0 1 W 230 231 0.99 1%(U) 17 0 0 G 340 383 1.89 11%(O) 18 1 1 R 480 337 1.42 42%(O) 18 1 0 G 300 342 0.88 12%(U) 18 0 1 W 300 372 0.81 19%(U) 18 0 0 Y 180 355 0.51 49%(U) 19 1 1 W 300 341 0.88 12%(U) 19 1 0 Y 270 315 0.86 14%(U) 19 0 1 R 450 410 1.09 9%(O) 19 0 0 G 780 850 0.92 8%(U) 20 1 1 W 210 366 0.57 43%(U) 20 1 0 G 110 230 0.48 52%(U) 20 0 1 R 390 554 0.70 30%(U) 20 0 0 Y 405 231 1.75 75%(O) 21 1 0 Y 300 166 1.81 81%(O) 21 1 1 R 480 207 2.32 32%(O) 21 0 0 G 900 365 2.47 47%(O) 21 0 1 W 600 162 3.70 270%(O) 22 1 0 Y 87 194 0.45 55%(U) 22 1 1 W 135 173 0.78 22%(U) 22 0 0 G 300 276 1.09 9%(O) 22 0 1 R 195 159 1.22 22%(O) 23 1 0 G 165 322 0.51 49%(U) 23 1 1 R 345 437 0.79 21%(U) 23 0 0 Y 600 592 1.01 1%(O) 23 0 1 W 420 484 0.87 23%(U) 24 1 0 G 420 365 1.15 15%(O) 24 1 1 W 600 372 1.61 61%(O) 24 0 0 Y 300 149 2.01 101%(O) 24 0 1 R 600 456 1.32 32%(O)

70 Participant VR No-Task Maze Perceived Actual Ratio Diff/ ID Time Time Actual 25 0 1 R 390 338 1.15 15%(O) 25 0 0 Y 330 232 1.42 42%(O) 25 1 1 W 205 376 0.54 56%(U) 25 1 0 G 180 389 0.46 54%(U) 26 0 1 R 265 165 1.60 60%(O) 26 0 0 G 340 245 1.39 39%(O) 26 1 1 W 95 135 0.70 30%(U) 26 1 0 Y 130 363 0.36 64%(U) 27 0 1 W 240 167 1.43 43%(O) 27 0 0 Y 360 228 1.58 58%(U) 27 1 1 R 120 141 0.85 15%(U) 27 1 0 G 135 214 0.63 37%(U) 28 0 1 W 270 189 1.43 43%(O) 28 0 0 G 340 266 1.28 28%(O) 28 1 1 R 90 132 0.68 32%(U) 28 1 0 Y 130 203 0.64 36%(U) 29 0 0 Y 460 326 1.41 41%(O) 29 0 1 R 265 146 1.82 82%(O) 29 1 0 G 185 382 0.48 52%(U) 29 1 1 W 150 246 0.61 39%(U) 30 0 0 Y 300 244 1.23 23%(O) 30 0 1 W 420 313 1.34 34%(O) 30 1 0 G 480 355 1.35 35%(O) 30 1 1 R 480 398 1.21 21%(O) 31 0 0 G 210 274 0.77 23%(U) 31 0 1 R 150 246 0.61 39%(U) 31 1 0 Y 135 226 0.60 40%(U) 31 1 1 W 150 236 0.64 36%(U) 32 0 0 G 660 329 2.00 100%(O) 32 0 1 W 480 382 1.26 26%(O) 32 1 0 Y 210 265 0.80 20%(U) 32 1 1 R 240 350 0.69 31%(U)

71 Participant VR No-Task Maze Perceived Actual Ratio Diff/ ID Time Time Actual 33 1 1 R 220 281 0.78 22%(U) 33 1 0 Y 210 301 0.70 30%(U) 33 0 1 W 420 331 1.27 27%(O) 33 0 0 G 525 475 1.10 10%(O) 34 1 1 R 240 288 0.83 17%(U) 34 1 0 G 180 349 0.52 48%(U) 34 0 1 W 332 277 1.20 20%(O) 34 0 0 Y 450 363 1.24 24%(O)

Table E.1: User Study Data