Virtual and Mixed Reality Support for Activities of Daily Living Item Type Thesis or dissertation Authors Day, Thomas W. Citation Day, T.W. (2019). Virtual and Mixed Reality Support for Activities of Daily Living. (Doctoral dissertation). University of Chester, United Kingdom. Publisher University of Chester Rights Attribution-NonCommercial-NoDerivatives 4.0 International Download date 07/10/2021 01:06:17 Item License http://creativecommons.org/licenses/by-nc-nd/4.0/ Link to Item http://hdl.handle.net/10034/622175 School of Computer Science Faculty of Science & Engineering Virtual and Mixed Reality Support for Activities of Daily Living Thesis submitted in accordance with the requirements of the University of Chester for the Degree of Doctor of Philosophy by Thomas W. Day Supervisor Prof. N. W. John 28th March 2019 Statement of Originality The material being presented for examination is my own work and has not been submitted for an award of this or another HEI except in minor particulars which are explicitly noted in the body of the thesis. Where research pertaining to the thesis was undertaken collaboratively, the nature and extent of my individual contribution has been made explicit. Student: Thomas W. Day Abstract Virtual and Mixed Reality Support for Activities of Daily Living Thomas W. Day Rehabilitation and training are extremely important process that help people who have suffered some form of trauma to regain their ability to live independently and successfully complete activities of daily living. VR and MR have been used in rehabilitation and training, with examples in a range of areas such as physical and cognitive rehabilitation, and medical training. However, previous research has mainly used non-immersive VR such as using video games on a computer monitor or television. Immersive VR Head-Mounted Displays were first developed in 1965 but the devices were usually large, bulky and expensive. In 2016, the release of low-cost VR HMDs allowed for wider adoption of VR technology. This thesis investigates the impact of these devices in supporting activities of daily living through three novel applications: training driving skills for a powered wheelchair in both VR and MR; and using VR to help with the cognitive rehabilitation of stroke patients. Results from the acceptability study for VR in cognitive rehabilitation showed that patients would be likely to accept VR as a method of rehabilitation. However, factors such as visual issues need to be taken into consideration. The validation study for the Wheelchair-VR project showed promising results in terms of user improvement after the VR training session but the majority of the users experienced symptoms of cybersickness. Wheelchair-MR didn’t show statistically significant results in terms of improvements but did show a mean average improvement compared to the control group. The effects of cybersickness were also greatly reduced compared to VR. We conclude that VR and MR can be used in conjunction with modern games engines to develop virtual environments that can be adapted to accelerate the rehabilitation and training of patients coping with different aspects of daily life. Acknowledgments I would like to thank the many people who have made this thesis possible and those who have made the past three years a very enjoyable and rewarding experience. In particular, I would like to thank my Supervisor, Prof. Nigel John, for allowing me to complete my Ph.D at the University of Chester and providing me with the resources and assistance I needed to carry out my research. I would also like to thank Christopher Headleand, Serban Pop and Panagiotis Ritsos for helping me throughout these last three years. Many thanks to Dr Kausik Chatterjee, Katy Cottrell and the therapy team at the Countess of Chester Hospital NHS Trust for providing me with the feedback and resources that I needed to help me design and run the StrokeVR project. I would like to thank Liverpool Wheelchair Service, Wirral Community NHS Trust Wheelchair Service, Cardiff and Vale UHB Posture Mobility Centre, and Betsi Cadwaladr UHB Posture and Mobility Service for providing help and assistance during the Wheelchair-VR project and Invacare UK for the loan of the Spectra wheelchair. I would like to thank my family and friends for their support over the past three years, especially my grandfather John Page for being a massive inspiration to me, my partner Liz and our cat Murphy who have kept me sane and helped me throughout, and my colleagues Peter and Mark for the office banter. Contents 1 Introduction 1 1.1 Hypothesis . 2 1.2 Contributions . 3 1.3 List of Publications . 3 1.4 Thesis Structure . 4 2 Background 6 2.1 Chapter Overview . 6 2.2 Human Perception . 6 2.2.1 Immersion . 7 2.2.2 Presence . 8 2.3 Virtual Reality . 9 2.3.1 Large-Volume Displays (LVDs) . 10 Monitor-based . 10 Projector-based . 11 Tracking Systems . 13 2.3.2 Head-Mounted Displays (HMDs) . 14 Tethered HMDs . 15 Mobile HMDs . 19 2.3.3 VR Content Development . 21 Desktop VR . 21 Mobile VR . 22 Web-based VR . 22 2.3.4 Current Issues . 23 2.4 Mixed Reality . 24 2.4.1 Headsets . 24 Mobile . 27 2.4.2 XR . 28 2.5 Patient support using Virtual Reality and Mixed Reality . 29 2.5.1 Rehabilitation . 29 Stroke . 30 2.5.2 Training . 34 Medical Training . 36 i Powered Wheelchair Manoeuvres . 38 2.6 Methodology for VR Human Factors Studies . 40 2.6.1 Develop the Focus of the Study . 41 2.6.2 Develop the Experimental Protocol . 41 2.6.3 Subject Recruitment . 41 Sampling Techniques . 41 2.6.4 Data Collection . 42 2.6.5 Data Analysis . 43 2.6.6 Examples of Data Analysis in VR Studies . 45 2.7 Summary . 47 3 Wheelchair-VR 49 3.1 Chapter Overview . 49 3.2 Motivation . 49 3.3 Methods and Tools . 50 3.3.1 Powered Wheelchair Specifications . 50 3.3.2 Hardware Components . 52 3.3.3 Software Implementation . 53 Wheelchair Model . 53 Training Tasks . 55 3.3.4 Testing . 58 3.4 Validation Study . 58 3.4.1 Experimental Design . 60 3.4.2 Results . 64 Simulator Sickness Questionnaire . 68 3.5 Configurations Study . 70 3.5.1 Experimental Design . 71 3.5.2 Results . 72 3.6 Conclusions . 72 4 Wheelchair-MR 75 4.1 Chapter Overview . 75 4.2 Motivation . 75 4.3 Method and Tools . 76 4.3.1 Hardware Components . 77 4.3.2 Software Implementation . 77 User Interface . 77 Spatial Understanding . 78 Chair Simulation . 83 Training Tasks . 85 4.4 Initial User Testing . 86 4.5 Validation Study . 89 ii 4.5.1 Experimental Design . 89 4.5.2 Results . 92 Simulator Sickness Questionnaire . 94 4.6 Conclusions . 98 5 StrokeVR - Stroke Rehabilitation 101 5.1 Chapter Overview . 101 5.2 Motivation . 101 5.3 Initial Implementation . .102 5.3.1 Software Implementation . .102 Tasks . .102 Interactions . .103 5.3.2 Hardware Components . 104 5.4 Initial Experiment . .105 5.4.1 Experimental Design . .105 5.4.2 Feedback . .106 5.5 Revised Implementation . 107 5.5.1 Hardware Components . 107 5.5.2 Software Implementation . 107 Interactions . 111 Visual Feedback . .112 5.6 Acceptability Study . .112 5.6.1 Inclusion Criteria . .113 5.6.2 Experimental Design . 114 Staff Training . 114 Infection Control . .115 5.6.3 Results . .115 5.7 Conclusion . .119 6 Conclusions and Future Work 121 6.1 Conclusions . 121 6.2 Future Work . .125 6.2.1 Wheelchair-VR . .125 6.2.2 Wheelchair-MR . 127 6.2.3 StrokeVR . .128 A References 130 iii iv List of Figures 2.1 Milgram’s definition of the reality-virtuality continuum [77] 9 2.2 The mock-up of a 3D view on the z-Space display from the vCath project [56] . 11 2.3 Fake Space Immersive Workbench N-239 showing skull for reconstructive surgery studies [5] . 12 2.4 The CAVE installation at the University of Keele. Here there are three projection walls but note that CAVES have been built with all four walls, the floor and the ceiling as projection surfaces. 13 2.5 A Powerwall display (left) that uses two polarized Sony 4K projectors (right) with passive stereo glasses . 13 2.6 Desktop-based VR HMDs. 17 2.7 One of the base stations used by the VIVE’s lighthouse system . 18 2.8 Mobile-based VR HMDs. 20 2.9 The Oculus Quest with controllers [80] . 21 2.10 Mixed Reality Headsets . 26 3.1 The Wheelchair-VR Hardware components consist of the PC, an Oculus Rift DK2 HMD, and an XBox controller. Insert shows the adapted joystick on the XBox controller. 52 3.2 Physics based wheelchair model with wheel colliders . 54 3.3 Kinematic Virtual Wheelchair Models . 56 3.4 Wheelchair-VR provides a first person view of the scene from the perspective of sitting in a wheelchair. 56 3.5 Snapshots taken from different rooms in Wheelchair-VR . 57 3.6 The Spectra XTR2 Rear Wheel Drive Wheelchair used in the Validation Study . ..