Bachelor Thesis Implementation of Open Source applications “Serious Game” for Rehabilitation Author Jaime Herreros Díaz Tutor Alberto Jardón Huete October 2016 Implementation of Open Source applications “Serious Game” for Rehabilitation | Jaime Herreros Díaz i Implementation of Open Source applications “Serious Game” for Rehabilitation | Jaime Herreros Díaz TABLE OF CONTENTS TABLE OF CONTENTS ................................................................................. iii INDEX OF FIGURES ..................................................................................... v INDEX OF TABLES ...................................................................................... vii Abstract ................................................................................................... 1 1 Introduction ...................................................................................... 3 1.1 Background ................................................................................. 3 1.2 Motivation ................................................................................... 3 1.3 Objectives .................................................................................... 4 1.4 Structure ..................................................................................... 5 2 Target users ...................................................................................... 6 2.1 Stroke ........................................................................................... 6 2.1.1 Definition and classification ..................................................... 6 2.1.2 Causes ...................................................................................... 7 2.1.3 Risk factors .............................................................................. 8 2.1.4 Treatment ................................................................................ 8 2.1.5 Rehabilitation .......................................................................... 9 2.1.6 Outcome Measures in Stroke Rehabilitation .......................... 10 2.2 Virtual Reality in rehabilitation ........................................... 14 2.2.1 BRUTM (Balance Rehabilitation UnitTM) ................................... 14 2.2.2 BioTrak© ............................................................................... 16 3 Technologies .................................................................................... 17 3.1 Game Engines .......................................................................... 17 3.1.1 Unreal Engine 4 ...................................................................... 18 3.1.2 Unity ...................................................................................... 20 3.1.3 CryEngine ............................................................................... 22 3.2 Creation, modelling and animation of objects .................... 24 3.2.1 Blender .................................................................................. 24 3.2.2 MakeHuman .......................................................................... 25 ii Implementation of Open Source applications “Serious Game” for Rehabilitation | Jaime Herreros Díaz 3.3 Motion Capture devices .......................................................... 27 3.3.1 Kinect camera ........................................................................ 29 3.3.2 Sony PlayStation Eye .............................................................. 32 3.3.3 PrimeSense Sensor ................................................................ 33 4 Materials and Methods ................................................................. 36 4.1 Hardware .................................................................................. 36 4.1.1 Kinect v2 ................................................................................ 36 4.1.2 Computer ............................................................................... 39 4.2 Software ..................................................................................... 40 4.2.1 Unity ...................................................................................... 41 4.2.2 Blender and MakeHuman ...................................................... 48 5 Serious Games design and Results ............................................. 50 5.1 Serious Games features .......................................................... 50 5.2 Serious Games design basis ................................................... 50 5.2.1 First steps .............................................................................. 53 5.2.2 Game environment and avatar .............................................. 54 5.2.3 Acquiring and saving data ...................................................... 57 5.3 Developed Serious Games ...................................................... 58 5.3.1 Menu module ........................................................................ 59 5.3.2 The Reach Game .................................................................... 60 5.3.3 The Balance Game ................................................................. 63 6 Conclusions and Future Work ..................................................... 66 6.1 Conclusions ............................................................................... 66 6.2 Experiences and problems occurred during the project development ....................................................................................... 67 6.3 Future work .............................................................................. 68 References ............................................................................................. 69 Annex ..................................................................................................... 73 A.1 Work Hours Breakdown ......................................................... 73 A.2 Project budget ........................................................................... 74 iii Implementation of Open Source applications “Serious Game” for Rehabilitation | Jaime Herreros Díaz A.2.1 Personnel ................................................................................. 74 A.2.2 Equipment ................................................................................ 74 A.2.4 Estimation of total costs ........................................................... 75 A.3 GetJointPosition.cs .................................................................. 76 iv Implementation of Open Source applications “Serious Game” for Rehabilitation | Jaime Herreros Díaz INDEX OF FIGURES Figure 2.1: Stroke's Classification ............................................................... 7 Figure 2.2: Example using Fugl-Meyer Assesment-Upper Extremity with post stroke patients ................................................................................. 12 Figure 2.3: Virtual Reality Headset used for BRUTM .................................. 15 Figure 2.4: BRUTM workstation ................................................................. 16 Figure 2.5: BioTrak© workstation and software interface ........................ 16 Figure 3.1: Unreal engine 4 logotype ........................................................ 18 Figure 3.2: Image of the videogame Gears of War ................................... 19 Figure 3.3: Unity Logotype ....................................................................... 20 Figure 3.4: Game engine’s Global market participation ............................ 20 Figure 3.5: CryEngine logotype ................................................................. 22 Figure 3.6: Blender logotype .................................................................... 24 Figure 3.7: MakeHuman logotype ............................................................ 26 Figure 3.8: MakeHuman interface ............................................................ 26 Figure 3.9: Example of Marker based motion capture .............................. 27 Figure 3.10: Example of depth camera motion capture............................ 28 Figure 3.11: First-generation Kinect sensor (Xbox 360) ............................ 29 Figure 3.12: Kinect sensor components .................................................... 30 Figure 3.13 Depth image creation ............................................................ 31 Figure 3.14: PlayStation Eye device .......................................................... 32 Figure 3.15: Eye petTM for PlayStation Eye ................................................ 33 Figure 3.16: PrimeSense sensor Carmine 1.08 .......................................... 33 Figure 3.17: PrimeSense Sensor workflow ............................................... 34 Figure 4.1: Kinect sensor v2 ...................................................................... 36 Figure 4.2: Xbox Kinect adapter ................................................................ 38 Figure 4.3: MSI GT72 2QE DOMINATOR PRO ............................................ 40 Figure 4.4: Unity User Interface ................................................................ 41 Figure 4.5: Project Window ...................................................................... 42 Figure 4.6: Scene view .............................................................................. 42 Figure 4.7: Game view .............................................................................. 43 Figure 4.8: Hierarchy window ................................................................... 44 Figure 4.9: Inspector window ..................................................................