A Language of Tactile Motion Instructions for Physical Activities
Total Page:16
File Type:pdf, Size:1020Kb
A Language of Tactile Motion Instructions for Physical Activities Von der Fakultät für Mathematik, Informatik und Naturwissenschaften der RWTH Aachen University zur Erlangung des akademischen Grades eines Doktors der Naturwissenschaften genehmigte Dissertation vorgelegt von Dipl.-Inform. Daniel Spelmezan aus Bistriţa, Rumänien Berichter: Prof. Dr. Jan Borchers Prof. Dr. Martina Ziefle Tag der mündlichen Prüfung: 28. Juni 2011 Diese Dissertation ist auf den Internetseiten der Hochschulbibliothek online verfügbar. iii Contents Abstract xvii Uberblick¨ xix Acknowledgements xxi Conventions xxiii 1 Introduction 1 1.1 Tactile Instructions Correct Wrong Posture . .2 1.2 Context and Scope of this Thesis . .3 1.2.1 Visual and Audio Feedback . .4 1.2.2 Tactile Feedback and Force Feedback . .5 1.2.3 Outlook . .6 1.3 Thesis Contributions . .7 1.4 Thesis Structure . .8 I Wearable Sensing and Feedback Devices 11 2 Wearable Computing 13 2.1 Wearable Computers . 13 2.2 Context-Awareness . 14 iv Contents 2.3 Toolkits for Building Wearable Computing Applications . 16 2.3.1 Sensing Devices and Prototyping Platforms . 16 2.3.2 Software Toolkits for Sensor Data Analysis . 19 2.3.3 Tactile Feedback Devices . 19 2.4 Iterative Design: A Challenge in Designing Wearable Com- puting Applications . 20 2.5 Closing Remarks . 22 3 A Wearable Sensing and Feedback System 23 3.1 Hardware Platform . 23 3.1.1 The SensAct Box . 24 3.1.2 The Mobile Host Device . 27 3.2 Software Tools . 27 3.2.1 Sensor Monitor and Motor Control . 27 3.2.2 iSense . 28 3.2.3 Tactile Editor and SensAct Control . 29 3.3 Closing Remarks . 30 4 Towards a Wearable Snowboarding Assistant 33 4.1 Interviews with Snowboard Instructors . 34 4.2 Common Mistakes in Snowboarding . 36 4.3 Initial Testing of the Technology . 38 4.3.1 Hardware and Sensor Setup . 38 4.3.2 Participants and Study Procedure . 40 4.3.3 Preliminary Results . 41 4.3.4 The Instructors' Opinion . 43 4.4 Sensing Basic Context Information While Snowboarding . 44 Contents v 4.4.1 Hardware Setup . 45 4.4.2 Participants and Study Procedure . 46 4.4.3 Classification of the Riding Edge and of Turns . 46 Results . 48 Discussion . 50 4.4.4 Knee Flexion during the Ride . 51 Results . 53 Discussion . 54 4.4.5 Activity Recognition: Riding and Pausing . 54 Results . 55 Discussion . 55 4.5 Closing Remarks . 56 II Tactile Motion Instructions 57 5 Fundamentals of Tactile Perception 59 5.1 The Cutaneous Sense . 59 5.2 Tactile Display Technologies . 62 5.3 Parameters for Tactile Information Transfer . 64 5.4 Applications for Tactile Feedback . 68 5.4.1 Sensory Substitution . 68 5.4.2 Navigation Systems . 69 5.4.3 Warning Signals . 70 5.4.4 Interaction with Mobile Devices . 71 5.4.5 Distant and Emotional Communication . 72 5.4.6 Virtual Reality . 72 vi Contents 5.5 Closing Remarks . 73 6 Tactile Motion Instructions 75 6.1 Informing the Design . 77 6.1.1 Expressive Sensations . 78 6.1.2 Stimulus Duration and Perception under Workload . 79 6.1.3 Stimulus Locations on the Body . 81 6.1.4 A Notation for Tactile Patterns . 82 6.1.5 Tactile Suit and Hardware Setup . 83 6.2 Intuitive Interpretation of Tactile Stimuli . 86 6.2.1 Participants and Experimental Setup . 87 6.2.2 Results . 87 6.2.3 Discussion . 89 6.2.4 First Design Recommendations . 91 6.3 Tactile Motion Instructions in Stationary and in Active Sit- uations . 94 6.3.1 Perception under Increasingly Realistic Conditions . 94 Participants and Experimental Setup . 95 Results . 97 Discussion . 101 6.3.2 Perception under Real-World Conditions: Snowboarding . 102 Participants . 103 Hardware Setup . 103 Experimental Setup . 104 Results . 105 Discussion . 109 Contents vii 6.3.3 Perception under Real-World Conditions: Horseback Riding . 110 Participants and Hardware Setup . 111 Experimental Setup . 112 Results . 112 Discussion . 114 6.3.4 Revised Design Recommendations . 115 6.4 Instructions for Upper Body Rotation . 116 6.4.1 Participants . 117 6.4.2 Experimental Setup . 118 6.4.3 Results . 119 6.4.4 Discussion . 120 6.5 Preliminary Design Recommendations . 121 6.6 Closing Remarks . 124 7 A Language of Tactile Motion Instructions 127 7.1 Tactile Languages . 128 7.1.1 Characters as Basic Elements . 128 7.1.2 Words as Basic Elements . 129 7.1.3 A Tactile Language for Physical Activities . 130 7.2 Compound Tactile Motion Instructions . 130 7.2.1 Participants and Experimental Setup . 133 7.2.2 Results . 136 7.2.3 Discussion . 141 7.3 Characteristics of Tactile Motion Instructions . 144 7.3.1 Participants and Experimental Setup . 145 viii Contents 7.3.2 Results . 147 7.3.3 Discussion . 150 7.4 Closing Remarks . 152 8 Learning Motor Skills 153 8.1 Feedback in Motor Skill Learning . 154 8.2 Concurrent Tactile Motion Instructions for Learning Motor Skills in Snowboarding . 156 8.3 Experimental Setup . 159 8.3.1 Hardware Setup . 160 8.3.2 Pilot Test with Snowboard Instructor . 161 8.3.3 Participants . 163 8.3.4 Experimental Procedure . 164 8.4 Results . 166 8.4.1 Classification of the Riding Edge . 166 8.4.2 The Participants' Riding Skills (Video Analysis) . 170 Riding Mistakes . 170 Quality of Descents . 171 Individual Progress . 173 8.4.3 The Participants' Opinion (Questionnare Results) . 174 8.5 Discussion . 177 8.5.1 The Wearable System . 177 8.5.2 Video Analysis and Questionnaire Results . 179 8.5.3 Study Limitations . 182 8.6 A General Set of Tactile Motion Instructions . 184 8.7 Design Recommendations for Tactile Motion Instructions . 190 Contents ix 8.8 Closing Remarks . 192 III Conclusion 193 9 Conclusions 195 10 Future Work 199 10.1 Wearable Computing . 199 10.2 Tactile Motion Instructions . 200 10.3 Motor Learning . 202 IV Appendix 205 A Motor Shield Schematic 207 B Post-Study Questionnaire: Comparing Tactile Instructions to Audio Instructions While Snowboarding 209 C Post-Study Questionnaire: Perception of Tactile Instruc- tions While Horseback Riding 213 D Post-Study Questionnaire: Learning New Motor Skills 217 E The Structure of Languages 223 Bibliography 225 Index 245 Publications 253 Curriculum Vitae 255 xi List of Figures 1.1 Tennis versus snowboard lessons . .2 1.2 The Ski Coach. .5 1.3 ACM SIGCHI Curricula for HCI . .8 2.1 XSens MTx motion capture device and MVN suit . 16 2.2 Shake SK6 and Arduino BT . 17 2.3 TactaBoard and TactaVest . 19 2.4 Iterative design . 20 3.1 System architecture of the wearable platform . 24 3.2 The first sensor box . 25 3.3 The custom-built motor shield . 25 3.4 The second prototype of the SensAct box . 26 3.5 The final SensAct box . 26 3.6 iSense Mobile Software . 28 3.7 iSense Software . 28 3.8 Tactile Editor Software . 29 3.9 SensAct Control Software . 30 4.1 Snowboard terminology . 36 4.2 Neutral position, frontside turn, backside turn . 36 xii List of Figures 4.3 Correct posture for basic turns . 37 4.4 Typical snowboarding mistakes . 37 4.5 Bend sensors at the knees . 39 4.6 Setup of pressure sensors . 39 4.7 The sensing system for the first field study . 40 4.8 Raw force sensor values while snowboarding . 41 4.9 Raw bend sensor values while snowboarding . 42 4.10 Optical bend sensor . 45 4.11 Classification of turns while snowboarding . 49 4.12 Raw optical bend sensor values while snowboarding.