University of Calgary PRISM: University of Calgary's Digital Repository Graduate Studies The Vault: Electronic Theses and Dissertations 2018-08-29 Mechanisms of Integrating Vibrotactile and Force Cues for 3D User Interaction within Virtual Environments Tarng, Stanley Tarng, S. (2018). Mechanisms of Integrating Vibrotactile and Force Cues for 3D User Interaction within Virtual Environments (Unpublished master's thesis). University of Calgary. Calgary. AB. doi:10.11575/PRISM/32876 http://hdl.handle.net/1880/107698 master thesis University of Calgary graduate students retain copyright ownership and moral rights for their thesis. You may use this material in any way that is permitted by the Copyright Act or through licensing that has been assigned to the document. For uses that are not allowable under copyright legislation or licensing, you are required to seek permission. Downloaded from PRISM: https://prism.ucalgary.ca UNIVERSITY OF CALGARY Mechanisms of Integrating Vibrotactile and Force Cues for 3D User Interaction within Virtual Environments by Stanley Tarng A THESIS SUBMITTED TO THE FACULTY OF GRADUATE STUDIES IN PARTIAL FULFILMENT OF THE REQUIREMENTS FOR THE DEGREE OF MASTER OF SCIENCE GRADUATE PROGRAM IN ELECTRICAL AND COMPUTER ENGINEERING CALGARY, ALBERTA AUGUST, 2018 © Stanley Tarng 2018 Abstract A model for the mechanism of integration for vibrotactile and force cues in the haptic modality is important to facilitate task performance of human users in a three-dimensional (3D) virtual environment (VE). To investigate this mechanism, the research in this thesis used maximum likelihood estimation (MLE) and proposed proportional likelihood estimation (PLE) as the models of the integration. Two significant findings are as follows: (1) I found that based on the task accuracy, MLE was unable to integrate vibrotactile and force cues despite its ability to integrate cues of different modalities in literature; (2) Integration with PLE revealed that the mechanism of integration of vibrotactile and force cues may not be entirely additive as assumed in MLE. This work sheds an insight for proper model of integration between vibrotactile and force cues for interactive tasks in VEs. ii Table of Contents ABSTRACT ........................................................................................................................ II TABLE OF CONTENTS ...................................................................................................III LIST OF TABLES .............................................................................................................. V LIST OF FIGURES AND ILLUSTRATIONS................................................................. VI 1 INTRODUCTION ......................................................................................................1 1.1 Research Motivation ................................................................................................5 1.2 Objectives and Contributions ...................................................................................6 1.3 Thesis Organization .................................................................................................9 2 BACKGROUND ......................................................................................................10 2.1 Virtual Environments .............................................................................................10 2.2 Haptic Modality .....................................................................................................12 2.2.1 Haptics Devices in VEs ...................................................................................13 2.3 Multi-sensory Integration.......................................................................................15 3 APPLICABILITY OF MAXIMUM LIKELIHOOD ESTIMATION FOR VIBROTACTILE AND FORCE CUE INTEGRATION .........................................18 3.1 Maximum Likelihood Estimation in Cue Integration ............................................19 3.2 Experiment 1: Reliance Unbiased ..........................................................................22 3.2.1 Methods............................................................................................................22 3.2.2 Results ..............................................................................................................29 3.3 Experiment 2: Reliance Biased ..............................................................................32 3.3.1 Methods............................................................................................................32 3.3.2 Results ..............................................................................................................33 3.4 Experiment 3: Intensity Reduced ...........................................................................36 3.4.1 Methods............................................................................................................36 3.4.2 Results ..............................................................................................................37 3.5 General Discussion ................................................................................................40 3.6 Conclusion .............................................................................................................44 4 PROPORTIONAL LIKELIHOOD ESTIMATION FOR VIBROTACTILE AND FORCE CUE INTEGRATION.................................................................................45 4.1 Proportional Likelihood Estimation .......................................................................46 4.2 Analyses with PLE and Discussion .......................................................................49 4.3 Potential Applications ............................................................................................52 5 SUMMARY AND FUTURE WORK ......................................................................55 5.1 Summary ................................................................................................................55 5.2 Future Work ...........................................................................................................57 iii BIBLIOGRAPHY ..............................................................................................................59 A COPYRIGHT TRANSFER ......................................................................................67 B UNIVERSITY OF CALGARY ETHICS APPROVAL ...........................................75 C DATA COLLECTION .............................................................................................78 C.1 Objective Data .......................................................................................................78 C.2 Subjective Data ......................................................................................................79 C.2.1 Cybersickness Questionnaire ..........................................................................80 C.2.2 Perceptual Questionnaire ................................................................................81 C.2.3 NASA Task Load Index (NASA-TLX) ..........................................................82 D LIST OF SOFTWARE TOOLS................................................................................83 iv List of Tables 2.1: Different nerve receptors for the sense of touch. ................................................................... 12 3.1: Formulation of MLE weights for vibrotactile and force cues. .............................................. 21 3.2: The means and standard deviations of the subjective data and their ANOVA results among all testing blocks in Experiment 1 ............................................................................. 31 3.3: The means and standard deviations of the subjective data and their ANOVA results among all testing blocks in Experiment 2 ............................................................................. 35 3.4: The means and standard deviations of the subjective data and their ANOVA results among all testing blocks in Experiment 3. ............................................................................ 39 4.1: Summary of formulation of MLE and PLE for vibrotactile and force cues .......................... 47 v List of Figures and Illustrations 1.1: Examples of devices providing visual feedback in VEs; (a) Oculus Rift; (b) CAVE ............. 1 1.2: Examples of haptic devices; (a) PHANToM Omni with force feedback on the stylus; (b) VibroTac bracelet with six vibrators in each segment; and (c) HTC Vive controllers with circular touchpad control and vibration feedback. .................................................................. 3 2.1: Haptic devices for force and vibrotactile feedback respectively: (a) PHANToM Omni with a force feedback stylus to be held by a user; (b) VibroTac bracelet, to be wrapped around the users’ limbs for vibrotactile feedback; ................................................................ 14 3.1: MLE for an integration of vibrotactile and force cues. Accuracy is the normalized estimates from a series of detection tasks by human users. .................................................. 19 3.2: The VE and devices utilized for 3D user interaction: (a) the visual scene in the VE; (b) 3D shutter glasses and an IR emitter; (c) a VibroTac bracelet; (d) ear plugs and an E4 wristband; and (e) an Omni device. ...................................................................................... 23 3.3: (a) The visual scene