DESIGN, TESTING AND IMPLEMENTATION OF A SMARTPHONE PERIPHERAL FOR BIMANUAL SKIN STRETCH FEEDBACK AND USER INPUT by Markus Nils Montandon A thesis submitted to the faculty of The University of Utah in partial fulfillment of the requirements for the degree of Master of Science Department of Mechanical Engineering The University of Utah May 2013 Copyright © Markus Nils Montandon 2013 All Rights Reserved The University of Utah Graduate School STATEMENT OF THESIS APPROVAL The thesis of Markus Nils Montandon has been approved by the following supervisory committee members: William R. Provancher Chair 3/14/2013 Date Approved Jake J. Abbott Member 3/14/2013 Date Approved Donald S. Bloswick Member 3/14/2013 Date Approved and by Timothy A. Ameel Chair of the Department of_______________ Mechanical Engineering and by Donna M. White, Interim Dean of The Graduate School. ABSTRACT Haptic interactions with smartphones are generally restricted to vibrotactile feedback that offers limited distinction between delivered tactile cues. The lateral movement of a small, high-friction contactor at the fingerpad can be used to induce skin stretch tangent to the skin’s surface. This method has been demonstrated to reliably communicate four cardinal directions with 1 mm translations of the device’s contactor, when finger motion is properly restrained. While earlier research has used a thimble to restrain the finger, this interface has been made portable by incorporating a simple conical hole as a finger restraint. An initial portable device design used RC hobby servos and the conical hole finger restraint, but the shape and size of this portable device wasn’t compatible with smartphone form factors. This design also had significant compliance and backlash that must be compensated for with additional control schemes. In contrast, this thesis presents the design, fabrication, and testing of a low-profile skin-stretch display (LPSSD) with a novel actuation design for delivering complex tactile cues with minimal backlash or hysteresis of the skin contactor or “tactor.” This flatter mechanism features embedded sensors for fingertip cursor control and selection. This device’s nonlinear tactor motions are compensated for using table look-up and high-frequency open-loop control to create direction cues with 1.8 mm radial tactor displacements in 16 directions (distributed evenly every 22.5°) before returning to center. Two LPSSDs are incorporated into a smartphone peripheral and used in single-handed and bimanual tests to identify 16 directions. Users also participated in “relative” identification tests where they were first provided a reference direction cue in the forward/north direction followed by the cue direction that they were to identify. Tests were performed with the user’s thumbs oriented in the forward direction and with thumbs angled inward slightly, similar to the angled- thumb orientation console game controllers. Users are found to have increased performance with an angled-thumb orientation. They performed similarly when stimuli were delivered to their right or left thumbs, and had significantly better performance judging direction cues with both thumbs simultaneously. Participants also performed slightly better in identifying the relative direction cues than the absolute. iv TABLE OF CONTENTS ABSTRACT.................................................................................................................................iii ACKNOWLEDGMENTS....................................................................................................... vii Chapter 1 INTRODUCTION.............................................................................................................1 1.1 Thesis Overview...........................................................................................................6 2 BACKGROUND............................................................................................................... 9 2.1 Consumer Devices with Haptic Feedback............................................................... 9 2.2 Prior Finger Pad Skin Slip and Stretch Feedback Research.................................12 2.3 Prior Research on Portable Haptic Feedback Devices......................................... 16 3 DESIGN OF A LOW-PROFILE SKIN-STRETCH DISPLAY AND SMARTPHONE PERIPHERAL..................................................... 18 3.1 Prior Skin-Stretch Device Overview and Performance Summary.....................18 3.2 Design Targets for Low-Profile Skin-Stretch Display......................................... 22 3.3 Low-Profile Skin-Stretch Display Design and Fabrication..................................25 3.4 Integrated User Input Interfaces.............................................................................. 33 3.5 Electrical Components..............................................................................................36 3.6 Android Interface and Control.................................................................................37 3.7 Bimanual Skin-Stretch Device Assembly..............................................................40 3.8 Additional Microcontroller and Android Development for Input Sensor Integration ................................................................................................. 42 4 DEVELOPMENT OF DIRECTIONAL CUES AND DEVICE CALIBRATION...................................................................................44 4.1 Calibration Hardware and Initial Direction Cues.................................................. 44 4.2 Sliding-Plate Workspace Curvature and Cardinal Directions............................ 46 4.3 Path Quality of Angular Direction Cues................................................................ 48 4.4 Multiple Waypoint Path Correction and Control Setup....................................... 50 4.5 Development of Direction Cue Tactor Paths.........................................................53 4.6 Visual Verification of Calibration Hardware and Direction C ues......................56 4.7 Low-Profile Skin-Stretch Display Performance................................................... 58 4.8 General Compensation for Workspace Correction...............................................61 4.9 Overall Device Design Performance.......................................................................64 5 DIRECTION IDENTIFICATION USER EXPERIMENTS.....................................66 5.1 Pilot Test Method and Procedures..........................................................................66 5.2 Pilot Test Results....................................................................................................... 67 5.3 Absolute Identification and Relative Identification Test Methods for Main Experiment....................................................................................71 5.4 Main Experiment Results......................................................................................... 77 5.4.1 Results for Absolute Identification Experiment with Straight-Thumb Orientation.....................................................................79 5.4.2 Results for Absolute Identification Experiment with Angled-Thumb Orientation......................................................................90 5.4.3 Results for Relative Identification Experiment with Straight-Thumb Orientation.....................................................................99 5.4.4 Results for Relative Identification Experiment with Angled-Thumb Orientation................................................................... 105 5.4.5 Performance Comparison Between Absolute Versus Relative Identification.....................................................................................111 5.4.6 Information Transfer....................................................................................... 111 5.4.7 Direction Perception Bias and Oblique Effect.............................................117 5.4.8 d’ Signal Detection Analysis.........................................................................121 6 CONCLUSIONS AND FUTURE W ORK................................................................ 123 6.1 Conclusions..............................................................................................................123 6.2 Contributions............................................................................................................124 6.3 Future W ork.............................................................................................................126 REFERENCES........................................................................................................................127 vi ACKNOWLEDGMENTS First, I want to thank my incredible family. I would like to share my deepest appreciation to my wife Brandi and daughter Katey, who have lifted me up in difficult times, been so patient and have given me confidence when in doubt. You girls make every day an adventure. Thank you both for your love and laughter. I need to thank my parents, Michel and Susan, for their unrelenting support and encouragement. I have always been able to look to them for examples of compassion, hard work and joy. Thank you for listening to wild and rambling ideas for all these years. Special thanks to my advisor, Prof. William Provancher, for his technical knowledge, seasoned guidance, and reassurance. I have been pushed further than I thought possible and learned so much because of his promptings. It has been a privilege to collaborate