THE HAND-HELD FORCE MAGNIFIER: SURGICAL INSTRUMENTS TO AUGMENT THE SENSE OF TOUCH by Randy Lee B.S., University of California, San Diego, 2010 M.S., Carnegie Mellon University, 2012 Submitted to the Graduate Faculty of Swanson School of Engineering in partial fulfillment of the requirements for the degree of Doctor of Philosophy University of Pittsburgh 2017 UNIVERSITY OF PITTSBURGH SWANSON SCHOOL OF ENGINEERING This dissertation was presented by Randy Lee It was defended on March 21, 2017 and approved by Roberta L. Klatzky, Ph.D., Professor, Department of Psychology, Carnegie Mellon University Zhi-Hong Mao, Ph.D., Associate Professor Department of Electrical and Computer Engineering and Department of Bioengineering Anne Robertson, Ph.D., Professor Department of Mechanical Engineering and Materials Science and Department of Bioengineering Dissertation Director: George Stetten, M.D., Ph.D., Professor, Department of Bioengineering ii Copyright © by Randy Lee 2017 iii THE HAND-HELD FORCE MAGNIFIER: SURGICAL TOOLS TO AUGMENT THE SENSE OF TOUCH Randy Lee, Ph.D. University of Pittsburgh, 2017 Modern surgeons routinely perform procedures with noisy, sub-threshold, or obscured visual and haptic feedback, either due to the necessary approach, or because the systems on which they are operating are exceeding delicate. For example, in cataract extraction, ophthalmic surgeons must peel away thin membranes in order to access and replace the lens of the eye. Elsewhere, dissection is now commonly performed with energy-delivering tools – rather than sharp blades – and damage to deep structures is possible if tissue contact is not well controlled. Surgeons compensate for their lack of tactile sensibility by relying solely on visual feedback, observing tissue deformation and other visual cues through surgical microscopes or cameras. Using visual information alone can make a procedure more difficult, because cognitive mediation is required to convert visual feedback into motor action. We call this the “haptic problem” in surgery because the human sensorimotor loop is deprived of critical tactile afferent information, increasing the chance for intraoperative injury and requiring extensive training before clinicians reach independent proficiency. Tools that enhance the surgeon’s direct perception of tool-tissue forces can therefore potentially reduce the risk of iatrogenic complications and improve patient outcomes. Towards iv this end, we have developed and characterized a new robotic surgical tool, the Hand-Held Force Magnifier (HHFM), which amplifies forces at the tool tip so they may be readily perceived by the user, a paradigm we call “in-situ” force feedback. In this dissertation, we describe the development of successive generations of HHFM prototypes, and the evaluation of a proposed human-in-the-loop control framework using the methods of psychophysics. Using these techniques, we have verified that our tool can reduce sensory perception thresholds, augmenting the user’s abilities beyond what is normally possible. Further, we have created models of human motor control in surgically relevant tasks such as membrane puncture, which have shown to be sensitive to push-pull direction and handedness effects. Force augmentation has also demonstrated improvements to force control in isometric force generation tasks. Finally, in support of future psychophysics work, we have developed an inexpensive, high-bandwidth, single axis haptic renderer using a commercial audio speaker. v TABLE OF CONTENTS 1.0 INTRODUCTION ........................................................................................................ 1 1.1 MOTIVATION: EMERGENCE OF THE “HAPTIC” PROBLEM .............. 2 1.1.1 Minimally Invasive Surgery ........................................................................... 3 1.1.2 Microsurgery .................................................................................................... 5 1.1.2.1 Vascular Surgery................................................................................... 6 1.1.2.2 Ophthalmic Surgery ............................................................................. 8 1.2 THE ROLE OF FORCE FEEDBACK IN SURGERY .................................. 11 1.3 THESIS OUTLINE ........................................................................................... 13 2.0 LITERATURE REVIEW .......................................................................................... 15 2.1 MECHANISMS OF FORCE SENSING AND ACTUATION ...................... 15 2.1.1 Force Sensing ................................................................................................. 16 2.1.1.1 Hookean Force Measurement ............................................................ 16 2.1.1.2 Resistive Force Measurement ............................................................ 17 2.1.1.3 Piezoelectric Force Measurement ...................................................... 19 2.1.1.4 Measurement Errors........................................................................... 21 2.1.2 Force Actuation.............................................................................................. 22 2.1.2.1 Motors .................................................................................................. 22 2.1.2.2 Solenoids and Voice Coils ................................................................... 23 vi 2.2 PSYCHOPHYSICS AND HAPTICS ............................................................... 24 2.2.1 Psychophysics ................................................................................................. 25 2.2.2 Anatomy and Neurophysiology of Touch .................................................... 28 2.2.3 Psychophysics of Touch................................................................................. 35 2.2.4 Haptic Renderers ........................................................................................... 36 2.3 HAPTICS IN SURGICAL ROBOTICS .......................................................... 39 2.3.1 Micron............................................................................................................. 41 2.3.2 Microtactus ..................................................................................................... 43 2.3.3 Steady Hand Robot ........................................................................................ 44 2.3.4 Pneumatic Haptic Feedback in Telesurgery ............................................... 47 2.3.5 Imperial College London Hand-Held Manipulators .................................. 48 2.4 DISCUSSION ..................................................................................................... 50 3.0 THE HAND-HELD FORCE MAGNIFIER ............................................................ 51 3.1 HAND-HELD FORCE MAGNIFIER CONTROL FRAMEWORK ........... 54 3.2 THE MODEL-1 HAND-HELD FORCE MAGNIFIER ................................ 56 3.2.1 HHFM Model-1 Design and Development .................................................. 56 3.2.2 HHFM Model-1 Psychophysical Experiments ............................................ 58 3.2.2.1 Experiment 1: Absolute Force Threshold ......................................... 59 3.2.2.2 Experiment 2: Just Noticeable Difference ........................................ 61 3.2.2.3 Experiment 3: Subjective Estimation of Force and Stiffness .......... 64 3.3 THE MODEL-2 HAND-HELD FORCE MAGNIFIER ................................ 70 3.3.1 HHFM Model-2 Design and Development .................................................. 70 3.3.2 HHFM Model-2 Control ............................................................................... 72 vii 3.4 DISCUSSION ..................................................................................................... 76 4.0 THE MODEL-3 HAND-HELD FORCE MAGNIFIER ......................................... 77 4.1 HHFM MODEL-3 DESIGN AND DEVELOPMENT ................................... 77 4.1.1 HHFM Model-3 Sensing Subsystem ............................................................ 79 4.1.2 HHFM Model-3 Actuation Subsystem ........................................................ 82 4.1.3 HHFM Model-3 Control ............................................................................... 83 4.2 HHFM MODEL-3 PSYCHOPHYSICAL EXPERIMENTS ......................... 89 4.2.1 Experiment 4: Membrane Puncture ............................................................ 89 4.2.2 Experiment 5: Handedness in Membrane Puncture ................................ 102 4.2.3 Experiment 6: Magnification Effects on Motor Control ......................... 111 4.2.3.1 Force Matching Experiment ............................................................ 112 4.2.3.2 Minimum Contact Experiment ........................................................ 126 4.3 DISCUSSION ................................................................................................... 137 4.3.1 Failure Modes Analysis and Mitigation..................................................... 137 4.3.2 Insights from Psychophysics ....................................................................... 139 5.0 THE MODEL-4 HAND-HELD FORCE MAGNIFIER ....................................... 142 5.1 HHFM MODEL-4 DESIGN AND DEVELOPMENT ................................. 144 5.1.1 HHFM Model-4 Sensing Subsystem .......................................................... 145 5.1.1.1 Prototype 1: Solid Column Sensor .................................................. 146 5.1.1.2 Prototype 2: Hollow Column Sensor