2011 IEEE International Conference on Robotics and Automation Shanghai International Conference Center May 9­13, 2011, Shanghai, China Fingertip Force and Contact Position and OrientationSensor Yu Sun Abstract— This paper presents a novel integrated system and control algorithms have beendeveloped with inspirations that is composed of a fingerprint sensor and a force sensor from humangrasping studies. Many encouraging results have to measure contact position and orientation ofthe fingertip been achieved. However, moreand more researchers have along with thecontactforce. The system uses fingerprints from the fingerprint sensor to identify thecontact position realized the limitation of only including contact force and and orientation with fingerprintfeaturessuch as core point contact position on the object. Many researchers have sought and ridge orientations. Thecontact position and orientation for a way to measure thecontact position on the fingertip are represented in a fingerpad coordinate system for grasping tobetter model and understand more sophisticatedgrasping studies. An experiment has beendesigned to evaluate the behaviors like rolling and regrasping [4]. proposed system in terms of accuracyand resolution with three subjects. The proposed system canbe used in human grasping Commercialtactile sensorssuch as FingerTPS from Pres- studies to characterize the fingerpad contact. sure Profile Systems [15] and flexible tactile sensors from Tekscan [23] have been successfully embedded on many I. INTRODUCTION robotic fingers toprovide rich spatial grasping information Roboticists have learned a great deal from studying human for grasping control. However due to their stiffness, those grasping as the humanhand exhibitssome of the most tactile sensors cannot deform along with the human finger complex human motor skills. Many studies ([22], [12], [5], tissue topreserve theelastic profile or precisely measure [9]) have proposedvarious taxonomies of humangrasping. thecontact spatial features, which is notidealtobe used A basic distinction exists between a power grasp, where the to in many humangrasping studies as wearable sensors goalis to stabilize an objectto resist external forces, and on fingertips. Most studies attach the tactile sensors on the a precision grasp, which is an essential skill to manipulate force sensors other than on fingertips, whichprovides local objects including stable grasps on objects, changing the contact spatial profiles around thecontact point rather than grasps (re-grasp), and maintaining the grasps during tasks. thecontact position on the fingertip. Recently, a number of A preponderance ofresearchhas focused on precision flexible tactile sensors have beendeveloped [7], which could grasps and on instrumentation to measure the fingertip forces be worn on fingertips as a part of a glove or with a thimble and position on the object during contact. A rich experimen- setup. However, the glove or thimble setup may measure tal paradigm for the study of humangrasping was devised differentlybetween trials if the tactile sensor is not aligned by Johansson and Westling [10], consisting of parallel force- with the user’s fingertip consistently. sensing pads for opposition grasps between the index finger This paper presents a novel fingertip force and posi- and thumb. This approachhas beengeneralized to add more tion/orientation (FFPO) sensor system that uses a regu- touchpads to include the middle, ring, or little fingers. In larforce/torque sensor to measure fingertip force and a general, instrumented objects are typically created to incor- fingerprint sensor mounted on the force/torque sensor to porate miniature 6-axis force/torque sensors at predefined measurecontact position and orientation on the fingertip grasp points (e.g., [20], [21], [16], [19]). Special purpose simultaneously. The fingerprintimage obtained from the objects were fabricated to incorporate force sensors. fingerprint sensor is processed and registered to a precon- Besides thecomplexhand postures and forces during structed fingerprint map with fingerprint core point and ridge grasping, the great variance in whichparts of the fingertip orientations. The center position of the registered fingerprint contact an object makes the grasping studies even more on the fingerprint map in the fingerprint coordinate system complex. To understand humandexterity, it is very important is defined as thecontact position on the fingertip. Since to fully study the local mechanicalinteraction between the thecontactlocal spatial features are measured from the fingerpad and an object. Many studies have been carried fingerprint, the obtained contact position and orientation are outto characterize the mechanical response profile in the always consistent between trials. fingerpad responding to force. Serinaet al. [17], and Pawluk Fingerprints have beenused in Neuroscience literature and Howe [14] haveexamined the dynamic force response to characterize the skindeformation level. Westling and of the fingerpad to indentors indifferent sizes and shapes. Johansson [25] and Birznieks et al. [2] have measured the Withvarious kinds offorce sensors, the fingertip force and area of the fingerprintleft on a grasping pointto indicate the thecontact points on the object were measured indifferent skindeformation related togasping force. In the study, the experiments by many research teams to study humandexter- fingertip wasstained with ink to enable the measurement. ous grasping and manipulation. Many robot graspplanning In robotics literature, Levesqueand Hayward [11] have used Yu Sun is with the Department of Computer Science and Engineering, fingerprint features to study the stretch and compression of Universityof South Florida. [email protected] the human fingerpad skin during tactileexploration. Kurita 978­1­61284­380­3/11/$26.00 ©2011 IEEE 1114 et al. [26] wereable to inferred contact force by observing Fingerprint Development Kit and connected to the PC with fingerprint changes atthe point of contactthrough a glass a USB cable. The fingerprintimage is collected at 14 fps. plate by a camera to estimate the incipient slip movement The fingerprint sensor has a scanning window size 12.8x18 between the fingertip and the glass. From our best knowl- mm and image resolution is 256 x 360 pixel. The fingerprint edge, theapproachof using fingerprints to measurecontact sensor has weight at 6grams with the size of 20.4(L) x positions has not beenproposedbefore. 27(W) x3.5(H) mm [24]. II. SYSTEM SETUPS (A)(B) Fig. 3. (A) The second design of the proposed integrated FFPO sensor system that a ATI Nano 17 force sensor and a solid-state TouchChip TCS1C Fingerprint Sensor(B) The ATI Nano 17 sensor has beenused in many studies Fig. 1. A schematic drawing of the FFPO sensor system: it is composed of a fingerprint sensor and a force sensor as a standard instrumentto measure fingertip force during grasping [1]. Its weightis around 9 grams with the size As illustrated in the schematic drawing Figure 1, the of 15 mm height and 17 mm diameter. Compared with the proposed FFPO sensor system is composedof a force sensor dimensions of the solid-state fingerprint sensor, the weight and a fingerprint sensor thatis rigidly mounted on the top increase is not significant, considering in many studies those of the force sensor. The center of the fingerprintimaging is ATI Nano 17 sensors are mounted on aluminum plates. The aligned with the center of the force sensor, and the short side solid-state fingerprint sensor is very thin and light weight, and the long side of the fingerprint sensor are parallelto the which doesn’t affectthe propertyof the force sensor as much. x-axis and the y-axis of the force sensorrespectively. The For most grasping studies that use the ATI Nano 17, the small contact force between the user’s finger and the FFPO sensor dimension increase will not affect its usage. Even the optical system is measured with the force sensor and thecontact fingerprint sensor is larger, it still couldbe useable in many spatial properties is characterized with the fingerprint sensor. applications. Figure 2 and 3 demonstrate twodesigns with twodifferent Compared withhigh sampling rate of the force sensor, types of fingerprint sensors: an optical fingerprint sensor and thecurrent off-the-shelf fingerprint sensors haveavery a solid-state fingerprint sensor. In Figure 2-A, a Futronic FS low sampling rate. However, as thecontact position in the 90 optical fingerprint sensor(Figure 2-C) is mounted on the fingertip doesn’t change very fast for many cases, the relative tool face of an ATI Nano 17 6-axis force sensor(Figure 2-B). low sampling rate will unlikely posealimitation of its use The force readings arecollected from a Sensoray 626 data for typical grasping studies. For high temporal resolution acquisition board and sampled at 1000 Hz. The Futronic FS requirement, a high speed fingerprint sensor couldbe used. 90 fingerprint sensor is connected to a PC with a USB cable. A Windows PC was used to collectthe data from the force The fingerprintimage is taken at 10 frame-per-second (fps). sensor and the fingerprint sensor. A Sensoray 626 data acqui- The fingerprint sensor has a scanning window size 15x22 sition card reads the force data from the ATI Nano 17 force mm and image resolution is 440x300 pixel. Its weightis 60 sensor at 1000 Hz sampling rate. A software was developed grams with the size of 43(L) x 24(W) x 25(H) mm. It uses to read the ATI Nano 17 force sensor through a Sensoray 626 infrared LEDs to illuminate the fingertip. data acquisition card at 1000 Hzand the fingerprintimage