Specialized CNT-Based Sensor Framework for Advanced Motion Tracking
Total Page：16
File Type：pdf, Size：1020Kb
Proceedings of the 54th Hawaii International Conference on System Sciences | 2021 Specialized CNT-based Sensor Framework for Advanced Motion Tracking Federico Gelsomini1,4, Patrick C. K, Hung2,4, Bill Kapralos2,4, Alvaro Uribe-Quevedo2, Michael Jenkin3,4, Akira Tokuhiro2, Kamen Kanev2,4, Makoto Hosoda4, Hidenori Mimura4 1 Sapienza University of Rome, Rome, Italy 2 Ontario Tech University, Oshawa, ON, Canada 3 York University, North York, ON, Canada 4 Shizuoka University, Hamamatsu, Japan [email protected], {patrick.hung, bill.kapralos, alvaro.quevedo, akira.tokuhiro}@uoit.ca, [email protected], [email protected], {hosoda.makoto, mimura.hidenori}@shizuoka.ac.jp Abstract The situation is very different, however, when we In this work, we discuss the design and development consider the tracking of the human hand with all of its of an advanced framework for high-fidelity finger fingers. Handheld (e.g., smartphones) and wrist-based motion tracking based on Specialized Carbon Nanotube (e.g., wristbands) tracking devices usually provide only (CNT) stretchable sensors developed at our research a synthesized six degrees-of-freedom (DOF) position facilities. Earlier versions of the CNT sensors have been and orientation feedback. Although modern tracking employed in the high-fidelity finger motion tracking devices may employ a combination of tracking sensors Data Glove commercialized by Yamaha, Japan. The such as accelerometers, gyroscopes, and framework presented in this paper encompasses our magnetometers, their combined feedback is typically continuing research and development of more advanced limited to nine DOF [7]. The full tracking of the five CNT-based sensors and the implementation of novel fingers of the human hand, in contrast, requires over 18 high-fidelity motion tracking products based on them. DOF which makes the task much more complex. The CNT sensor production and communication Practical complications also arise since attaching framework components are considered in detail and physical tracking devises to all fingers severely impedes wireless motion tracking experiments with the the subtle finger motions. developed hardware and software components Instead of using tracking devices physically integrated with the Yamaha Data Glove are reported. attached to the fingers, finger tracking experiments have been conducted by attaching markers to the fingers and 1. Introduction tracking the fingers optically using a small camera mounted on the palm [8]. The resulting system allowed the gathering of research data but was of limited Experimental tracking and digitization of hand and practical use mainly due to its complexity and the arm motions in research, education, and training burden imposed on the user by the size of the markers environments are common and their adoption in various and the presence of the camera. In other research practical application areas continues to expand [1, 2]. projects, the classic motion tracking approach based on Since the late 1970s when the first hand tracking gloves spherical retro-reflective markers was used [9]. were reported [3], motion tracking and digitization Although small, the retro-reflective markers had to be technologies have progressed significantly. Nowadays, attached to all of the fingers, hampering natural finger most smartphones have embedded motion tracking movements. Experiments with finger tracking of piano sensors enabling direct gait tracking and hand gesture players were conducted by employing painted markers interactions [4, 5]. Various types of compact wearable that were less invasive [10]. However, the optical motion trackers are also available from companies such tracking method in this system was not self-sufficient as as Mbientlab (https://mbientlab.com/), Xsens additional capacitive and Musical Instrument Digital (https://www.xsens.com/), and other producers Interface (MIDI) sensors were attached to the piano worldwide. Following this trend, motion tracking and keys. Furthermore, experiments with optical tracking analysis are rapidly expanding to the Internet of Things using the Microsoft Kinect and the Leap Motion (IoT), addressing applications in sports, health reported in [11], revealed high latencies and numerous monitoring, medical rehabilitation, amongst many false positives/negatives during piano playing. As a others [6]. URI: https://hdl.handle.net/10125/70843 978-0-9981331-4-0 Page 1898 (CC BY-NC-ND 4.0) result, an alternative ultrasound-based tracking of research, our team is developing rapid-response, approach was deemed more robust. widely stretchable sensors based on carbon nanotubes Given the difficulties with optical finger motion specialized for the detection of human motions [25, 26, tracking, more recent systems have shifted to other, non- 27]. optical methods for direct sensing of finger motions. Such systems are often designed as wired gloves with 2. CNT sensor production framework embedded sensors for direct tracking of the angular positions of the finger joints. Employing wired gloves The specialized stretchable sensors stem from the for finger motion tracking is particularly suitable for synthesis of vertically aligned ultralong multi-walled applications where gloves are worn as a code of Carbon Nanotubes (CNTs) using iron chloride powder. practice. Wired gloves can be seamlessly adopted, for This unique process, co-invented by one of the authors example, in medical training where surgical gloves are of the present work [25], is essential for the reliable and an indispensable component of many medical cost-effective fabrication of the CNT material procedures. Of course, such wired gloves should mimic embedded in the high-fidelity motion tracking sensors the physical properties of the surgical gloves and as discussed below. provide the trainees with a touch and feel similar to real Initially, a 2.1 mm long bulk CNT array is grown surgical gloves. by conventional thermal chemical vapor deposition on a Technological advancements and novel materials, quartz surface with a single gas flow of acetylene for 20 specifically designed to meet the requirements of the min. The grown CNT is then spun into a yarn which target application domain become, therefore, essentially transforms the three-dimensional array into a indispensable in wired glove designs. Along with the horizontally aligned web representing a two- use of special materials, finger motion tracking sensors dimensional network of CNTs. As a result, CNT webs that blend into the glove fabric and allow for seamless can be efficiently assembled into large-scale structures integration into the wired glove are also required. without twisting. The early wired glove designs [3] employed Afterward, well-aligned CNT sheets are fabricated embedded optical sensors constructed from a light by stacking and shrinking the CNT webs [26]. We have emitter and a photocell connected by a flexible rubber produced such a CNT sheet by winding, stacking on a tube that bends following the motions of the human drum, and cutting for proper flattening. The typical fingers [12]. In such sensors, the light reaching the number of CNT web layers wound on the drum is photocell decreases proportionally to the tube bend between eight and 12. which is reflected in the output voltage of the photocell. Experimental CNT strain sensors are then In a similar manner, more performant and easy to handle manufactured by placing the CNT sheet on a flat and sensors have been constructed from an optical fiber with smooth substrate (e.g., glass) in a direction parallel to an intentionally damaged surface that attenuates the the stretching direction and impregnating it with light propagation proportionally to the bend [13]. More elastomeric resin. Rubber-like elastomeric resin with advanced fiber-optic sensors based on double cladding low elasticity and low-loss properties is required for this fiber have also been reported [14]. Such fiber optic process. After a careful investigation, we selected sensors used to be employed in high-end wired gloves, Polycarbonate-urethane (PCU) resin for its ease of (e.g., the Virtual Programming Languages (VPL) bonding, durability, and hydrolysis and chemical DataGlove [15]), while lower-cost capacitive [16], and resistance. To enhance the contraction properties of the resistive bend sensors were commonly used in substrate, we have also employed segmented consumer-grade products (e.g., the Nintendo Power Polytetramethylene ether Glycol-urethane (PTMGU) Glove [17]). Some bendable resistive sensors for wired [27]. The expansion-contraction capabilities of our CNT gloves are produced by applying conductive inks to a sensors are illustrated in Figure 1. flexible substrate [18]. However, such sensors are more The fabrication of high-fidelity CNT strain sensors sensitive to expansion than to compression and thus requires precise alignment of the CNT bundle direction generate an asymmetric response. with the CNT sheet and the electrode directions. Once For high fidelity finger motion tracking and the this is confirmed, the multi-layer structure is combined detection of subtle finger movements, sensing of the with the elastomer resin layer. A spin coater has been bending motion is too restrictive and may not provide used to apply the elastomer resin uniformly over the sufficient information. In such cases, stretchable sensors surface of the CNT bundle to a thickness of several are deemed more appropriate, and active research is dozens of micrometers. continuing in this field. Currently,