Prototyping a Portable, Affordable Sign Language Glove A Major Qualifying Project Submitted to the Faculty of Worcester Polytechnic Institute In partial fulfillment of the requirements for the Degree of Bachelor of Science Electric and Computer Engineering By ____________________________ Princesa Cloutier ____________________________ Alex Helderman ____________________________ Rigen Mehilli Date: April 27, 2016 Project Advisors: ___________________________________ Professor Alexander Wyglinski, Advisor ___________________________________ Professor Alexander Emanuel, Advisor This report represents work of WPI undergraduate students submitted to the faculty as evidence of a degree requirement. WPI routinely publishes these reports on its web site without editorial or peer review. For more information about the projects program at WPI, see http://www.wpi.edu/Academics/Projects. Abstract Communication between speakers and non-speakers of American Sign Language (ASL) can be problematic, inconvenient, and expensive. This project attempts to bridge the communication gap by designing a portable glove that captures the user’s ASL gestures and outputs the translated text on a smartphone. The glove is equipped with flex sensors, contact sensors, and a gyroscope to measure the flexion of the fingers, the contact between fingers, and the rotation of the hand. The glove’s Arduino UNO microcontroller analyzes the sensor readings to identify the gesture from a library of learned gestures. The Bluetooth module transmits the gesture to a smartphone. Using this device, one day speakers of ASL may be able to communicate with others in an affordable and convenient way. ii | P a g e Acknowledgements Without the guidance and motivation of several groups of people, this project would not have achieved as much as it did. These groups of people include our advisors, other faculty of the Electrical and Computer Engineering Department at WPI, and other students. This project would not exist without the support of our two advisors, Professors Wyglinski and Professor Emanuel of WPI. We approached them separately with an idea for a project that was not quite in their research areas, but with enthusiasm they accepted advisorship of the project. Throughout the project they kept us motivated, and whenever we had a problem, they always had plenty of suggestions about how to solve it. Other faculty of WPI helped us with specific challenges. We sought help with designing our PCB. Professor Bitar created a thorough instructional video describing the Printed Circuit Board (PCB) design process. His video allowed us to complete the majority of the PCB without much outside help. Help with the final touches of the PCB came from Mr. Joseph “Joe” St. Germain of WPI’s Robotics Engineering Department. Without the PCB, the project would not be as compact and refined. Mr. Appleyard of the Electrical and Computer Engineering Department was crucial to the assembly of our prototype. For every troublesome problem, Mr. Appleyard had a greater solution. He helped us with selecting the best components, ordering parts, soldering, wiring, gluing, and much more. The assembled prototype is sleek thanks to his help. Of course, without the Electrical and Computer Engineering Department, we would not have the resources to complete this project. Ms. Charmaine Jemison and the other administrative assistants offered help whenever we asked. Lastly, we would like to thank Worcester Polytechnic Institute for the opportunity and the preparatory education required to complete this project. iii | P a g e Contents Abstract .................................................................................................................................................. ii Acknowledgements ................................................................................................................................ iii Contents................................................................................................................................................. iv List of Figures ....................................................................................................................................... viii List of Tables .......................................................................................................................................... xi List of Equations ..................................................................................................................................... xi Executive Summary ............................................................................................................................... xii 1. Introduction ........................................................................................................................................ 1 2. Technology for Capturing Gestures ...................................................................................................... 4 2.1 Communicating between the Hearing and Non-Hearing ................................................................. 4 2.2 The Mechanics of ASL .................................................................................................................... 8 2.3 Sensing, Signaling, and Processing Gestures ................................................................................. 10 2.4 Software for Classifying Gestures ................................................................................................. 12 2.5 Chapter Summary ........................................................................................................................ 13 3. Proposed Design ................................................................................................................................ 15 3.1 User Requirements and Design Specifications .............................................................................. 15 Explicit Requirements .................................................................................................................... 15 Implicit Requirements .................................................................................................................... 15 3.2 Design Selection .......................................................................................................................... 16 3.4 Sensor System ............................................................................................................................. 19 3.6 Processing Subsystem .................................................................................................................. 21 3.7 Output Subsystem ....................................................................................................................... 21 3.8 Chapter Summary ............................................................................................................................ 22 iv | P a g e 4. Methodology ..................................................................................................................................... 22 4.1 Hardware Development ............................................................................................................... 23 4.1.1 Finger Position Sensing.......................................................................................................... 23 4.1.2 Contact Sensing..................................................................................................................... 34 4.1.3 Motion Sensing ..................................................................................................................... 42 4.2 Standalone Arduino UNO ............................................................................................................. 43 4.3 Software Development ................................................................................................................ 44 4.4 Visualizing the Output .................................................................................................................. 54 Loopback Test: Bluetooth Serial Monitoring with Termite .............................................................. 56 Test1: Bluetooth Upload and Monitoring with Arduino IDE ............................................................ 58 Test 2: USB Upload and Monitoring with Termite .......................................................................... 59 Test 3: Bluetooth Upload and Monitoring with Termite ................................................................. 60 Test 4: Bluetooth Upload and Monitoring with Smartphone .......................................................... 60 Observations about Bluetooth Upload and Monitoring .................................................................. 61 Android App Development ............................................................................................................ 61 4.5 Powering the Glove...................................................................................................................... 63 4.6 PCB Process ................................................................................................................................. 64 4.7 Glove Material ............................................................................................................................. 67 4.8 Assembling the Glove Prototype .................................................................................................. 71 4.9 Chapter Summary .......................................................................................................................