ASLBoT: Assistive Sign Language Bot Translator University of Central Florida College of Engineering and Computer Science Department of Electrical and Computer Engineering Final Report Senior Design II Dr. Samuel Richie Group 14 Members Gustavo Camero Computer Engineering [email protected] Luis Hurtado Electrical Engineering [email protected] Michael Loyd Electrical Engineering [email protected] Jared Spinks Computer Engineering [email protected] Client/Advisor Dr. Chung Yong Chan [email protected] Page i Group 14 Final Report Table of Contents 1. Executive Summary ···················································································································· 1 2. Project Description ······················································································································ 2 2.1 Project Background ····················································································· 2 2.2 Motivation ····································································································· 2 2.2.2 GPU Based ASLBoT ············································································································ 3 2.2.3 GPU-FPGA Based ASLBoT ·································································································· 3 2.2.4 SBC-MCU Based ASLBoT ···································································································· 3 2.3 Goals and Objectives ····················································································· 4 2.4 Requirement Specifications ········································································· 5 2.5 Marketing Requirements. ············································································ 5 3. Research Related to the Project··································································································· 7 3.1. Existing Projects and Products ··································································· 7 3.1.1. Google Translate ·············································································································· 7 3.1.2. Amazon Translate ············································································································ 8 3.1.3. Stratus Video ···················································································································· 8 3.2 Relevant Technologies ················································································· 8 3.2.1. American Sign Language Typography ··············································································· 9 3.2.1.1. SignWriting ················································································································ 9 3.2.1.2 Si5s ··························································································································· 10 3.2.1.3 SignFont ··················································································································· 11 3.2.1.4 Stokoe Notation ········································································································ 11 3.2.1.5 Gloss Notation··········································································································· 11 3.2.1.6 Extended Linear Stokoe (ELS) Notation ···································································· 13 3.2.2 Machine Learning ············································································································ 19 3.2.2.1. Speech to Text ·········································································································· 19 3.2.2.2 Text to Text Translation ···························································································· 24 3.2.2.3 Text-to-Speech ·········································································································· 33 3.2.3 Processor Technologies ··································································································· 35 3.2.3.1. CPU ·························································································································· 36 3.2.3.2. MCU ························································································································· 36 3.2.3.3. FPGA ························································································································ 37 3.2.3.4. GPU ·························································································································· 38 3.2.3.5 GPU versus FPGA ······································································································ 39 3.2.4 3D Graphics API Technologies ························································································· 40 3.2.4.1 OpenGL ····················································································································· 40 3.2.4.2 Direct3D 12 ·············································································································· 41 3.2.4.3 Metal ························································································································ 42 3.2.4.4 Vulkan ······················································································································ 42 3.2.5 Internet Connectivity ······································································································· 42 3.2.6 Voltage Regulation ··········································································································· 43 3.2.6.1 Linear Voltage Regulator··························································································· 43 3.2.6.2 Switching Voltage Regulator ····················································································· 44 3.2.6.3 Considerations for Voltage Regulators ······································································ 44 3.2.7. 3D Rendering Software Platforms ··················································································· 45 3.2.7.1. Unreal Engine 4 ········································································································ 46 3.2.7.2. Unity ························································································································ 46 3.2.7.3. MikuMikuDance ······································································································· 47 Page ii Group 14 Final Report 3.2.7.3. Blender ···················································································································· 47 3.2.7.4 Godot ························································································································ 48 3.3. Initial Components and Part Selections ··················································· 49 3.3.1 Sound System Selection ··································································································· 49 3.3.1.1 Amplifier ·················································································································· 49 3.3.1.2 Speaker ····················································································································· 51 3.3.1.3 Microphone ·············································································································· 52 3.3.2. FPGA Selection ················································································································ 54 3.3.2.1 Altera Cyclone IV – EP4CE22E22xxx ········································································· 55 3.3.2.2 Spartan 3E - XC3S500E-xPGx208C ············································································ 55 3.3.2.3 Artix 7 – XC7A35T-1CPG236C ··················································································· 55 3.3.3 Computer-on-Module (COM) Selection ············································································ 56 3.3.3.1 NVIDIA Jetson Nano Developer Board ······································································· 56 3.3.3.2 Nvidia Jetson TX2 Developer Board··········································································· 57 3.3.3.3 ASUS Tinker Board···································································································· 58 3.3.3.4 Raspberry Pi 4 Model B ····························································································· 58 3.3.4 Display Selection ············································································································· 59 3.3.4.1 10.1" Display & Audio IPS Panel ················································································ 60 3.3.4.2 Sceptre E205W-16003S LED Monitor ······································································· 60 3.3.4.3. UPERFECT 12.3” Touch Monitor··············································································· 60 3.3.5 Controller Selection ········································································································· 60 3.3.5.1 IR Remote ················································································································· 61 3.3.5.2 RF Remote ················································································································