Real-Time Translator with ASL Interpretation (RTAI) University of Central Florida Department of Electrical and Computer Engineering Final Report Senior Design I Dr. Samuel Richie Group 14 Members Gustavo Camero Computer Engineering [email protected] Luis Hurtado Electrical Engineering [email protected] Michael Loyd Electrical Engineering [email protected] Jared Spinks Computer Engineering [email protected] Client/Advisor Dr. Chung Yong Chan [email protected] Page i Group 14 Final Report Table of Contents 1. Executive Summary ····················································································································· 1 2. Project Description ······················································································································ 2 2.1 Project Background ····················································································· 2 2.2 Motivation ··································································································· 2 2.2.2 GPU based RTAI ············································································································ 3 2.2.3 GPU-FPGA based RTAI ·································································································· 3 2.3 Goals and Objectives ···················································································· 3 2.4 Requirement Specification ·········································································· 4 2.5 Marketing Requirements. ············································································ 4 3. Research Related to the Project ··································································································· 6 3.1. Existing Projects and Products ·································································· 6 3.1.1. Google Translate ··········································································································· 6 3.1.2. Amazon Translate ········································································································· 7 3.1.3. Stratus Video ················································································································ 7 3.2 Relevant Technologies ················································································· 7 3.2.1. American Sign Language Typography ············································································ 7 3.2.1.1. SignWriting ············································································································ 8 3.2.1.2 Si5s ························································································································· 9 3.2.1.3 SignFont ·················································································································· 9 3.2.1.4 Stokoe Notation ······································································································ 10 3.2.1.5 Gloss Notation ········································································································ 10 3.2.1.6 Extended Linear Stokoe (ELS) Notation ··································································· 12 3.2.2 Machine Learning ········································································································· 18 3.2.2.1. Speech to Text ······································································································· 18 3.2.2.2 Text to Text Translation ·························································································· 23 3.2.2.3 Text-to-Speech ······································································································· 32 3.2.3 Processor Technologies ································································································· 35 3.2.3.1. CPU ······················································································································· 35 3.2.3.2. MCU ······················································································································ 36 3.2.3.3. FPGA ····················································································································· 37 3.2.3.4. GPU ······················································································································· 37 3.2.3.5 GPU versus FPGA ···································································································· 39 3.2.4 3D Graphics API Technologies ······················································································· 40 3.2.4.1 OpenGL ·················································································································· 40 3.2.4.2 Direct3D 12 ············································································································ 41 3.2.4.3 Metal ····················································································································· 42 3.2.4.4 Vulkan ··················································································································· 42 3.2.5 Internet Connectivity ···································································································· 42 3.2.6 Voltage Regulation ········································································································ 43 3.2.6.1 Linear Voltage Regulator ························································································ 44 3.2.6.2 Switching Voltage Regulator ··················································································· 44 3.2.6.3 Considerations for Voltage Regulators ····································································· 45 3.2.7. 3D Rendering Software Platforms ················································································· 47 3.2.7.1. Unreal Engine 4 ····································································································· 47 3.2.7.2. Unity ····················································································································· 48 3.2.7.3. MikuMikuDance ····································································································· 48 3.2.7.3. Blender ················································································································· 49 Page ii Group 14 Final Report 3.2.7.4 Godot ····················································································································· 49 3.3. Components and Part Selections ····························································· 51 3.3.1 Sound System Selection ································································································· 51 3.3.1.1 Amplifier ················································································································ 51 3.3.1.2 Speaker ·················································································································· 52 3.3.1.3 Microphone ············································································································ 53 3.3.2. FPGA Selection ············································································································· 55 3.3.2.1 Altera Cyclone IV – EP4CE22E22xxx ········································································ 56 3.3.2.2 Spartan 3E - XC3S500E-xPGx208C ·········································································· 57 3.3.2.3 Artix 7 – XC7A35T-1CPG236C ················································································· 57 3.3.3 Computer-on-Module (COM) Selection ··········································································· 58 3.3.3.1 Nvidia Jetson Nano Developer Board ······································································· 58 3.3.3.2 Nvidia Jetson TX2 Developer Board ········································································· 59 3.3.3.3 ASUS Tinker Board ································································································· 59 3.3.3.4 Raspberry Pi 4 Model B ·························································································· 60 3.3.4 Display Selection ··········································································································· 61 3.3.4.1 10.1" Display & Audio IPS Panel ·············································································· 61 3.3.4.2 Sceptre E205W-16003S LED Monitor ······································································ 62 3.3.4.3. UPERFECT 12.3” Touch Monitor ············································································· 62 3.3.5 Controller Selection ······································································································ 62 3.3.5.1 IR Remote ·············································································································· 62 3.3.5.2 RF Remote ············································································································· 63 3.3.5.3 Push Button ··········································································································· 63 3.3.6 Wi-Fi Module Selection ··································································································