The Use of Mobile Phones As Service-Delivery Devices in a Sign Language Machine Translation System
Total Page:16
File Type:pdf, Size:1020Kb
View metadata, citation and similar papers at core.ac.uk brought to you by CORE provided by UWC Theses and Dissertations UNIVERSITY OF THE WESTERN CAPE The Use of Mobile Phones as Service-Delivery Devices in a Sign Language Machine Translation System by Mehrdad Ghaziasgar A thesis submitted in fulfillment for the degree of Master of Science in the Faculty of Science Department of Computer Science August 2010 Declaration of Authorship I, Mehrdad Ghaziasgar, declare that this thesis titled, `The Use of Mobile Phones as Service-Delivery Devices in a Sign Language Machine Translation System' and the work presented in it are my own. I confirm that: This work was done wholly or mainly while in candidature for a research degree at this University. Where any part of this thesis has previously been submitted for a degree or any other qualification at this University or any other institution, this has been clearly stated. Where I have consulted the published work of others, this is always clearly at- tributed. Where I have quoted from the work of others, the source is always given. With the exception of such quotations, this thesis is entirely my own work. I have acknowledged all main sources of help. Where the thesis is based on work done by myself jointly with others, I have made clear exactly what was done by others and what I have contributed myself. Signed: Date: i \All containers are reduced in capacity by what is placed in them, except a container of knowledge, which expands." -Ali son of Abu Talib UNIVERSITY OF THE WESTERN CAPE Abstract Faculty of Science Department of Computer Science Master of Science by Mehrdad Ghaziasgar This thesis investigates the use of mobile phones as service-delivery devices in a sign language machine translation system. Four sign language visualization methods were evaluated on mobile phones. Three of the methods were synthetic sign language visu- alization methods. Three factors were considered: the intelligibility of sign language, as rendered by the method; the power consumption; and the bandwidth usage associ- ated with each method. The average intelligibility rate was 65%, with some methods achieving intelligibility rates of up to 92%. The average file size was 162 KB and, on average, the power consumption increased to 180% of the idle state, across all methods. This research forms part of the Integration of Signed and Verbal Communication: South African Sign Language Recognition and Animation (SASL) project at the University of the Western Cape and serves as an integration platform for the group's research. In order to perform this research a machine translation system that uses mobile phones as service-delivery devices was developed as well as a 3D Avatar for mobile phones. It was concluded that mobile phones are suitable service-delivery platforms for sign language machine translation systems. Acknowledgements The past 6 years have been a journey, the longest most difficult journey of my life. Countless challenges and ordeals. But I am here. I thank God for this blessing. For His help and for growing me from frailty into strength, from inability into competence and from fear into conviction. I would to thank my parents. You toiled for me and supported me. You taught me so many things and this thesis has your essences in it. You believed in me and here I am. Thank you. I would like to thank Mr. Moses Kanaabi, my Grade 6 teacher. Sir, I don't know where you are, but this thesis has your essence in it. You believed in me when no one else did, not even myself. You found me and gave me direction and here I am. I would like to thank Mrs. Sharon Slinger. You believed in me and helped me excel and here I am. I would like to thank Dr. Roderick Julies. Sir, your positive outlook and your focus on transferring knowledge rather than discharging duty is praiseworthy. I enjoyed your lectures. I would like to thank the Rector of the University of the Western Cape Prof. Brian O'Connell. Sir, your positive, kind, humble, wise and helpful outlook is praiseworthy. Thank you for your help with the Imagine Cup. Thank you for everything. I would especially like to thank Mr. James Connan. Sir, your expansive knowledge, wisdom, creativity, confidence and positive outlook never cease to leave me in awe. Under your supervision, I went from frailty into conviction. Your balanced approach to supervision is exemplary. You supported in times when it was needed most. I have always, do always and will always speak of you in acclaim. I am indebted to you always. Thank you. I would like to thank all of the people that provided companionship during these six years. Afshin, Mohsen and Ayesha. Thank you. Fakhree, Theo, Riyaad and Badrudeen. Thank you. Nathan, we went through some amazing experiences together. From the Imagine Cup South Africa awards ceremony, to doing four or five absolute last minute, but expertly done, presentations to putting together the first ever two-way sign language to English translation system. Thanks for everything buddy. Jameel, we've been through some difficult times together. But here we are. Thank you for your friendship. iv Contents Declaration of Authorshipi Abstract iii Acknowledgements iv List of Figures ix List of Tables xi Abbreviations xiii 1 Introduction1 1.1 Background...................................1 1.2 Research Objectives..............................2 1.3 Research question................................3 1.4 Premises.....................................4 1.5 Thesis outline..................................4 2 Communication Tools for the Deaf6 2.1 General-Purpose Communication Tools...................7 2.1.1 Text-Based Tools............................7 2.1.1.1 IM systems..........................7 2.1.1.2 SMS..............................8 2.1.1.3 Email.............................8 2.1.1.4 Faxing............................8 2.1.1.5 Advantages and disadvantages of text-based tools....9 2.1.2 Video-Based Tools...........................9 2.1.2.1 Synchronous video communication tools..........9 2.1.2.2 Asynchronous video communication tools......... 10 2.1.2.3 Advantages and disadvantages of video-based tools.... 10 2.1.3 Bridging communication between devices.............. 11 2.2 Deaf-Specific Communication Tools...................... 13 2.2.1 Text-Based Tools............................ 13 2.2.2 Video-Based Tools........................... 14 v Contents vi 2.2.3 Gesture-Based Tools.......................... 16 2.2.3.1 Spoken Language to SL Gesture-Based Tools....... 17 2.2.3.2 SL to Spoken Language Gesture-Based Tools....... 21 2.3 Summary.................................... 22 3 Java Micro Edition 24 3.1 Java ME in Context { A Background on Java Editions........... 25 3.2 Java ME Architecture............................. 25 3.2.1 The Java Community Process (JCP)................. 26 3.2.2 Configurations............................. 28 3.2.2.1 The Connected Device Configuration (CDC)....... 29 3.2.2.2 The Connected, Limited Device Configuration (CLDC). 30 3.2.3 Profiles................................. 32 3.2.3.1 The CDC Profiles...................... 32 3.2.3.2 The CLDC Profiles..................... 33 3.2.3.3 The Mobile Information Device Profile.......... 33 3.2.4 Additional APIs............................ 35 3.2.4.1 The Mobile Media API................... 36 3.2.4.2 The Mobile 3D Graphics API................ 39 3.3 Summary and Conclusions........................... 43 4 Sign Language Avatar Creation 44 4.1 SL Avatar Creation............................... 44 4.1.1 Avatar Modelling and Acquisition.................. 45 4.1.1.1 Interactive Modelling.................... 45 4.1.1.2 Parametric Modelling.................... 46 4.1.1.3 Other Modelling Techniques................ 46 4.1.2 Avatar Parameterization and Deformation.............. 48 4.1.2.1 Skeletal Subspace Deformation............... 48 4.1.2.2 Other Parameterization and Deformation Techniques.. 49 4.1.3 Avatar Animation........................... 50 4.1.3.1 Keyframing.......................... 50 4.1.3.2 Motion Capture....................... 51 4.1.4 Existing Standards and Tools..................... 52 4.1.4.1 MakeHuman......................... 52 4.1.4.2 H-Anim............................ 52 4.1.4.3 MPEG-4 Facial Definition Parameters........... 53 4.2 Blender { an Open Source 3D Graphics Tool................ 54 4.2.1 Background............................... 55 4.2.2 Tools and Features........................... 56 4.3 Van Wyk's Methodology............................ 57 4.3.1 Methodology Overview........................ 57 4.3.2 Implementation of Man ........................ 58 4.3.3 Performance.............................. 60 4.4 Summary.................................... 62 5 Methodology and Implementation of the Avatars 63 Contents vii 5.1 Creation of the Avatar Phlank ........................ 64 5.1.1 Modelling and Acquisition....................... 64 5.1.2 Parameterization............................ 70 5.2 Animation and Exportation of Phlank and Man .............. 73 5.2.1 Animation................................ 73 5.2.2 Exportation............................... 74 5.3 Summary.................................... 76 6 Implementation of a Prototype Mobile Framework 77 6.1 The SASL Machine Translation System................... 78 6.1.1 Conceptual Overview......................... 78 6.1.2 Existing Components......................... 79 6.2 The iSign Mobile Framework Architecture.................. 80 6.3 Implementation of the iSign Mobile Framework............... 81 6.3.1 The Communication Protocol....................