AN INTERACTIVE SYSTEM to ENHANCE SOCIAL and VERBAL COMMUNICATION SKILLS of CHILDREN with AUTISM SPECTRUM DISORDERS by Maria Jose Minan
Total Page:16
File Type:pdf, Size:1020Kb
AN INTERACTIVE SYSTEM TO ENHANCE SOCIAL AND VERBAL COMMUNICATION SKILLS OF CHILDREN WITH AUTISM SPECTRUM DISORDERS by Maria Jose Minan A Thesis Submitted to the Faculty of The College of Engineering and Computer Science in Partial Fulfillment of the Requirements for the Degree of Master of Science Florida Atlantic University Boca Raton, FL December 2014 Copyright 2014 by Maria Jose Minan ii ACKNOWLEDGEMENTS I would like to express my sincere gratitude to my thesis advisor, Dr. Shihong Huang for her guidance, constructive criticism, friendly advice, and support dur- ing my graduate and undergraduate studies. My deepest gratitude to Dr. Mirjana Pavlovic and Dr. Emmanuelle Tognolli for their insight regarding Autism Spectrum Disorder. Special thanks to Dr. Daniel Raviv and Dr. Maria Petrie for taking the time to review my thesis and provide constructive feedback. My completion of this project could not have been accomplished without the support of my managers and co-workers at Belcan Engineering. I would also like to thank my family for be- ing encouraging and supportive during my academic career. Special thanks to my mother and grandmother for teaching me the importance of education and for their unconditional support. Additional thanks to Becca Huy and Michelle Huang for their professional opinions. Thank you to the Software Engineering Lab at FAU, including Ante Bosnjak, Jose Hurtado, Sheryl Lafleur, Pedro Miranda, and James Mulcahy for their support. Finally, thank you to all my friends for always encouraging me to achieve my goals. iv ABSTRACT Author: Maria Jose Minan Title: An Interactive System To Enhance Social And Verbal Communication Skills Of Children With Autism Spectrum Disorders Institution: Florida Atlantic University Thesis Advisor: Dr. Shihong Huang Degree: Master of Science Year: 2014 Affecting one in every 68 children, Autism Spectrum Disorder (ASD) is one of the fastest growing developmental disabilities. Scientific research has proven that early behavioral intervention can improve learning, communication, and social skills. Similarly, studies have shown that the usage of off-the-shelf technology boosts mo- tivation in children diagnosed with ASD while increasing their attention span and ability to interact socially. Embracing perspectives from different fields of study can lead to the development of an effective tool to complement traditional treatment of those with ASD. This thesis documents the re-engineering, extension, and evolu- tion of Ying, an existing web application designed to aid in the learning of autistic children. The original methodology of Ying combines expertise from other research areas including developmental psychology, semantic learning, and computer science. In this work, Ying is modified to incorporate aspects of traditional treatment, such as Applied Behavior Analysis. Using cutting-edge software technology in areas like voice recognition and mobile device applications, this project aspires to use software engineering approaches and audio-visual interaction with the learner to enhance so- v cial behavior and reinforce verbal communication skills in children with ASD, while detecting and storing learning patterns for later study. vi AN INTERACTIVE SYSTEM TO ENHANCE SOCIAL AND VERBAL COMMUNICATION SKILLS OF CHILDREN WITH AUTISM SPECTRUM DISORDERS List of Tables .............................. xi List of Figures ............................. xii 1 Introduction .............................. 1 1.1 Overview of Autism Spectrum Disorder.................1 1.2 Problem Statement............................3 1.3 Technology and ASD...........................3 1.4 Approach and Contribution.......................4 1.5 Thesis Organization............................8 2 Background ............................... 9 2.1 Autism Spectrum Disorder........................9 2.2 Common Symptoms........................... 10 2.3 Types of Autism Spectrum Disorder................... 11 2.3.1 Asperger's Disorder........................ 12 2.3.2 Autistic Disorder......................... 13 2.3.3 Childhood Disintegrative Disorder................ 13 2.3.4 Rett's Disorder.......................... 13 2.3.5 Pervasive Developmental Disorder-Not Otherwise Specified. 14 2.4 Diagnosis................................. 15 2.5 Traditional Treatments.......................... 16 vii 2.5.1 Applied Behavior Analysis.................... 16 2.5.2 Floortime............................. 17 2.5.3 Pivotal Response Therapy.................... 17 2.5.4 Verbal Behavior Therapy..................... 18 3 Research Focus ............................. 19 3.1 Verbal Communication.......................... 19 3.2 Social Communication.......................... 20 3.3 Adaptation................................ 20 4 Literature Review ........................... 22 4.1 Technology to Enhance Verbal Communication Skills in ASD Children 23 4.1.1 Interactive Applications..................... 23 4.1.2 Static Applications........................ 25 4.2 Technology to Enhance Social Communication Skills in ASD Children 28 4.2.1 Interactive Applications..................... 28 4.2.2 Static Applications........................ 31 4.3 Technology to Enhance Adaptation Skills of ASD Children...... 33 4.3.1 Interactive Applications..................... 33 4.3.2 Static Applications........................ 37 5 Methodology .............................. 39 5.1 Objective................................. 39 5.2 Approach................................. 40 5.2.1 Learn............................... 42 5.2.2 Play................................ 44 5.2.3 Schedule.............................. 48 5.2.4 Rewards.............................. 48 5.2.5 Progress.............................. 49 viii 5.2.6 Configuration........................... 50 6 Implementation ............................. 51 6.1 System Architecture of Ying....................... 51 6.1.1 Presentation Layer........................ 52 6.1.2 Business Logic Layer....................... 52 6.1.3 Data Layer............................ 54 6.2 Main Components............................ 55 6.2.1 Speech Recognition........................ 55 6.2.2 Text to Speech.......................... 56 6.2.3 Graphing............................. 57 6.2.4 The Avatar............................ 58 6.3 Modules.................................. 59 6.3.1 Learn............................... 60 6.3.2 Play................................ 61 6.3.3 Schedule.............................. 61 6.3.4 Rewards.............................. 62 6.3.5 Settings.............................. 62 6.3.6 Progress.............................. 63 6.4 Platform.................................. 64 6.5 Activities................................. 64 6.5.1 Virtual Flashcards........................ 64 6.5.2 Eye Gaze............................. 65 6.5.3 Story Teller............................ 66 6.5.4 Say my Name........................... 67 6.5.5 Take a Peek............................ 68 6.5.6 Message in a Bottle........................ 69 6.5.7 Tappy Time............................ 69 ix 6.5.8 Fill in the Blank......................... 70 7 Conclusion ............................... 71 7.1 Research Contribution and Summary.................. 71 7.2 Evaluation................................. 72 7.3 Future Areas of Development...................... 72 Appendices ............................... 74 A Evaluation Questionnaire......................... 75 A.1 Learning Activity 1: Virtual Flashcards............. 75 Bibliography .............................. 91 x LIST OF TABLES 1.1 Prevalence of ASD [1]...........................2 4.1 Existing apps for children with ASD.................. 23 xi LIST OF FIGURES 1.1 Welcome Page of Ying [29]........................5 1.2 High Level Overview of the Original Ying [29].............6 1.3 "Gazy Gazy" - Eye Contact Game [29].................7 2.1 Synapses are points of communication between two nerve cells [46]. 10 2.2 Brain Cell of an autistic patient (left) and a non-autistic person(right) [46]..................................... 10 2.3 Types of Autism Spectrum Disorders.................. 12 2.4 Types of ASD and Functioning Levels [18]............... 15 2.5 Therapy Session [62]........................... 16 4.1 Classification of Software for Children with ASD............ 22 4.2 Vocabulary - Proloquo2go [63]...................... 23 4.3 Proloquo2go Features [64, 63]...................... 24 4.4 Results of the Evaluation of Proloquo2go [63]............. 25 4.5 Vast Autism 1 Core Main Menu (Left), Modeling Video (Right) [52] 26 4.6 Video Categories [52]........................... 27 4.7 Training Faces Interface [53]....................... 28 4.8 Training Faces Emotion Recognition Process............. 29 4.9 Tom Cat [37]............................... 30 4.10 The Transporters............................. 32 4.11 First Then Visual HD Icon........................ 33 4.12 First Then Visual HD Interface..................... 34 4.13 Schedule and instructions to complete a task [67]........... 34 xii 4.14 AutisMate's User Interface [68]..................... 36 4.15 Zac Browser................................ 37 4.16 YouTube video accessed through Zac Browser [9]........... 37 5.1 Logo of the Ying system......................... 40 5.2 Overview of Ying............................. 41 5.3 Virtual Flashcards for Words...................... 42 5.4 Virtual Flashcards for Phrases...................... 43 5.5 Story Teller................................ 44 5.6 Eye Gaze................................. 45 5.7 Say My Name..............................