Computer Assisted Autism Interventions for India

Computer assisted autism interventions for India

Citation of the final article: Vellanki, Pratibha, Greenhill, Stewart, Duong, Thi, Phung, Dinh, Venkatesh, Svetha, Godwin, Jayashree, Achary, Kishna V and Varkey, Blessin 2016, Computer assisted autism interventions for India, in OzCHI 2016: Proceedings of the 28th Australian Conference on the Human-Computer Interaction, Association for Computing Machinery, New York, N.Y., pp. 618-622.

This is the peer reviewed accepted manuscript.

This is the author's version of the work. It is posted here by permission of ACM for your personal use. Not for redistribution. The definitive version was published in OzCHI ’16: http://www.dx.doi.org/10.1145/3010915.3011007

Downloaded from DRO: http://hdl.handle.net/10536/DRO/DU:30093647

DRO Deakin Research Online, Deakin University’s Research Repository Deakin University CRICOS Provider Code: 00113B Computer Assisted Autism Interventions for India Pratibha Vellanki1 , Stewart Greenhill 1 , Thi Duong1 , Dinh Phung1 , [email protected] [email protected] [email protected] [email protected]

Svetha Venkatesh1 , Jayashree Godwin2 , Krishna V. Achary2 , Blessin Varkey2 , [email protected] [email protected] [email protected] [email protected]

1Centre for Pattern recognition and Data Analytics, Deakin University, Australia 2Tamana School of Hope, New Delhi, India

ABSTRACT expand language skills from receptive to expressive Early intervention is critical for children with autism. To language, and maintain learning environments that sustain provide affordable computer assisted therapies for interest. TOBY already includes these aspects, and thus is developing countries, we construct infrastructures for a state-of-art therapy. translating and adapting early intervention programs such as TOBY to an Indian context. A Hindi prototype is built This work describes the adaptation of TOBY to the and two trials are conducted, showing that the technology language (Hindi), culture and contextual setting (in a was accepted and that the children learnt skills using both school) of India. Early intervention needs precision, so language versions, with the children using the Hindi domain experts are required for translation, and we build prototype achieving slightly better measurable outcomes. systems to facilitate this process. We construct: a) technologies to organise media, maintaining dependency Author Keywords on language and culture, b) collaborative environments to Autism, early intervention, India, Hindi, translation, facilitate a team of experts to create new tasks, c) tools to assistive technology. manage translation of tasks and interface. ACM Classification Keywords Using this framework, a Hindi and an English prototype H5.m. Information interfaces and presentation (e.g., HCI): was developed for India. A 3-person team from India Miscellaneous. adapted a subset of 19 skills and trialled the system. An INTRODUCTION initial trial studied iPad acceptance in an Indian setting Autism touches lives of children, their families and with 32 children. A second trial examined learning with 16 communities. The estimated societal cost is $3.2 million children using the English and Hindi prototypes. Our during the lifespan of an individual with autism spectrum results show that our translational goals are met - the disorder (ASD) (NAC 1990). ASD diagnosis is on the rise, children accept both the technology and the therapists, with 1 in 89 children diagnosed with ASD (Baio 2012). progressing through the syllabus. Such systems are Thus of the 158 million children aged 0-6 living in India significant because they empower therapists in developing (MOSPI 2012), there may be 1.8 million children with countries to deliver early intervention during critical stages ASD. of development. Importantly, they provide an affordable option for the first time. In autism, early intervention is crucial, but support is sparse even in developed countries (FaHCSIA 2016). FRAMEWORK Without assistance children may face lifelong cognitive The TOBY syllabus and social disability. In developing countries problems are TOBY Playpad is an autism therapy app for iPad based on compounded by social and economic handicap, and lack of Applied Behaviour Analysis (ABA). In ABA therapy we support. Urgent solutions are required to deliver timely and present a stimulus to which the child responds. If the child cost effective early interventions. gives the desired response we give reinforcement in the TOBY is a recent technology (Venkatesh et al. 2013) that form of verbal praise, or a favourite toy or activity. uses iPads to deliver therapy to children, in English, and in Otherwise, we prompt the child by demonstrating the Western context. It allows parents and teachers to target correct response. Prompting is faded over time so that daily skill development within the home or school eventually the responses are self-initiated. environment, and allows therapists to monitor progress The TOBY syllabus is divided into four major areas: remotely. It delivers a program spanning a range of early developmental skills - imitation, sensory matching, a) Sensory includes skills related to perception and receptive and expressive language, and social skills. It uses discrimination of sensory cues, such as colour, shape, a mix of activities on and off the iPad, providing rigorous same-ness and difference; b) Imitation includes copying recording of stimuli and responses, adapting the program actions, shapes, or producing pre-speech sounds; c) to the child's progress. Trials show promising learning Language involves recognising and producing the names outcomes (Moore et al. 2013). for objects; d) Social includes interpersonal skills such as making eye contact, and joint-attention. A recent review considers technological applications assisting therapy (Wainer & Ingersoll 2011) and identifies For some therapy tasks such as picture matching, the future program developments. This includes programs that system can present the stimulus, measure the response, extend delivery of exclusively computer assisted programs give reinforcement and prompting. The child performs to include teacher-led and computer-led interactions, these “solo” tasks without assistance. The remaining

b) Adaptation to language constraints. Some items need to be changed because of language constraints. For example several words in English map to one word in Hindi. There is only one word for “ankle” and “leg” in Hindi, so one category is removed in the Hindi version. c) Adaptation to differences in function. Some items have different utility in the target country - for example, a bucket can be a toy in Australia, but is a household item in India. Thus the sub-category was reclassified. The following sections describe the implementation of changes in specific parts of the syllabus. Online syllabus management system for text The largest resource that needs translation are instructions to the parent or teacher to do NET activities. The TOBY Figure 1. Syllabus manager, Task edit form syllabus covers 51 skills through a mixture of play and adaptive NET activities. There are 292 NET tasks in total, roughly 6 per skill. Each task contains approximately 260 words, with a total of 77500 words for the entire syllabus. NET tasks include instructional text, formatting, and structure data. The text for each task includes a goal, materials, activity, prompts, and reinforcement. We built a web-based system to manage the translation task. We imported the TOBY English syllabus, and created an initial version of the Hindi syllabus using a machine translation service. While these results were poor, they provided a base to be refined by a human translator. Figure 1 shows the layout of the task view in the syllabus manager. The left panel shows the document attributes and the translated text which can be edited via the form. The Figure 2. Syllabus manager, Preview of NET task right panel shows attributes and text of the original “partner ” tasks involve a parent or a teacher: to measure document. The system verifies that essential items are the child's response, to prompt the child, or to present a specified. When an edit is completed it enters “review” stimulus. Another category of “partner” tasks involving state, and must then be accepted by a different user. routines such as meal time and bathing, is done away form the iPad in a natural setting. This Natural Environment The syllabus manager also offers a preview function (see Training (NET) helps to generalise and transfer skills into Figure 2), which launches TOBY in the web browser using real-world situations. the live syllabus data. This allows the editors to see how their tasks will appear on the iPad screen. This is useful for Customising content and language checking layout and readability. If a task is too wordy to TOBY uses a tree-like category schema to organise fit on a single page, then the editor can see what can be content. The main categories are nouns (food, personal simplified to improve the layout. items, household items, outdoor, clothes, toys, animals and body parts) and Skills (Gross-motor Imitation, Actions Adaptation to cultural context of use - the classroom with Objects, etc). To translate the language and culture we TOBY was originally designed for home settings, and must identify objects or tasks that not in common use in a many adaptive tasks are based on familiar routines such as culture or are constrained by either target language or bath time, and meal time. This means that adaptive functional utility. Categories are included, or added based routines may not be suitable for use in schools, where the on culture. For example, “meat” is excluded from the daily routines are different. Even in a home setting, Indian version of “food” and replaced by “dhal” and routines may vary with culture. “dosa”. For the Indian version, 19 categories were added The Hindi trial syllabus includes 7 NET skills, including and 40 categories were removed. The following outlines 26 adaptive tasks and 19 play tasks. Of the 26 tasks, 21 the reasons for these changes. were found to be related to home-based routines that may a) Adaptation to cultural aspects. We identified sub- not be appropriate for school settings. Thus an additional categories that are not in colloquial use. Between Australia 21 NET tasks were written by modifying the existing tasks and India, the categories that were culturally independent for a different school-based routine. For example, a home- included personal items, and household items. The based routine for sensory matching involves matching categories that were culturally specific included food (6 objects such as cups and spoons at meal time. For a school added, 7 removed), clothes (+3,-5), toys (+3,-6), animals setting the corresponding task involves matching art (+5,-4), and outdoor objects (+1,-7). materials (paper, pencils, glue) during art activities. Table

2 Table 1. Examples of home routines that were modified for a school environment Skill Home Materials School Materials Matching Meals cups, Art art supplies Objects cutlery Expressive Dressing clothes Music musical Nouns instruments

Gaze Bathing bath toys Reading books, Sharing animal toys Figure 3. The cardboard Figure 4. Instructor settles child Matching Washing towels, dance scarves iPads and flash cards to work Colour socks children started to work on TOBY. Most children were 1 shows examples of some of the adaptations that were delighted with the iPads and no iPads were damaged. made. Children in the early intervention group (under 6 years old) Adaptation of multimedia content were most responsive to TOBY and found the built-in TOBY uses voice prompts to introduce the names for rewards motivating. But their attention spans varied objects, prompt the user during solo tasks, and provide widely. The preparatory junior and senior (8-15 years old), feedback and reinforcement. For the translation we and high functioning group (6-7 years old) quickly warmed recorded approximately 220 Hindi words, which includes to the new TOBY instructors when using the iPad. Some all of the nouns and prompts used for receptive language boredom was noted in the high functioning group, who tasks. We also recorded 19 new English words, which wanted to play more difficult “games”. Some children correspond to the categories added for the Indian region. recognised TOBY instructors and searched for iPads even outside TOBY sessions. Others needed external rewards to TOBY uses video modelling for imitation tasks, which are continue their work on TOBY. used to teach skills in gross motor, fine motor, and oral motor imitation. Imitation tasks include an average of 13 Trial 2 - Evaluation of TOBY syllabus adaptations actions, with a range between 4 and 30 actions. TOBY The second trial was conducted for 6 weeks between July includes 174 imitation videos with 100 actions and 74 and August 2013 at the Tamana School. Both Hindi and echoic (oral-motor) videos. The videos use actors that English TOBY programs have the same syllabus. 16 would look normal for the target country. We recorded 67 children were selected for the trial - 10 to work on TOBY new imitation videos, for the skills included in this trial. in Hindi, and 6 in English. This grouping was based on the child's mother tongue. The cohort consisted of children TOBY uses images for sensory, receptive and expressive from 3 to 16 years, with the majority (12 children) between language training. For the new categories we added 73 new 6 and 11 years. exemplar images, drawn from original photographs and creative-commons licensed material sourced from web Qualitative Analysis sites like Wikipedia and Flickr. TOBY adaptation: Several adaptations were introduced to TOBY, including new videos for imitation, and new EXPERIMENTS images/words in both Hindi and English. Children did not Trial 1 - Study of iPad acceptance have a preference for Indian or Caucasian persons giving In April 2013 we approached parents of 47 children who instructions in video. It was also observed that some Hindi were diagnosed with autism (DSM-IV 1994), more than words are difficult for children. For example, “nashpati” three years old, and enrolled at Tamana School of Hope in (i.e. “pear”) is simple in English but uncommon in Hindi. New Delhi, India. 32 children aged 3 to 14 years were Thus word ordering in Hindi needs to be different from recruited. The cohort had markedly different functioning English. levels - (13/16) are verbal, with language ability ranging TOBY session: Each TOBY session started with iPad from single words to some fluency, and (3/16) are activities, then moved to NET tasks and ended with nonverbal. “rhyme time”. Children preferred iPads tasks, so on The children attended school every weekday and worked occasions they were encouraged to finish more iPad on TOBY either in their normal classroom or in the special activities in the time allocated for NET. Instructors found TOBY research lab. Each child was given a 15-20 minute iPad tasks were easier to implement, and this is indicative time slot depending on attention span and ability. On of the fact that NET requires experience in delivery. TOBY average, 15 sessions were given for each child. instructors at Tamana were given only online training and support prior to the trial. The online training modules were When iPads were initially introduced to the children, there perceived as being complicated, and the Australian accent were some cases of banging, hard tapping, and licking the was found challenging. devices. Hence, cardboard models of iPads (Figure 3) with flash cards were introduced to desensitise the children to Despite these drawbacks, the instructors made meaningful such devices and build trust. adaptions while implementing NET. For example, imitation and gaze sharing were incorporated into “rhyme After familiarity was built with instructors and the time”. These successful adaptions helped TOBY's goal of cardboard iPads, real iPads were re-introduced and the generalising skills via daily routines.

The children: After 2-3 weeks most children were able to �� make connections between iPads, soft toys (NET �� materials, rewards) and “rhyme times”. Some children occasionally asked for “rhyme time” before leaving the �� TOBY lab. Most children warmed to the iPads and instructors in the first few sessions, apart from the �� following two cases. One child refused to sit down. After, a few sessions instructors found an external reinforcement ��

- a tiger toy - that helped him to settle and work. Another �� child liked to run. After 2-3 weeks, she managed to sit �� down, sometimes in the instructor's lap (Figure 4). �� Quantitative Analysis �� Summary of language spoken at home: Most children Figure 5. Trial 2: Skill completion rates for English and (10/16) came from homes where both English and Hindi Hindi children. are spoken. Only 6 children came from homes that spoke From the therapist's perspective, three main outcomes were only Hindi. Children who did the English TOBY came demonstrated: (a) The syllabus material could be translated from dual-language speaking homes. in a decentralised way, by non-experts. The trial was iPad Access and skill improvement: Limited access to completed satisfactorily in both English and Hindi. This iPads was noted. Only 4 children had iPads at home and of was achieved in spite of the trying working conditions. (b) these only 3 had used iPads before the trial. Among the Both Hindi and English users were able to progress remaining 12 children without iPads at home, only 1 had through tasks in 6 weeks. Measurable learning outcomes access to iPads before the trial. Hence, there were a total of demonstrated that the children progress through nodes 12 children who did not use iPads before the trial. with mastery. Children spent between 5 - 12 hours in doing iPad tasks. Children who spent more time progressed The children were rated pre- and post-trial with respect to further in the syllabus. (c) There is some evidence that their pointing and dragging skills (1 indicates no ability to progress was slightly better in Hindi, so it is worth demonstrate skill, 5 indicates fluency). After the trial, half completing a full Hindi version of the syllabus. the children (7/12) had a rating above 4. The children’s ability to drag items on iPads had similar results: almost The main suggestions for future work is that the order of half (8/12) achieved ratings above 3. “stimulus words” in English is not optimal when translated into Hindi. The order of Hindi words needs to be adjusted Analysis of Performance data: Children in the trial worked for complexity and cultural popularity. Some NET with TOBY for approximately 30 minutes each day, on activities were sent home to parents, but no feedback was tasks on and off the iPad. The total times ranged between received. This suggests more support is required for 5 and 12 hours. A significant amount of time, up to 50% of parents to appreciate the value of NET. A separate module the total, was spent between management functions to train parents needs to be built. Furthermore, a trial with (logging in and out of the system) and on the plan screen rigorous pre- and post-intervention measures and/or (choosing activities, displaying report summary). Usage on control group would be needed to further evaluate the NET was generally low, but this is to be expected as NET effectiveness of TOBY technologies in India. tasks are performed off the iPad, and therefore are not recorded by the system. CONCLUSION We construct a system to translate and adapt the TOBY Figure 5 shows the variation of success rates between the computer assisted early intervention program to India. We English and Hindi groups. This is the proportion of the build frameworks for translation and creation of new tasks children who completed the task out of the set of children and content, caring for cultural context. We conduct two who attempted the task. Each point represents one TOBY trials and show that the children were able to learn using skill, and the horizontal and vertical positions are the the translated prototype in Hindi effectively. We show that success rates for the English and Hindi groups. Note the the therapists were able to deliver the early interventions clustering in the upper part of the graph, indicating that in spite of difficult work in conditions in India, and that relatively more success is achieved by the Hindi group. they formulated innovative adaptations in the intervention. The probability that this variation is due to chance alone is 0.10 for iPad skills, and 0.082 for NET skills (single-tailed REFERENCES paired Student's T scores). Helping Children with Autism (FaHCSIA), cited June 2016. https://www.dss.gov.au/our- DISCUSSION responsibilities/disability-and-carers/program-services/for- We consider the translational aims to have been achieved. people-with-disability/early-intervention-services-for- From the child's perspective, two broad outcomes were children-with-disability achieved: the technology and the therapy was accepted. American Psychiatric Association. Diagnostic and The iPads were used without any incident, which is statistical manual of mental disorders: DSM-IV, 4th. ed. significant given the socio-economic context. No iPads Washington, DC, 1994. were broken, and all were used appropriately. The therapy was accepted as the children warmed to people delivering Baio, J. Prevalence of Autism Spectrum Disorders: Autism therapy on the iPad. and Developmental Disabilities Monitoring Network, 14

4 Sites, United States, 2008. Morbidity and mortality National Autism Centre. National Standards Report. weekly report. Survelience summaries. Volume 61, Massachusetts, USA, cited June 2016. number 3. Centers for Disease Control and Prevention http://www.nationalautismcenter.org/ (2012). Venkatesh, S., Phung, D., Duong, T., Greenhill, S., and Ministry of Statistics and Programme Implementation, Adams, B. TOBY: early intervention in autism through Government of India. Children in India 2012 - A technology. In Proceedings of the SIGCHI Conference Statistical Appraisal, cited June 2016. on Human Factors in Computing Systems ACM (2013), http://mospi.nic.in/mospi_new/upload/Children_in_India_2 3187--3196.

012.pdf Wainer, A.L., and Ingersoll, B.R. The use of innovative Moore, D., Venkatesh, S., Anderson, A., Greenhill, S., computer technology for teaching social communication Phung, D., Duong, T., Cairns, D., Marshall, W., and to individuals with autism spectrum disorders", Whitehouse, A. TOBY Playpad application to teach Research in Autism Spectrum Disorders 5, 1 (2011), pp. children with ASD: A pilot trial. Developmental 96—107. Neurorehabilitation (2013).