Autism, New Music Technologies and Cognition by Adam Boulanger B.A. Music Therapy Berklee College of Music (2004) Submitted to the Program in Media Arts and Sciences, School of Architecture and Planning, in partial fulfillment of the requirements for the degree of MASSACHUSETTS INS1TTWR OF TECHNOLOGY Master of Science in Media Arts and Sciences SEP1 2006 at the MASSACHUSETTS INSTITUTE OF TECHNOLOGY LIBRARIES September, 2006 @ Massachusetts Institute of Technology 2006. All rights reserved. ROTCH Author................................................................. Program in Media Arts and Sciences August 16, 2006 Certified by........................................................-.-. Tod Machover Prof r of Music and Media Thesis Supervisor A ccepted by ............................................ ... -- Andrew Lipp Chairman, Department Committee on Graduate Students Autism, New Music Technologies and Cognition by Adam Boulanger Submitted to the Program in Media Arts and Sciences, School of Architecture and Planning, on August 16, 2006, in partial fulfillment of the requirements for the degree of Master of Science in Media Arts and Sciences Abstract Central coherence accounts of autism have shown dysfunction in the process- ing of local versus global information that may be the source of symptoms in social behavior, communication and repetitive behavior. An application was developed to measure cognitive abilities in central coherence tasks as part of a music composition task. The application was evaluated in collaboration with the Spotlight Program, an interdisciplinary social pragmatics program for chil- dren with Asperger's syndrome. This research indicates that it is possible to embed cognitive measure as part of a novel music application. Implications for current treatment interventions, and longitudinal experimentation designs are presented. Thesis supervisor: Tod Machover Title: Professor of Music and Media Thesis Committee Thesis supervisor........................... .......... .... Tod.......e. lu Tod Machover Professor of Music and Media MIT Program in Media Arts and Sciences Thesis reader........ .................................. .. Gottfried Schlaug, M.D., PLY. Associate Professor of Neurology Beth Israel Deaconess Medical Center, Harvard Medical School Thesis reader .................................... ...... Mg u Ph.D. Mriganka Sur, Ph.D. Department Head Brain and Cognitive Sciences, MIT T hesis read er............................................................................. Antonio Damasio, M.D., Ph.D. Dornsife Professor of Neuroscience and Director Brain and Creativity Institute, USC Thesis Committee T hesis supervisor......................................................................... Tod Machover Professor of Music and Media MIT Program in Media Arts and Sciences T hesis reader............................................................................. Gottfried Schlaug, M.D., Ph.D. Associate Professor of Neurology Beth Israel Deaconess Medical Center, Harvard Medical School Thesis reader.................................................................... Mriganka Sur, Ph.D. Department Head Brain and Cognitive Sciences, MIT T hesis reader............................................................................. Antonio Damasio, M.D., Ph.D. Dornsife Professor of Neuroscience and Director Brain and Creativity Institute, USC Acknowledgments The construction of this thesis was made possible by the generous support of many friends, colleagues, and mentors. In particular, I'd like to thank the following people: My advisor Tod Machover, who has supported this work since the beginning, providing tremendous opportunities to entertain ideas, applications, and new directions. My thesis readers Antonio Damasio, Gottfried Schlaud, and Mriganka Sur. Matt Lerner and Krystal Demaine, brilliant practitioners serving the autistic population, committed to new models of intervention. The students of the Spotlight Program. Fellow students of the Hyperinstruments Group. My family. And most importantly, Cati Vaucelle, with whom everything is possible. Acknowledgments 9 Table of Contents 1 Introduction and Motivation 19 1.1 An Opportunity in Music Therapy and Autism ... .. ... 20 1.2 A Technological Solution .. .. ... .. .. .. .. .. .. .. 22 2 Literature Review 23 2.1 Autism Spectrum Disorders .. .. .. .. .. .. .. .. .. 23 2.1.1 Neuropathology . .. .. .. .. .. .. .. .. .. .. 24 2.1.2 Perception and Cognition . .. .. .. .. .. .. .. 26 2.2 Music Therapy . .. .. .. .. .. .. .. .. .. .. .. .. .. 32 2.2.1 A Brief History of Music Therapy . .. .. .. .. 33 2.2.2 Music Therapy and Autism .. .. .. .. .. .. 37 2.3 Cognitive Training .. .. .. .. .. .. .. .. .. 38 2.4 Summary . .. .. .. .. .. .. .. .. 43 3 Initial Work 45 3.1 The Hyperinstruments Group . .. .. .. .. .. 45 3.1.1 The Hyperscore Application .. .. .. .. 46 3.1.2 Hyperscore at Tewksbury State Hospital.. .. .. 48 3.1.3 Differences Between Hyperscore and a Clinical Tool . 49 3.2 Paired Associates Learning and AD......... ..... 52 3.2.1 Rationale....... .. ........ .......... 52 3.2.2 D esign .................... 53 3.2.3 Research Questions Raised ......... 55 3.3 Sum m ary ....... ........................ 56 4 System Design and Implementation 57 4.1 Melodic Contour Task ................... 58 4.1.1 Interface and Interaction.......... .... .. 58 4.1.2 Delayed Matching to Sample ................ 61 4.2 Embedded Rhythms Generator ................... 64 4.2.1 Interface and Interaction.......... .... .. 65 4.2.2 Embedded Figures Test ...... ............. 66 4.3 Technical Implementation..... ................. 69 4.3.1 M ax/M SP ........ ........... 69 4.3.2 C sound ........ ....... ....... ...... 73 4.4 Sum m ary .... ........ ........ ........ ... 74 5 Evaluation 75 5.1 The Spotlight Program. ........... ........... 76 5.2 Asperger's Syndrome....... ...... ..... ... .. 77 12 Contents 5.3 The Interface .. ... .. ... .. .. ... .. .. ... .. .. 78 5.4 Interface Issues and Perspective Taking . .. .. ... .. .. .. 80 5.5 Repurposing for Intervention .. ... .. .. ... .. .. ... 82 5.6 Adaptability . .. .. .. .. ... .. .. .. .. .. .. .. ... 83 5.7 Sum m ary . .. .. ... .. .. .. .. .. .. .. ... .. .. .. 84 6 Conclusion and Future Work 85 6.1 From Reductionism to Collaborative Opportunities . .. .. .. 85 6.2 Integration Into Current Treatment Practices . .. .. .. ... 88 6.3 Final Remarks . .. .. .. .. .. .. .. .. .. .. .. .. 88 Appendix A 91 A.1 Purpose of Longitudinal Study ... .. .. .. .. ... .. .. 91 A.2 Study Protocol .. .. .. .. .. .. .. .. .. .. .. .. .. 91 A.3 Human Subjects .. .. .. .. .. .. .. .. .. .. .. .. 94 A.3.1 What are the criteria for inclusion or exclusion? . .. 94 A.3.2 Are any inclusion or exclusion criteria based on age, gen- der, or race/ethnic origin? . .. .. .. .. .. .. 94 A.3.3 Explanation of any vulnerable population, and why that population is being studied. .. .. .. .. .. .. 94 A.3.4 Subject recruitment .. .. .. .. .. .. 95 A.3.5 Potential risks . .. .. .. .. .. .. .. .. .. .. 95 A.3.6 Potential benefits . .. .. .. .. .. .. .. .. 96 A.3.7 Data collection, storage and confidentiality . .. .. .. 96 Contents 13 Bibliography 99 14 Contents List of Figures 2-1 An algorithm for autism diagnosis, from [Filipek et al., 2 0 0 0] . ........... ........... .......... 27 2-2 Anatomy of a block design task, from [Happe, 1999] . 29 2-3 Sample CPT codes from a music therapy session with an individual with autism..................... 38 3-1 Hyperscore motive and sketch windows, taken with per- mission from [Farbood, 2006] .................. 47 3-2 Translation of the melody in 3-1 through the given sketch window, taken from [Farbood, 2006] ................ 47 3-3 Software implementation of controller for Paired Asso- ciates software. The black square is a cartesian plane allowing users to click and drag the mouse in two di- mensions, which is mapped to two parameter values. .. 53 3-4 Visual stimuli created simultaneously while editing sound. 54 4-1 The interface transport, for specifying features of notes to be entered into the melody window. Buttons, from left to right, change note duration, loudness, selected instrument, and control melody playback. The knob changes one timbre parameter of the selected instru- ment. The bottom button, marked save, writes the en- tered melody to the user's library of melodies, and ini- tiates the contour task. ............................ 59 4-2 Window for notating melodies in piano roll format. Y- axis corresponds to pitch. X-axis corresponds to time of note excitation and note length. .. ....... ....... 60 4-3 Flow diagram depicting the user interaction with the Melodic Contour Generator. .............. .... 61 4-4 Formula for smoothing a curve using a moving average filter. ................................. 62 4-5 The resultant curve after using a moving average filter with three different constants for window size n. ..... 63 4-6 The interface window where the DMS task takes place. One of the four lines is the correct contour for the melody input by the user in the melody window. The other three are incorrect contours. Users must select the best fitting contour for the melody they created without being able to refer back to the original piano roll notation, where the melody was entered. .... ................. 64 4-7 The window where users operate the embedded rhythms application. Users are able to save the rhythms they create, respond to task questions, receive instructions, receive feedback, and reset the application to enter