Assessing Text and Web Accessibility for People with Autism Spectrum Disorder Victoria Yaneva A thesis submitted in partial fulfilment of the requirements of the University of Wolverhampton for the degree of Doctor of Philosophy 2016 This work or any part thereof has not previously been presented in any form to the University or to any other body whether for the purposes of as- sessment, publication or for any other purpose (unless otherwise indicated). Save for any express acknowledgements, references and/or bibliographies cited in the work, I confirm that the intellectual content of the work is the result of my own efforts and of no other person. The right of Victoria Yaneva to be identified as author of this work is as- serted in accordance with ss.77 and 78 of the Copyright, Designs and Patents Act 1988. At this date copyright is owned by the author. Signature: . Date: . Abstract People with Autism Spectrum Disorder experience difficulties with reading comprehension and information processing, which affect their school perfor- mance, employability and social inclusion. The main goal of this work is to investigate new ways to evaluate and improve text and web accessibility for adults with autism. The first stage of this research involved using eye-tracking technology and comprehension testing to collect data from a group of participants with autism and a control group of participants without autism. This series of studies resulted in the development of the ASD corpus, which is the first multimodal corpus of text and gaze data obtained from participants with and without autism. We modelled text complexity and sentence complexity using sets of fea- tures matched to the reading difficulties people with autism experience. For document-level classification we trained a readability classifier on a generic corpus with known readability levels (easy, medium and difficult) and then used the ASD corpus to evaluate with unseen user-assessed data. For sentence- level classification, we used for the first time gaze data and comprehension testing to define a gold standard of easy and difficult sentences, which we then used as training and evaluation sets for sentence-level classification. The ii results showed that both classifiers outperformed other measures of complex- ity and were more accurate predictors of the comprehension of people with autism. We conducted a series of experiments evaluating easy-to-read documents for people with cognitive disabilities. Easy-to-read documents are written in an accessible way, following specific writing guidelines and containing both text and images. We focused mainly on the image component of these doc- uments, a topic which has been significantly under-studied compared to the text component; we were also motivated by the fact that people with autism are very strong visual thinkers and that therefore image insertion could be a way to use their strengths in visual thinking to compensate for their difficul- ties in reading. We investigated the effects images in text have on attention, comprehension, memorisation and user preferences in people with autism (all of these phenomena were investigated both objectively and subjectively). The results of these experiments were synthesised in a set of guidelines for improving text accessibility for people with autism. Finally, we evaluated the accessibility of web pages with different levels of visual complexity. We provide evidence of existing barriers to finding relevant information on web pages that people with autism face and we explore their subjective experiences with searching the web through survey questions. iii Acknowledgements Four years ago I came to the University of Wolverhampton to spend a month working on a project related to text simplification for people with autism. Little did I know that this one-month placement would be the start of a journey in academia and that today I would be writing the acknowledgments section of a PhD thesis. That placement did something even more important for me: it introduced me to the world of autism, which I am now so fascinated with. There are so many people I have to thank: Professor Ruslan Mitkov, for encouraging me to start this PhD in the first place and for believing in me. Thank you for your faith in me, for giving me the academic freedom to follow my own ideas and for all your support; Professor Kenneth Manktelow, for always being there for me and for showing me that statistics can actually be interesting. After each meeting with you I have felt more relaxed and confident that any problem could be solved; and Dr. Irina Temnikova for being available in times of crises via all forms of social media in almost all hours of the day and night. You have provided me with very valuable support and advice (and occasionally with a drink, in case the first two did not work out, for which I am grateful, too!). iv A huge thank you goes to Dr. Miguel Angel Rios Gaona: Miguel, I have learnt so much from you that I simply cannot imagine finishing this PhD without you by my side. Thank you for all your valuable ideas, your criticism, your code for the randomised presentation of the texts and questions, and last but far from least, for all the cooking and all the “fiestas locas", which made Wolverhampton a warmer place. I deeply regret that, because of confidentiality, I am not able to name the dozens of participants who took part in this research. Working with you and getting to know the everyday battle some of you face coping with autism has been a transformative experience for me. It has turned this work from a mere research topic into a cause. Thank you for getting involved despite not having any incentive other than to help other people with reading difficulties. I promise I will work really, really hard to achieve this with you. The work presented in this thesis was dependent on the involvement of the community and of many charity organisations. Special thanks in partic- ular go to Nick Parry, Pawel Cwik and Emilia-Maria Greenwood for helping me find more and more participants during the most difficult period of my PhD. Autism West Midlands, represented by Dr. Elisabeth Hurley, Joanne Barford and Stephanie Taylor, deserve applause for their commitment, for their support of this project and for teaching me so much about autism. Others who offered me valuable help include Linda Cooper from the Goscote Greenacres Centre and the entire team of the Impact Hub in Birmingham, who helped me reach out to the public and widened the impact of this work. v I am also very grateful to all of the members of our research group. Dis- cussions with Richard Evans and Constantin Orasan have been crucial in shaping the ideas presented in this thesis. Special thanks go to Iain Mansell for his genuine enthusiasm in helping with the outreach of this work and to Emma Franklin and George Mitkov for proofreading hundreds of pages. I met some great people in the group and, as I write these lines, I feel quite nostalgic for times past. There are too many people to mention here and I do not want to single anyone out as I have really learnt a lot from each and every one of you. You all know who you are and I am very happy that we got to share a small part of our lives together in Wolverhampton. Finally, there are a handful of people who are not directly related to this research but who are very dear to me: My friend Sachin Sasikumar, for helping me keep a decent work-life balance and for all his songwriting; and my friends who are so far away and yet so close: Petko, Borislav, Venera, Ani, Jeni, Elena, Polina and Dobrin. Thank you, Simeon Ivanov, for being certain that I could achieve whatever I wanted to. This belief of yours has turned into a self-fulfilling prophecy for me and, even though I am not as powerful as that, I could not have done without you and I am very glad to have known you. Thanks, too, to my sister Yana and to my parents: I will not even attempt to put into words what you mean to me. Last but extremely far from least, many special thanks go to Aaron Montgomery for... well, for putting up with me through this entire writing period (I know I am a handful) and for being the wonderful human vi being that I am lucky enough to have by my side. vii List of Publications Parts of this thesis appeared in the following peer-reviewed publications: • Yaneva, V., Mitkov, R., Orasan, C. and Temnikova, I. Can Image In- sertion Aid Reading Comprehension and Memorisation in Adults with Autism? User Studies, A Prototype and Guidelines. Under Review, 2017. • Eraslan, S., Yaneva, V., Yesilada, Y., Harper, S. and Mitkov, R. Web Users with Autism: Eye-tracking evidence of Barriers and Distractors. Under Review, 2017. • Yaneva, V., Temnikova, I. and Mitkov, R. 2016. A Corpus of Text Data and Gaze Fixations from Autistic and Non-autistic Adults. Pro- ceedings of the 10th edition of the Language Resources and Evaluation Conference (LREC), Portoroz, Slovenia, 25 - 28 May • Yaneva, V., Temnikova, I. and Mitkov, R. 2016. Evaluating the Read- ability of Text Simplication Output for Readers with Cognitive Dis- abilities. Proceedings of the 10th edition of the Language Resources and Evaluation Conference (LREC), Portoroz, Slovenia, 25 - 28 May • Yaneva, V., Evans, R. and Temnikova, I. 2016. Predicting Read- ing Difculty for Readers with Autism Spectrum Disorder. Proceed- ings of Workshop on Improving Social Inclusion using NLP: Tools and viii Resources (ISI-NLP) held in conjunction with LREC 2016, Portoroz, Slovenia, 23 May • Yaneva, V., Temnikova, I.