USABILITY TESTING OF A MOBILE TECHNOLOGY FOR CHILDREN WITH HIGH-FUNCTIONING AUTISM SPECTRUM AND ATTENTION-DEFICIT/ HYPERACTIVITY DISORDERS by David Groot A dissertation submitted to the faculty of The University of Utah in partial fulfillment of the requirements for the degree of Doctor of Philosophy College of Social Work The University of Utah August 2014 Copyright © David Groot 2014 All Rights Reserved The University of Utah Graduate School STATEMENT OF DISSERTATION APPROVAL The dissertation of David Groot has been approved by the following supervisory committee members: Joanne Yaffe , Chair April 14, 2014 Date Approved Jason Castillo , Member June 6, 2014 Date Approved Leanne Hawken , Member May 28, 2014 Date Approved Lee Hollaar , Member May 22, 2014 Date Approved Lawrence Henry Liese , Member April 14, 2014 Date Approved and by Jannah H. Mather , Dean of the College of Social Work and by David B. Kieda, Dean of The Graduate School. ABSTRACT Children with attention-deficit/hyperactivity disorder (ADHD) and high- functioning autism spectrum disorder (HFASD) have organizational skills deficits. Organizational skills include the ability to manage materials (e.g., belongings, books, homework) and temporal skills such as organizing, planning, and managing tasks to completion. This study was a usability test of a prototype mobile technology designed to improve organizational skills. The prototype was assessed for usability and feasibility for future development. A field-based mixed methods usability test was conducted. Sixteen children with ADHD and HFASD aged 8 to 12 years and their parents participated. The study was conducted in an 8-week summer treatment program. The usability test lasted 15 days, with data collected via observation, child and parent daily logs, surveys, and focus groups. During the usability test, children brought the prototype technology to camp 95% of the time and used it to record items to bring to camp 85% of the time. Parents completed a daily log simulating mobile functions 88% of the time. Using the prototype device for homework tracking resulted in three times the likelihood that homework was completed. Establishing a contingency between device game time and homework completion resulted in four times the likelihood that homework was completed. Qualitative results suggested that children valued carrying the device and children were motivated by having game time on the device as a reward. In addition, qualitative results showed that parents valued the device as a contingent reward, desired novelty in the device’s games and features, and expressed an urgent need for help with their children’s organizational skills. Children will utilize a mobile technology intended for task tracking with game time having a high reward value. Parents value the concept of using a mobile technology to improve their children’s organizational skills. The use of mobile technology for building and sustaining organizational skills via performance rewards is a promising intervention for effective home and school-related task management. The effectiveness of a more fully developed mobile technology needs to be assessed in future research. iv TABLE OF CONTENTS ABSTRACT ..……………………………………………………………...................... iii LIST OF TABLES ………………………………………………………...................... vii ACKNOWLEDGMENTS …………………………………................…......……....... viii Chapters 1 INTRODUCTION ..................................................................................................... 1 Attention-Deficit/Hyperactivity Disorder (ADHD) ................................................... 2 Autism Spectrum Disorder (ASD) ............................................................................. 9 Comorbidity and Executive Function ........................................................................ 12 Organizational Skills .................................................................................................. 14 Statement of the Problem ........................................................................................... 16 Purpose of the Study .................................................................................................. 17 Significance for Social Work ..................................................................................... 19 Summary .................................................................................................................... 21 2 ORGANIZATIONAL SKILLS INTERVENTIONS.................................................. 22 Organizational Skills in the School Setting ............................................................... 22 Organizational Skills Intervention Research and ADHD .......................................... 25 Organizational Skills Intervention Research and ASD .............................................. 36 ASD and Mobile Technology .................................................................................... 41 Implications for the Design of a Mobile Technology ................................................ 43 3 DESIGN AND SOCIAL WORK RESEARCH ......................................................... 45 Design Research ......................................................................................................... 45 Social Work Design and Development Research ...................................................... 53 From Human Factors to Child Computer Interaction ............................................... 57 Usability Testing ........................................................................................................ 59 Usability Testing as Social Work Research ............................................................... 66 4 METHOD .................................................................................................................. 68 Research Questions and Study Design ....................................................................... 68 Participants and Screening Assessments .................................................................... 72 Measures and Data Analysis Plan .............................................................................. 79 Procedures for Usability Testing ................................................................................ 95 Triangulation and Chapter Summary ......................................................................... 99 5 RESULTS .................................................................................................................. 103 Research Question #1 Children and Usability ........................................................... 104 Research Question #2 Parents and Usability .............................................................. 116 Research Question #3 Technology Features .............................................................. 122 Chapter Summary ....................................................................................................... 130 6 DISCUSSION ............................................................................................................ 131 Discussion of Results by Research Question ............................................................. 131 Findings in Relation to Previous Research ................................................................ 140 Strengths and Limitations ........................................................................................... 143 Implications for Social Work Research, Policy, and Practice .................................... 152 Future Development of the Technology ..................................................................... 159 Conclusion .................................................................................................................. 160 Appendices A. CONSENT AND ASSENT FORMS ......................................................................... 162 B. CHILD AND PARENT DAILY LOGS ..................................................................... 171 C. CHILD AND PARENT SURVEYS .......................................................................... 174 D. FOCUS GROUP QUESTIONS AND RESPONSES ................................................ 177 REFERENCES ............................................................................................................... 184 vi LIST OF TABLES Table Page 1. Organizational Skills Intervention Studies for ADHD ……………………....... 27 2. Organizational Skills Intervention Studies for ASD ..……..….......................... 38 3. Results of Children’s Assessment Measures .........................……………......... 76 4. Fleiss’ Kappa for Interrater Reliability .............................……….……….….... 94 5. Child iPod Utilization Based on Daily Logs ..…………………………............ 105 6. Generalized Estimating Equation Results ..............................................…........ 114 7. Summary for Research Question #1 – Children and Usability .......................... 115 8. iPod use at Home for Reward and for Free Play Time ....................................... 117 9. Summary for Research Question #2 – Parents and Usability ............................. 121 10. Child Survey Results ..............................................…………….....…..….….... 123 11. Parent Survey Results ......................................................................................... 127 12. Summary for Research Question #3 – Technology Features ............................. 130 ACKNOWLEDGMENTS My morning meditation includes the phrase, “recognizing interdependence