TOWARDS TASK TRANSPARENCY IN ALTERNATIVE COMPUTER ACCESS: SELECTION OF TEXT THROUGH SWITCH-BASED SCANNJNG G. Fraser Shein A thesis submitted in conformity with the requirements for the degree of Doctor of Philosophy Graduate Department of Mechanical and Industrial Engineering University of Toronto O Copyright by G. Fraser Shein, 1997 National Library Bibliothèque nationale 1+1 of Canada du Canada Acquisitions and Acquisitions et Bibliographic Services sewices bibliographiques 395 Wellington Street 395. rue Wellington OttawaON K1AON4 Ottawa ON KIA ON4 Canada Canada The author has granted a non- L'auteur a accordé une Licence non exclusive licence allowing the exclusive permettant à la National Library of Canada to Bibliothèque nationale du Canada de reproduce, loan, distribute or sell reproduire, prêter, distribuer ou copies of this thesis in microform, vendre des copies de cette thèse sous paper or electronic formats. la forme de microfiche/^ de reproduction sur papier ou sur format électronique. The author retains ownership of the L'auteur conserve la propriété du copyright in this thesis. Neither the droit d'auteur qui protège cette thèse. thesis nor substantial extracts fkom it Ni la thèse ni des extraits substantiels may be printed or otherwise de celle-ci ne doivent être imprimés reproduced without the author's ou autrement reproduits sans son permission. autorisation. Towards Task Transparency in Alternative Computer Access: Selection Of Text Through Switch-Based Scanning Doctor of Philosophy, 1997 G. Fraser Shein Graduate Department of Mechanical and Industrial Engineering, University of Toronto ABSTRACT Current access technology for people with disabilities is based on a concept called transparency. Developen have generally interpreted this by building access systems that emulate the keyboard and mouse. In contrast, this thesis proposes and studies a refinement to transparent access. called task transparency. This approach has much potential impact on scanning access within a GUI. Here, scaming is applied to the underlying tasks, rather than emulating the input devices. The goal is to denve new knowledge and understanding of the interactions that anse with a number of strategies that apply switch-based scanning to text selection in a task transparent fashion. In the case of selecting text. it was demonstrated that directness to the task could be achieved with minimal effort by applying scanning within the text area itself. The concepts are readily extended across other GUI objects including menus, buttons, and window controls. These objects can be incorporated within the access system such that they are directly engaged in the scaming. Through such strategies, usen can significantly reduce their efforts and in doing so achieve greater overall productivity, and accompl ish tasks that are otherwise strenuous. Text scanning was implemented using repeating keys injected by an on-screen keyboard such that the user perceived scaming in the text area. Two test applications were developed and used to evaluate a number of proposed text scanning strategies. In addition, predictive performance models were derived to predict performance measures under error-fiee conditions. Initial design guidelines for scaming in a task transparent fashion were also formulated. An access system developer could extend the concepts in new ways with scanning and other access methods. Future areas of research are identified that may improve overall productivity. Through this work 1 have conuibuted new knowledge towards improving scanning access and 1 have demonstrated the feasibility of task transparency by which further gains are possible. Acknowledgments 1 wish to acknowledge my advisor, Dr. Mark Chignell. who provided me with guidance and expertise in the developrnent of this thesis. I also acknowledge Dr. Mickry Milner and Dr. Steven Naumam of the Bloorview MacMillan Centre who guided me and provided ongoing support over many years at the Bloorview MacMillan Centre. Through their efforts 1 have bern able to hone my research skills and develop an ongoing programme of research related to cornputer systems for people with physical disabilities. 1 would also like to thank Dr. Paul Milgram for his insightful review and criticism that has helped me to bctter present my idsas. 1 extend much thanks to my research team, the Microcornputer Applications Programme. at the Bloorview MacMillan Centre who worked rnany long hours in developing the software solutions described here. In particular. 1 am grateful to Mr. Gil Hamann who assumed additional responsibilities during the final stages of this work and who provided in-depth rditorial and scientific review. Dr. Debra Fels provided encouragement and scientific review. Mr. Reinhard Schuller provided statistical and experimental protocol expertise as well as editorial revirw. 1 thank al1 of the staff at the Bloorview iMacMillan Centre who volunteered their time and cooperation as subjects within my sxperiments. The National Health Training Programme of the National Health Research and Development Programme, Health Canada supported my initial research. The Bloorview MacMillan Centre provided resources and space. IBM Canada Ltd. kindly donated computer equipment. The final phase of my research was supported through the Ontario Rehabilitation Technoiogy Consortium supponed by the Ontario Ministry of Health. Most of all, 1 extend rny appreciation to my partner. Patricia Stoddart, who persevered over many years and provided the constant encouragement and love required to complete this work. Table of Contents 1 Introduction ......................................................................................................... 1 i -1 Problem Statement and Motivation For This Thesis ........................................... 1 1.2 Research Goal and Objectives ................................................................................. 6 1.2.1 Goa1 ................................. .. .......................................................................6 1.7.2 Specitic Objectives ...................................................................................... 6 1 -3 Scope ....................................................................................................................... 7 1 -4 Road Map for this Thesis ........................................................................................ 8 2 Literature Review .............................................. ........................................1 O 2.1 Introduction ........................ ... .....................................................................*... IO 2.2 Rrview of GUI Access by Users with Physical Disabilities ................................. 11 2.3 Rrview of Indirect Scanning Access ..................................................................... II 2.4 Review of Models of Access ................................................................................. 2 1 2.4.1 Rehabilitation Engineering Models ...........................................................2 1 2.3.2 HCI Model of interaction .......................................................................... 27 2.5 Review of Transparent Accrss Approaches ..........................................................29 7-51 -4ltemate Kryboard and Pointing Devices ................................................ 29 2.5.2 Extemal Keyboard Emulating interfaces (KEI) and General Input Device Emulating Interfaces (GIDEI): Standard Ports ..................................................32 2.5.3 Extemal KEI and GIDEI: Sena1 Input ...................................................... 31 2.5.4 IntemalKEIandGIDEI ............................................................................ 35 2-53 Software KEI and GIDE1 .......................................................................... 36 2.6 Review of Mouse Emulators ................................................................................. 3 8 2.6.1 Keypad Mouse Ernulation ......................................................................... 38 2.6.2 Switch-Based Mouse Emulation ........................................................ 40 2 .6.3 Non-Transparent Cornmand-Based Pointer ............................................ 40 2.6.4 Non-Transparent Scanning Screen Pointer ................................. ......... 41 2.6.5 Transparent Scanning Screen Pointer .....................................................43 2.6.6 Scanning Screen Pointer Enhancements ................................................. 46 2.7 Review of Text Selection Methods .................................................................... 47 2.8 Summary ........................................................................................................... 49 Task Transparency: A New Design Approach ...............................................51 3 .L Introduction ....................................................................................................... 51 3.2 Interpretation of the Literature........................................................................... 51 3 -3 Task Transparency and Scanning ...................................................................... 55 3.4 Applying Task Transparency to Text Selechon ................................................. 58 3.4.1 Text Selection Task ................................. ....................................... 58 3.4.2 Keyboard Channel as Information Carrier .............................................. 60 3 .4.3 On-Screen