Principles for the Design of Auditory Interfaces to Present Complex Information to Blind People
Total Page:16
File Type:pdf, Size:1020Kb
Principles for the Design of Auditory Interfaces to Present Complex Information to Blind People Robert David Stevens Submitted for the degree of Doctor of Philosophy The University of York The Human Computer Interaction Group, The Department of Computer Science. January 1996 Abstract This thesis proposes a set of principles to aid the design of user interfaces that enable blind users to read complex information by listening. Prior to this work speech based interfaces tended to `read at', rather than being read by the listener. By addressing the themes of control of information ¯ow and the lack of external memory, a set of guidelines have been produced that transform the passive listener to an active reader. Prosody was used to add information to a spoken presentation of algebra in order to enhance its role as an external memory. A set of rules were developed that inserted prosodic cues for algebra into synthetic speech. An experiment found that these cues enhanced the recovery of syntactic structure; the recovery of content and reduced mental workload. A structure vbased browsing method and associated command language were used to add control over the information ¯ow.An iterative cycle of design and evaluation allowed the development of a style of browsing that would allow the fast and accurate control needed for active reading. The ®nal component of the system was an audio glance at the structure of an algebra expression. This was a combination of the prosodic rules that enabled presentation of structure and audio messages called earcons. Experiments were conducted that showed these algebra earcons were able to rapidly convey a suitable representation of an expression from which structural complexity and type could be judged, thus facilitating the planning of what browsing moves to use. The three components of the system wer drawn together and evaluated. A comparison was made with a conventional style of presentation. The new design was found to be more effective, ef®cient and satisfying by the users of the system. The design guidelines set forth in this thesis offer a method to make the access to complex information by blind people more usable. Contents 1 Introduction 1 1.1 Introduction . 1 1.1.1 Simple and Complex Types of Information . 2 1.1.2 Active Reading and External Memory . 4 1.2 The Mathtalk Program and the Maths Project . 5 1.3 The Wider Field . 7 1.4 A De®nition of Terms . 10 1.4.1 Visual Disability . 10 1.4.2 Algebraic De®nitions . 10 1.4.3 Usability . 11 1.4.4 Musical Notation Used in the Thesis . 11 1.5 Thesis Aims . 12 1.6 Contents of the Thesis . 13 2 Literature Review 17 2.1 Introduction . 17 2.2 Reading . 18 2.2.1 External Memory . 19 2.2.2 Characteristics of Visual Control . 21 2.3 Listening . 22 2.3.1 Short-term Memory . 23 i 2.3.2 Listening to Synthetic Speech . 25 2.4 Reading Algebra . 26 2.5 Visually Disabled People and Mathematics . 31 2.5.1 Current Speech Based Solutions . 32 2.6 The Prosodic Component of Speech . 40 2.6.1 Prosody in Spoken English . 41 2.6.2 Prosodic Function . 43 2.6.3 Prosody and Memory for Speech . 44 2.6.4 Prosody and Demarcation of Structure . 45 2.6.5 Prosody in Spoken Algebra . 47 2.6.6 Prosody and Speech Synthesis . 48 2.7 Browsing . 49 2.7.1 Components of Browsing . 50 2.7.2 Navigation in Browsing . 52 2.7.3 Program Browsing . 54 2.8 Conclusions . 55 3 Speaking Algebra Notation 57 3.1 Introduction . 57 3.2 What Notation to Speak and to Whom . 58 3.3 What Information to Speak . 59 3.4 Presenting Algebra Notation with Lexical Cues . 62 3.4.1 Names of Objects and Whole Expressions . 62 3.4.2 Sub-expressions and Radicals . 63 3.4.3 Fractions . 64 3.4.4 Superscripts . 65 3.4.5 General Rules . 66 3.5 The Prosodic Alternative . 69 3.5.1 Extending the Rules for Algebraic Prosody . 70 ii 3.5.2 Conclusions and Discussion . 80 3.5.3 Design Rules for Algebraic Prosody in Mathtalk . 82 3.6 Evaluating the Prosodic Component . 84 3.6.1 Design . 86 3.6.2 RESULTS AND DISCUSSION . 90 3.7 Conclusions . 99 4 Controlling the Information Flow 102 4.1 The Need for Browsing . 102 4.2 The Nature of the Control . 103 4.2.1 Hiding Complex Objects . 104 4.2.2 What Should be Spoken . 105 4.2.3 Elements of the Control . 106 4.3 The Command Language . 107 4.3.1 Unconstrained Browsing . 109 4.3.2 The Default Browsing Style . 112 4.3.3 Feedback during Browsing . 115 4.4 Evaluation of the Browsing Component . 116 4.4.1 Design . 118 4.4.2 Results and Discussion . 123 4.4.3 Improvements to the Control . 138 4.4.4 Problems with Experimental Design . 139 4.5 Discussion and Conclusions . 140 5 The Audio Glance 143 5.1 Introduction . 143 5.2 What is a Glance? . 144 5.3 The Need for and Nature of the Glance . 145 5.4 Choice of Medium . 146 iii 5.5 Development of the Audio Glance . 148 5.5.1 Review of Earcons . 149 5.5.2 Linking Prosody and Earcons . 151 5.6 Constructing Algebra Earcons . 153 5.7 Evaluating the Audio Glance: Experiment One . 157 5.7.1 Design . 157 5.7.2 Participants . 158 5.7.3 Materials . 158 5.7.4 Procedure . 161 5.7.5 Results and Discussion . 162 5.7.6 Conclusions . 169 5.8 Evaluation of Algebra Earcons: Experiment Two . 169 5.8.1 Design . 169 5.8.2 Results and Discussion . 171 5.9 Rapidity of the Glance . 185 5.10 Exploiting the Utility of Algebra Earcons . 186 5.11 Summary and Conclusions . 188 6 Con®rming the Design Principles 189 6.1 Introduction . 189 6.2 The Nature of the Mathtalk Evaluation . 190 6.3 The Evaluation of Mathtalk . 192 6.3.1 Design . 192 6.3.2 Changes to the Mathtalk Program . 192 6.3.3 Materials . 193 6.4 Results and Discussion . 204 6.4.1 Commands and Strategies . ..