Concurrency in Auditory Displays for Connected Television
Total Page:16
File Type:pdf, Size:1020Kb
Concurrency in auditory displays for connected television Alistair Francis Hinde Doctor of Philosophy University of York Electronics September 2016 Abstract Many television experiences depend on users being both willing and able to visually attend to screen-based information. Auditory displays offer an alternative method for presenting this information and could benefit all users. This thesis explores how this may be achieved through the design and evaluation of auditory displays involving varying degrees of concurrency for two television use cases: menu navigation and presenting related content alongside a television show. The first study, on the navigation of auditory menus, looked at onset asynchrony and word length in the presentation of spoken menus. The effects of these on task duration, accuracy and workload were considered. Onset asynchrony and word length both caused significant effects on task duration and accuracy, while workload was only affected by onset asynchrony. An optimum asynchrony was identified, which was the same for both long and short words, but better performance was obtained with the shorter words that no longer overlapped. The second experiment investigated how disruption, workload, and preference are affected when presenting additional content accompanying a television programme. The content took the form of sound from different spatial locations or as text on a smartphone and the programme's soundtrack was either modified or left unaltered. Leaving the soundtrack unaltered or muting it negatively impacted user experience. Removing the speech from the television programme and presenting the secondary content as sound from a smartphone was the best auditory approach. This was found to compare well with the textual presentation, resulting in less visual disruption and imposing a similar workload. Additionally, the thesis reviews the state-of-the-art in television experiences and auditory displays. The human auditory system is introduced and important factors in the concurrent presentation of speech are highlighted. Conclusions about the utility of concurrency within auditory displays for television are made and areas for further work are identified. 3 4 Table of contents Abstract 3 Table of Contents 5 List of Figures 11 List of Tables 15 Acknowledgements 17 Declaration 19 1 Introduction 21 1.1 Overview: The dominance of vision in television displays . 21 1.2 Users effected by visio-centric design . 23 1.3 Auditory display . 25 1.4 Hypothesis . 25 1.5 Contributions . 26 1.6 Structure of thesis . 27 2 Television experiences: Now and the near future 29 2.1 Introduction . 29 2.2 Programme delivery . 30 2.3 Television devices . 31 2.4 Elements of the television experience . 33 2.4.1 Navigation and choice . 34 2.4.2 Additional media activities . 38 2.4.3 Adaptive programming: object-based broadcasting . 43 2.4.4 Input mechanisms . 43 2.5 Discussion . 44 2.5.1 Programme delivery . 45 2.5.2 Devices . 45 2.5.3 Input mechanisms . 46 5 Table of contents 6 2.5.4 Navigation . 46 2.5.5 Additional media activities . 47 3 Audition 49 3.1 Introduction . 49 3.2 Hearing: key principles . 50 3.2.1 Physiology of the human auditory system . 50 3.2.2 Spatial auditory perception . 54 3.2.3 Masking . 57 3.2.4 Auditory streams . 59 3.2.5 Auditory attention . 61 3.2.6 The influence of vision . 62 3.3 Speech perception . 64 3.3.1 Speech as a signal . 64 3.3.2 Recognising words and understanding meaning . 65 3.3.3 Concurrent speech . 66 3.3.4 Influences of visual information on speech perception . 76 3.4 Summary . 76 4 Auditory Display 79 4.1 Serial speech displays . 80 4.2 Non-speech auditory displays . 82 4.2.1 Audification, parameter mapping and model-based sonification . 83 4.2.2 Auditory icons . 84 4.2.3 Earcons . 85 4.2.4 Musicons . 88 4.2.5 Spearcons . 89 4.2.6 Auditory scrollbars and Spindex . 91 4.2.7 Discussion . 92 4.3 Concurrent auditory displays . 93 4.3.1 Concurrent earcons . 94 4.3.2 Concurrent auditory icons . 97 4.3.3 Concurrent speech displays . 99 4.4 Summary . 110 5 Menu navigation 113 5.1 Introduction . 113 5.2 Display design . 115 5.2.1 Number of talkers . 115 5.2.2 Spatial configuration . 116 Table of contents 7 5.2.3 Interaction design . 116 5.2.4 Implementation . 119 5.3 Pilot Study . 120 5.3.1 Methodology . 120 5.3.2 Findings . 124 5.4 Experiment Proper . 125 5.4.1 Methodology . 125 5.4.2 Results . 131 5.4.3 Discussion . 138 5.5 Summary . 143 6 Orchestrated synchronous companion experiences: Non-visual design considerations 145 6.1 Introduction . 145 6.2 Attention with visual secondary programme content . 147 6.3 Auditory presentation of secondary programme content . 151 6.4 Design . 152 6.4.1 Content representation . 153 6.4.2 Concurrency of presentation . 155 6.4.3 Talker/performance factors . 160 6.4.4 Interaction design . 161 6.4.5 Notification design . 162 6.4.6 Spatial location . 164 6.4.7 Personal or shared experiences . 167 6.5 Secondary content applications . 169 6.6 Summary and outstanding questions . 170 7 The effects of secondary content modality, location and the modification of programme soundtrack on television user experience 173 7.1 Introduction . 173 7.2 Experimental design . 174 7.2.1 Independent variables . 174 7.2.2 Dependent variables . 175 7.3 Methodology . 179 7.3.1 Experiment structure . 179 7.3.2 Experiential limitations . 180 7.3.3 Prototype system . 181 7.3.4 Stimuli . 183 7.3.5 Pilot Study . 187 7.3.6 Methodological changes following the Pilot . 191 Table of contents 8 7.4 Experiment proper . 192 7.4.1 Demographics . 192 7.4.2 Procedure . 192 7.4.3 Experiment set-up . 193 7.4.4 Data treatment . 195 7.4.5 Quantitative Results . 197 7.4.6 Qualitative Results . 204 7.4.7 Discussion . 218 7.4.8 Limitations and future work . 228 7.5 Summary . ..