Using Sound to Represent Uncertainty in Spatial Data by Nicholas Edward Bearman Thesis submitted for the degree of Doctor of Philosophy at the University of East Anglia, Norwich School of Environmental Sciences January 2013 This copy of the thesis has been supplied on condition that anyone who consults it is understood to recognise that its copyright rests with the author and that use of any information derived there from must be in accordance with current UK Copyright Law. In addition, any quotation or extract must include full attribution. ii Abstract There is a limit to the amount of spatial data that can be shown visually in an effective manner, particularly when the data sets are extensive or complex. Using sound to represent some of these data (sonification) is a way of avoiding visual overload. This thesis creates a conceptual model showing how sonification can be used to represent spatial data and evaluates a number of elements within the conceptual model. These are examined in three different case studies to assess the effectiveness of the sonifications. Current methods of using sonification to represent spatial data have been restricted by the technology available and have had very limited user testing. While existing research shows that sonification can be done, it does not show whether it is an effective and useful method of representing spatial data to the end user. A number of prototypes show how spatial data can be sonified, but only a small handful of these have performed any user testing beyond the authors’ immediate colleagues (where n > 4). This thesis creates and evaluates sonification prototypes, which represent uncertainty using three different case studies of spatial data. Each case study is evaluated by a significant user group (between 45 and 71 individuals) who completed a task based evaluation with the sonification tool, as well as reporting qualitatively their views on the effectiveness and usefulness of the sonification method. For all three case studies, using sound to reinforce information shown visually results in more effective performance from the majority of the participants than traditional visual methods. Participants who were familiar with the dataset were much more effective at using the sonification than those who were not and an interactive sonification which requires significant involvement from the user was much more effective than a static sonification, which did not provide significant user engagement. Using sounds with a clear and easily understood scale (such as piano notes) was important to achieve an effective sonification. These findings are used to improve the conceptual model developed earlier in this thesis and highlight areas for future research. iii iv List of Contents Abstract ....................................................................................................................................... iii List of Contents ............................................................................................................................. v List of Figures ............................................................................................................................... xi List of Tables ............................................................................................................................... xv List of Acronyms and Glossary .................................................................................................. xvii Acknowledgements ................................................................................................................... xix List of Related Publications ....................................................................................................... xxi Chapter 1. Introduction ............................................................................................................ 1 1.1 How do Humans Understand the World? ........................................................................... 1 1.2 How do Humans Understand Spatial Data? ........................................................................ 3 1.2.1 How do humans understand maps and map symbols? ............................................ 11 1.2.2 Using sound to represent spatial data ...................................................................... 12 1.2.3 Using sound and vision together to represent spatial information .......................... 14 1.3 Human Computer Interaction ........................................................................................... 15 1.4 The Context of this Thesis ................................................................................................. 18 Chapter 2. Literature Review .................................................................................................. 21 2.1 How does Vision Influence the Interpretation of Spatial Data? ....................................... 21 2.1.1 Using the visual sense – geovisualization and cartography...................................... 21 2.1.2 Visual variables ......................................................................................................... 23 2.1.3 How does understanding of visual variables vary between users? ........................... 26 2.1.4 Non visual map users ................................................................................................ 28 2.1.5 Big data and visual saturation .................................................................................. 29 2.2 How does Sound Influence the Interpretation of Data? ................................................... 30 2.2.1 How do we use sound to understand the world? ...................................................... 30 2.2.2 Why non-speech sounds? .......................................................................................... 30 2.2.3 Background to sonification ....................................................................................... 31 2.2.4 Theory of sonification ................................................................................................ 34 2.2.5 Functions of sonifications .......................................................................................... 38 2.2.6 Types of sonification ................................................................................................. 40 2.2.7 Sound variables ......................................................................................................... 43 2.2.8 Mapping data to sound ............................................................................................. 45 2.2.9 Variation in understanding of sound ........................................................................ 47 2.2.10 Why is sonification useful? ........................................................................................ 49 2.2.11 Limits of sonification ................................................................................................. 50 2.3 How do Vision and Sound Influence the Interpretation of Spatial Data?......................... 51 2.3.1 Theoretical background ............................................................................................ 51 2.3.2 Spatial data examples ............................................................................................... 52 2.3.3 Virtual environments and gaming ............................................................................ 57 2.3.4 Using vision and sound together ............................................................................... 59 2.4 How does Uncertainty Influence the Interpretation of Spatial Data? .............................. 60 v 2.4.1 Uncertainty in spatial data ....................................................................................... 61 2.4.2 Uncertainty in this research ...................................................................................... 65 2.4.3 Why is uncertainty important? ................................................................................. 66 2.4.4 How is uncertainty estimated? ................................................................................. 68 2.4.5 How is uncertainty represented visually? ................................................................. 69 2.4.6 Potential non-visual options ..................................................................................... 72 2.5 Developing a Conceptual Model of the Sonification Process ........................................... 73 2.5.1 Synthesis of the literature ......................................................................................... 74 2.5.2 Extended conceptual model ...................................................................................... 77 2.6 Overall Aims and Hypotheses ........................................................................................... 80 2.6.1 Specific research aims ............................................................................................... 81 2.6.2 Research hypotheses ................................................................................................ 81 Chapter 3. Methodology ........................................................................................................ 85 3.1 Justification of Methodology ............................................................................................ 85 3.1.1 Multiple case studies or one case study? .................................................................. 85 3.1.2 One-to-one or group