DOCSLIB.ORG

SONIFICATION with MUSICAL CHARACTERISTICS: a PATH GUIDED by USER ENGAGEMENT Jonathan Middleton1, Jaakko Hakulinen2, Katariina Ti

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
SONIFICATION with MUSICAL CHARACTERISTICS: a PATH GUIDED by USER ENGAGEMENT Jonathan Middleton1, Jaakko Hakulinen2, Katariina Ti The 24th International Conference on Auditory Display (ICAD 2018) June 10-15, 2018, Michigan Technological University SONIFICATION WITH MUSICAL CHARACTERISTICS: A PATH GUIDED BY USER ENGAGEMENT Jonathan Middleton1, Jaakko Hakulinen2, Katariina Tiitinen2, Juho Hella2, Tuuli Keskinen2, Pertti Huuskonen2, Juhani Linna2, Markku Turunen2, Mounia Ziat3 and Roope Raisamo2 1 Eastern Washington University, Department of Music, Cheney, WA 99203 USA, [email protected] 2 University of Tampere, Tampere Unit for Computer-Human Interaction (TAUCHI), Tampere, FI-33014 Finland, {firstname.lastname}@uta.fi 3Northern Michigan University, Dept. Psychological Science, Marquette, MI 49855 USA, [email protected] ABSTRACT The use of musical elements and characteristics in sonification has been formally explored by members of Sonification with musical characteristics can engage the International Community for Auditory Display since users, and this dynamic carries value as a mediator the mid 1990’s [4], [5], [6]. A summary of music-related between data and human perception, analysis, and research and software development in sonification can interpretation. A user engagement study has been be obtained from Bearman and Brown [7]. Earlier designed to measure engagement levels from conditions attempts were made by Pollack and Flicks in 1954 [8], within primarily melodic, rhythmic, and chordal yet according to Paul Vickers [9] and other researchers, contexts. This paper reports findings from the melodic such as, Walker and Nees [10], the path for music in portion of the study, and states the challenges of using sonification remains uncertain or unclear. Vickers musical characteristics in sonifications via the recently raised the significant question: “how should perspective of form and function – a long standing sonification designers who wish their work to be more debate in Human-Computer Interaction. These results ‘musical’ approach the task?” Vickers asserts “the path can guide the design of more complex sonifications of we ought to take is currently unanswered.” Walker and multivariable data suitable for real life use. Nees [10] say that “questions of aesthetics and musicality remain open in the field of sonification.” The 1. INTRODUCTION interest in incorporating musical characteristics in sonification can be summarized from the research of Sonifications that rely on musical elements such as Brown et al. [11]: “the use of musical sounds has been rhythm, pitch, and harmony are often reserved for recommended because of the ease with which musical creative purposes, yet musical characteristics can offer sounds are perceived.” important contributions towards analysis [1]. When a variety of musical elements are integrated in auditory The use of musical characteristics in sonification can display for analytical contexts, such as process serve multiple purposes. To start, musical expressions that relate to aesthetics can enhance the user experience, monitoring, researchers can capture a variety of features and this can, in turn, enhance the perception of data. from multivariate data [2]. One concern, however, is that Positive user experiences can translate to more time musical representations can create distractions and that spent with the data analysis, and improving music adds something more than is needed for an interpretations. Finally, the mapping possibilities to analysis. In this sense, music is seen as a more favorable musical traits are numerous and this brings opportunities component of form rather than function – an issue tied to to associate multivariate data to different types of the long-standing Human-Computer Interaction (HCI) musical features and expressions. debate of which comes first [3]. The work reported in this paper was carried out in a This paper offers the view that musical elements can project called Data-to-Music. It focused on the facilitate analytical objectives from improved data development of a software to map data to musical perception, data analysis, and data interpretation through characteristics. The data came mostly from monitoring user engagement and enhanced experiences. the conditions of buildings, machines, weather, and athletic experiences. What made the project unique, was the focus on the user experience with music. In order to test the possibilities of musical This work is licensed under Creative Commons characteristics in data representation and display, the Attribution – Non Commercial 4.0 International research team designed and programmed the software License. The full terms of the License are available at http://creativecommons.org/licenses/by-nc/4.0/ called D2M (Figure 1). It enables the users to assign KWWSGRLRUJLFDG 35 The 24th International Conference on Auditory Display (ICAD 2018) June 10-15, 2018, Michigan Technological University Data-2-Music 2.1 Logged in as admin. weatherLog out weather Nov 20, 2016 - Nov 21, 2016 Edit Save settings Reset timelines 2 PM 06 PM Nov 20 06 AM 12 PM 06 PM Mon 21 06 AM 12 PM Playback controls Duration (seconds): 40 Loop: BPM: 60 Beats/timeline: 40 Play all Stop all Export MIDI Settings Clouds 8 Mute 6 4 2 0 12:00 15:00 18:00 21:00 00:00 03:00 06:00 09:00 12:00 Add stream Pitch Clouds: stream 1 Data to Pitch on instrument 6: marimba Settings Beeats/timeline 40 Scale c-major -4 -5 - Mute -6 MIDI Range 43 -7 60 -8 -9 enforce tonic 12:00 15:00 18:00 21:00 00:00 03:00 06:00 09:00 12:00 Figure 1: Weather mapping to MIDI notes with the D2M software. variables of time series data sets to “streams” which In the process of interpreting data, sound control different aspects of the generated audio. Each characteristics with musical qualities can contribute to stream can be associated with a unique or shared enhanced levels of engagement or distract from the instrument timbre and the data values can be set to analysis [12]. The challenge is to make sure the aesthetic control pitch, the complexity of rhythmic patterns, qualities of the auditory display do not compromise accent chords, articulation, or loudness of the coherence. We submit that data-to-music sonification instrument. The user can determine the duration of the can not only engage users, but levels of engagement generated music and the data are scaled accordingly. The with coherence can be increased by careful combinations data streams specify parameters of each generated note of musical characteristics. based on the given settings. The global structure of the A significant goal in HCI is the user experience, generated music is the result of the data content. which is often tied to aesthetics. Dillon [13] traces the Additional tools include the possibility to set importance of aesthetics, as a basic motivator, back to A. thresholds based both on data values and time range to Manslow (1954). Tractinsky et al. argue that aesthetic filter out some of the data. There are also preprocessing perceptions are highly correlated by individuals to tools for linear and logarithmic scaling, as well as ways interface usage; therefore, whether we intend to for inverting data values. The high degree of flexibility incorporate aesthetics by design into usage or not, of the tool enabled the team to create auditory displays aesthetics are most likely already tied to usage [3], [14]. that sound composed, even though the results were In 2006 Andrew Dillon [13] introduced the term prepared by an algorithm. mediation to describe an essential mechanism whose In this article, we first outline our sonification role is to translate information between data and human. approach and then describe a user survey that seeks to In describing this mechanism, he states that information measure to which extent the approach leads to greater technology is a mediator, among others, which “carries, levels of user engagement. The D2M software was used stores, retrieves, and presents data”; and from these to turn weather data into musical pieces used in the functions, information technology “can provide a survey. physical instantiation of data or the mechanism for making visible or audible the data of interest.” 2. FORM AND FUNCTION In this paper, we are guided by the research question: Can musical characteristics contribute to meaningful The form and function debate in HCI is similar to the data perception, analysis and interpretation? In one found in sonification with music. addressing this challenge, we reference two studies that focus on the significance of the audible experience 36 The 24th International Conference on Auditory Display (ICAD 2018) June 10-15, 2018, Michigan Technological University Playback controls Duration (seconds): 40 Loop: BPM: 60 Beats/timeline: 40 Play all Stop all Export MIDI Settings Clouds 6 Mute 4 2 0 00:00 12:00 15:00 18:00 21:00 00:00 09:00 06:00 00:00 Add stream Clouds: stream 2 Data to Pitch on instrument 6: marimba Settings Preprocessing BPM: 60 Beats/timeline 40 0 Mute -2 -4 -6 09:00 12:00 15:00 18:00 21:00 00:00 03:00 06:00 09:00 Clouds: stream 2 Rhythm Data to Pitch on instrument 6: marimba Settings meeline 40 data timestamps 6 varying 4 Mute simple 2 complex 2 0 09:00 12:00 15:00 18:00 21:00 00:00 03:00 06:00 09:00 Figure 2: Cloud cover data being mapped into two independent streams of pitch and rhythmic values. mentioned by Dillon and in particular the instantiation of time peripheral monitoring with sounds. Their main goal data called sonification. We will then seek to prove that was to create a continuous soundscape based on interpretive experiences with the aid of musical sonification that does not distract users from their main characteristics in the auditory display can offer task(s). In their study forest sounds were chosen for analytical advantages. continuous soundscapes during peripheral process In Human-Computer Interaction, auditory display monitoring. The decision to use sounds from nature has maintained an increasing role in offering new came after consulting the work of Gaver et al.
Load more

Recommended publications
  • Sonification As a Means to Generative Music Ian Baxter
    Sonification as a means to generative music By: Ian Baxter A thesis submitted in partial fulfilment of the requirements for the degree of Doctor of Philosophy The University of Sheffield Faculty of Arts & Humanities Department of Music January 2020 Abstract This thesis examines the use of sonification (the transformation of non-musical data into sound) as a means of creating generative music (algorithmic music which is evolving in real time and is of potentially infinite length). It consists of a portfolio of ten works where the possibilities of sonification as a strategy for creating generative works is examined. As well as exploring the viability of sonification as a compositional strategy toward infinite work, each work in the portfolio aims to explore the notion of how artistic coherency between data and resulting sound is achieved – rejecting the notion that sonification for artistic means leads to the arbitrary linking of data and sound. In the accompanying written commentary the definitions of sonification and generative music are considered, as both are somewhat contested terms requiring operationalisation to correctly contextualise my own work. Having arrived at these definitions each work in the portfolio is documented. For each work, the genesis of the work is considered, the technical composition and operation of the piece (a series of tutorial videos showing each work in operation supplements this section) and finally its position in the portfolio as a whole and relation to the research question is evaluated. The body of work is considered as a whole in relation to the notion of artistic coherency. This is separated into two main themes: the relationship between the underlying nature of the data and the compositional scheme and the coherency between the data and the soundworld generated by each piece.
    [Show full text]
  • 92 Airq Sonification As a Context for Mutual Contribution Between
    ARANGO, J.J. AirQ Sonification as a context for mutual contribution between Science and Music Revista Música Hodie, Goiânia, V.18 - n.1, 2018, p. 92-102 AirQ Sonification as a context for mutual contribution between Science and Music Julián Jaramillo Arango (Caldas University Manizales, Colombia) [email protected] Abstract. This paper addresses a high-level discussion about the links between Science and Music, focusing on my own sonification and auditory display practices around air quality. Grounded on recent insights on interdisciplina- rity and sonification practices, the first sections will point out some potential contributions from scientific research to music studies and vice versa. I will propose the concept of mutualism to depict the interdependent status of Mu- sic and Science. The final sections will discuss air contamination as a complex contemporary problem, and will re- port three practice-based-design projects, AirQ jacket, Esmog Data and Breathe!, which outline different directions in facing local environmental awareness. Keywords. Science and Music, Sonification, Environmental Awarness, Sonology. A sonificação do AirQ como um contexto de contribuição mutua entre a Ciência e a Música. Resumo. Este artigo levanta uma discussão geral sobre as relações entre a Ciência e a Música, e se concentra em minhas praticas de sonificação e display auditivo da qualidade do ar. Baseado em reflexões recentes sobre a interdis- ciplinaridade e as práticas de sonificação as primeiras seções identificam algumas contribuições potenciais da pes- quisa científica aos estudos musicais e vice versa. O conceito de mutualismo será proposto para expressar a interde- pendência entre Música e Ciência. As seções finais discutem a contaminação do ar como um problema complexo de nossos dias e descreve três projetos de design-baseado-na-prática, AirQ jacket, Esmog Data e Breathe! que propõem direções diferentes para gerar consciência ambiental no contexto local.
    [Show full text]
  • The Sonification Handbook Chapter 4 Perception, Cognition and Action In
    The Sonification Handbook Edited by Thomas Hermann, Andy Hunt, John G. Neuhoff Logos Publishing House, Berlin, Germany ISBN 978-3-8325-2819-5 2011, 586 pages Online: http://sonification.de/handbook Order: http://www.logos-verlag.com Reference: Hermann, T., Hunt, A., Neuhoff, J. G., editors (2011). The Sonification Handbook. Logos Publishing House, Berlin, Germany. Chapter 4 Perception, Cognition and Action in Auditory Display John G. Neuhoff This chapter covers auditory perception, cognition, and action in the context of auditory display and sonification. Perceptual dimensions such as pitch and loudness can have complex interactions, and cognitive processes such as memory and expectation can influence user interactions with auditory displays. These topics, as well as auditory imagery, embodied cognition, and the effects of musical expertise will be reviewed. Reference: Neuhoff, J. G. (2011). Perception, cognition and action in auditory display. In Hermann, T., Hunt, A., Neuhoff, J. G., editors, The Sonification Handbook, chapter 4, pages 63–85. Logos Publishing House, Berlin, Germany. Media examples: http://sonification.de/handbook/chapters/chapter4 8 Chapter 4 Perception, Cognition and Action in Auditory Displays John G. Neuhoff 4.1 Introduction Perception is almost always an automatic and effortless process. Light and sound in the environment seem to be almost magically transformed into a complex array of neural impulses that are interpreted by the brain as the subjective experience of the auditory and visual scenes that surround us. This transformation of physical energy into “meaning” is completed within a fraction of a second. However, the ease and speed with which the perceptual system accomplishes this Herculean task greatly masks the complexity of the underlying processes and often times leads us to greatly underestimate the importance of considering the study of perception and cognition, particularly in applied environments such as auditory display.
    [Show full text]
  • The Sonification Handbook Chapter 3 Psychoacoustics
    The Sonification Handbook Edited by Thomas Hermann, Andy Hunt, John G. Neuhoff Logos Publishing House, Berlin, Germany ISBN 978-3-8325-2819-5 2011, 586 pages Online: http://sonification.de/handbook Order: http://www.logos-verlag.com Reference: Hermann, T., Hunt, A., Neuhoff, J. G., editors (2011). The Sonification Handbook. Logos Publishing House, Berlin, Germany. Chapter 3 Psychoacoustics Simon Carlile Reference: Carlile, S. (2011). Psychoacoustics. In Hermann, T., Hunt, A., Neuhoff, J. G., editors, The Sonification Handbook, chapter 3, pages 41–61. Logos Publishing House, Berlin, Germany. Media examples: http://sonification.de/handbook/chapters/chapter3 6 Chapter 3 Psychoacoustics Simon Carlile 3.1 Introduction Listening in the real world is generally a very complex task since sounds of interest typically occur on a background of other sounds that overlap in frequency and time. Some of these sounds can represent threats or opportunities while others are simply distracters or maskers. One approach to understanding how the auditory system makes sense of this complex acoustic world is to consider the nature of the sounds that convey high levels of information and how the auditory system has evolved to extract that information. From this evolutionary perspective, humans have largely inherited this biological system so it makes sense to consider how our auditory systems use these mechanisms to extract information that is meaningful to us and how that knowledge can be applied to best sonify various data. One biologically important feature of a sound is its identity; that is, the spectro-temporal characteristics of the sound that allow us to extract the relevant information represented by the sound.
    [Show full text]
  • Gravity's Reverb: Listening to Space-Time, Or Articulating The
    GRAVITY’S REVERB: Listening to Space-Time, or Articulating the Sounds of Gravitational-Wave Detection STEFAN HELMREICH Massachusetts Institute of Technology http://orcid.org/0000-0003-0859-5881 I heard gravitational waves before they were detected.1 I was sitting in a pub in May 2015 with MIT physicists Scott Hughes and David Kaiser, my headphones looped into a laptop. Hughes was readying to play us some interstellar sounds: digital audio files, he explained, that were sonic simulations of what gravitational waves might sound like if, one day, such cosmic undulations arrived at detection devices on Earth. Gravitational waves are tiny but consequential wriggles in space-time, first theorized by Einstein in 1916— vibrations generated, for example, by such colossal events as the collision of black holes or the detonation of supernovae. Listening to Hughes’s .wav representations of black holes spiraling into one another, I heard washing-machine whirs, Ther- emin-like glissandos, and cybernetic chirps—noises reminiscent, I mused, of mid- twentieth century sci-fi movies (see Taylor 2001 on space-age music). Audio 1. Circular inspiral, spin 35.94% of maximum, orbital plane 0Њ,0Њ viewing angle. Created by Pei-Lan Hsu, using code written by Scott Hughes. Audio 2. Generic inspiral, eccentricity e = 0.7, inclination i =25Њ. Created by Pei-Lan Hsu, using code written by Scott Hughes. CULTURAL ANTHROPOLOGY, Vol. 31, Issue 4, pp. 464–492, ISSN 0886-7356, online ISSN 1548-1360. ᭧ by the American Anthropological Association. All rights reserved. DOI: 10.14506/ca31.4.02 GRAVITY’S REVERB These were sounds that, as it turned out, had a family resemblance to the actual detection signal that, come February 2016, would be made public by astronomers at the Laser Interferometer Gravitational-Wave Observatory (LIGO), an MIT-Caltech astronomy collaboration anchored at two massive an- tennae complexes, one in Hanford, Washington, another in Livingston, Louisiana.
    [Show full text]
  • Comparison and Evaluation of Sonification Strategies for Guidance Tasks Gaetan Parseihian, Charles Gondre, Mitsuko Aramaki, Sølvi Ystad, Richard Kronland Martinet
    Comparison and Evaluation of Sonification Strategies for Guidance Tasks Gaetan Parseihian, Charles Gondre, Mitsuko Aramaki, Sølvi Ystad, Richard Kronland Martinet To cite this version: Gaetan Parseihian, Charles Gondre, Mitsuko Aramaki, Sølvi Ystad, Richard Kronland Mar- tinet. Comparison and Evaluation of Sonification Strategies for Guidance Tasks. IEEE Transac- tions on Multimedia, Institute of Electrical and Electronics Engineers, 2016, 18 (4), pp.674-686. 10.1109/TMM.2016.2531978. hal-01306618 HAL Id: hal-01306618 https://hal.archives-ouvertes.fr/hal-01306618 Submitted on 25 Apr 2016 HAL is a multi-disciplinary open access L’archive ouverte pluridisciplinaire HAL, est archive for the deposit and dissemination of sci- destinée au dépôt et à la diffusion de documents entific research documents, whether they are pub- scientifiques de niveau recherche, publiés ou non, lished or not. The documents may come from émanant des établissements d’enseignement et de teaching and research institutions in France or recherche français ou étrangers, des laboratoires abroad, or from public or private research centers. publics ou privés. JOURNAL OF LATEX CLASS FILES, VOL. ?, NO. ?, ? 2016 1 Comparison and evaluation of sonification strategies for guidance tasks Gaetan¨ Parseihian, Charles Gondre, Mitsuko Aramaki, Senior Member, IEEE, Sølvi Ystad, Richard Kronland Martinet, Senior Member, IEEE Abstract This article aims to reveal the efficiency of sonification strategies in terms of rapidity, precision and overshooting in the case of a one-dimensional guidance task. The sonification strategies are based on the four main perceptual attributes of a sound (i.e. pitch, loudness, duration/tempo and timbre) and classified with respect to the presence or not of one or several auditory references.
    [Show full text]
  • Sonification Strategies for the Film Rhythms of the Universe
    The 20th International Conference on Auditory Display (ICAD-2014) June 22-25, 2014, New York, USA SONIFICATION STRATEGIES FOR THE FILM RHYTHMS OF THE UNIVERSE Mark Ballora Penn State University School of Music/School of Theatre Music Bldg I, University Park, PA 16802-1901, USA [email protected] ABSTRACT Smoot remarked that all complex information – whether it be the history of the universe or the mapping of the human Design strategies are discussed for sonifications that were genome, is rendered in such a way as to fit onto something the created for the short film Rhythms of the Universe, which was size of a book or newspaper. This is what the human mind can conceived by a cosmologist and a musician, with multi-media take in. Aesthetic interpretations are vital to creating contributions from a number of artists and scientists. informative renderings. Sonification functions as an engagement factor in this scientific Following a review in Section 2 of prior work done in this outreach project, along with narration, music, and visualization. area, Section 3 will describe seven sonification examples from This paper describes how the sonifications were created from the film, followed by Concluding Remarks in Section 4. Audio datasets describing pulsars, the planetary orbits, gravitational examples may be downloaded from [2]. Sonification work for waves, nodal patterns in the sun’s surface, solar winds, this film has been described previously [3], although this paper extragalactic background light, and cosmic microwave focuses mostly on different design approaches than those background radiation. The film may be viewed online at [1], described there.
    [Show full text]
  • SONIFICATION: a PREHISTORY David Worrall Audio Arts And
    The 24th International Conference on Auditory Display (ICAD 2018) June 10-15, 2018, Michigan Technological University SONIFICATION: A PREHISTORY David Worrall Audio Arts and Acoustics Department, Columbia College Chicago, 600 South Michigan Avenue Chicago, IL 60605, USA. [email protected] ABSTRACT generated data sonification is a relatively recent formalized practice, cultural antecedents can be identified in all periods The idea that sound can convey information predates the of history. In the spirit of the adage “know from whence modern era, and certainly the computational present. Data you came,” anecdotes and general trends are instructive sonification can be broadly described as the creation, study because they provide insights into what people thought and use of the non-speech aural representation of sound was and how it could be used to communicate. information to convey information. As a field of The idea that sound can convey non-speech contemporary enquiry and practice, data sonification is information predates the modern era, and certainly the young, interdisciplinary and evolving; existing in parallel to computational present. By considering some ways in which the field of data visualization. Drawing on older practices sound has been used in the past to convey specific such as auditing, and the use of information messaging in information, this paper attempts to provide a focused music, this paper provides an historical understanding of historical understanding of ideas about sound and how its how sound and its representational deployment in deployment in communicating information has changed in communicating information has changed. In doing so, it the modern era. In doing so, it is hoped that this will assist aims to encourage a critical awareness of some of the socio- us to critically examine the socio-cultural assumptions we cultural as well as technical assumptions often adopted in have adopted, particularly with regard to the current sonifying data, especially those that have been developed in practice of data sonification.
    [Show full text]
  • Communicating Air: Alternative Pathways to Environmental Knowing Through Computational Ecomedia by Andrea Polli
    University of Plymouth PEARL https://pearl.plymouth.ac.uk 04 University of Plymouth Research Theses 01 Research Theses Main Collection 2011 Communicating Air: Alternative Pathways to Environmental Knowing through Computational Ecomedia Polli, Andrea http://hdl.handle.net/10026.1/889 University of Plymouth All content in PEARL is protected by copyright law. Author manuscripts are made available in accordance with publisher policies. Please cite only the published version using the details provided on the item record or document. In the absence of an open licence (e.g. Creative Commons), permissions for further reuse of content should be sought from the publisher or author. Communicating Air: Alternative Pathways to Environmental Knowing through Computational Ecomedia by Andrea Polli BA (Johns Hopkins University) 1989 MFA (The School of the Art Institute of Chicago) 1992 A dissertation submitted to the University of Plymouth in partial fulfilment for the degree of DOCTOR OF PHILOSOPHY (PhD) Word count: 74,050 Supplemented by: Proof of Practice of three case studies on 1 DVD and 1 Audio CD Committee in Charge: First Supervisor: Dr Jill Scott, Zurich University of the Arts and the University of Plymouth Secondary Supervisor: Dr Angelika Hilbeck, ETHZ, Zurich Date of Submission: March 31, 2011 Polli ⓒ 2011 Acknowledgements In addition to her excellent supervisors, the author would like to express special thanks to the following persons who assisted in the development and implementation of this research: Kurt Ralske, for his programming expertise,
    [Show full text]
  • Interactive Sonification of Network Traffic to Support Cyber Security Situational Awareness
    Northumbria Research Link Citation: Debashi, Mohamed (2018) Interactive Sonification of Network Traffic to support Cyber Security Situational Awareness. Doctoral thesis, Northumbria University. This version was downloaded from Northumbria Research Link: http://nrl.northumbria.ac.uk/id/eprint/39458/ Northumbria University has developed Northumbria Research Link (NRL) to enable users to access the University’s research output. Copyright © and moral rights for items on NRL are retained by the individual author(s) and/or other copyright owners. Single copies of full items can be reproduced, displayed or performed, and given to third parties in any format or medium for personal research or study, educational, or not-for-profit purposes without prior permission or charge, provided the authors, title and full bibliographic details are given, as well as a hyperlink and/or URL to the original metadata page. The content must not be changed in any way. Full items must not be sold commercially in any format or medium without formal permission of the copyright holder. The full policy is available online: http://nrl.northumbria.ac.uk/policies.html Interactive Sonification of Network Traffic to support Cyber Security Situational Awareness Mohamed Debashi A thesis submitted in partial fulfilment of the requirements of the University of Northumbria at Newcastle for the degree of Doctor of Philosophy Research undertaken in the Department of Computer and Information Sciences October 2018 Abstract Maintaining situational awareness of what is happening within a computer network is challenging, not least because the behaviour happens within computers and communications networks, but also because data traffic speeds and volumes are beyond human ability to process.
    [Show full text]
  • Sonic Interaction Design Sonic Interaction Design
    Sonic Interaction Design Sonic Interaction Design edited by Karmen Franinovi ć and Stefania Serafin The MIT Press Cambridge, Massachusetts London, England © 2013 Massachusetts Institute of Technology All rights reserved. No part of this book may be reproduced in any form by any electronic or mechanical means (including photocopying, recording, or information storage and retrieval) without permission in writing from the publisher. MIT Press books may be purchased at special quantity discounts for business or sales promotional use. For information, please email [email protected] or write to Special Sales Depart- ment, The MIT Press, 55 Hayward Street, Cambridge, MA 02142. This book was set in Stone Sans and Stone Serif by Toppan Best-set Premedia Limited, Hong Kong. Printed and bound in the United States of America. Library of Congress Cataloging-in-Publication Data Sonic interaction design / edited by Karmen Franinovi ć and Stefania Serafin. pages cm Includes bibliographical references and index. ISBN 978-0-262-01868-5 (hardcover : alk. paper) 1. Sonic interaction design. 2. Product design. 3. Sound in design. 4. Human-computer interaction. I. Franinović , Karmen, 1975 — editor of compilation. II. Serafin, Stefania, 1973 — editor of compilation. TS171.S624 2013 004.01 ' 9 — dc23 2012028482 10 9 8 7 6 5 4 3 2 1 Contents Introduction vii I Emergent Topics 1 1 Listening to the Sounding Objects of the Past: The Case of the Car 3 Karin Bijsterveld and Stefan Krebs 2 The Experience of Sonic Interaction 39 Karmen Franinovi ć and Christopher Salter 3 Continuous Auditory and Tactile Interaction Design 77 Yon Visell, Roderick Murray-Smith, Stephen A.
    [Show full text]
  • Bell3d: an Audio-Based Astronomy Education System for Visually-Impaired Students Applications Research and and Research
    Bell3D: An Audio-based Astronomy Education System for Visually-impaired Students Applications Research and and Research Jamie Ferguson Keywords University of Aberdeen and University of Sonification, human-computer interaction, Glasgow, Scotland multimodal, visually impaired, software [email protected] This article presents a software system that allows astronomical data to be represented and analysed purely through sound. The goal is to use this for communicating astronomy to visually impaired students, as well as providing a new modality of astronomy experience for those who are sighted. The programme — known as Bell3D and developed at the University of Aberdeen — allows various stars to be selected from a database and then maps a variety of their astronomical parameters to audio parameters so that they can be analysed, compared and understood, solely through the medium of sound. Introduction and background is the “transformation of data relations to use of sonification as a method of pub- perceived relations in an acoustic signal” lic engagement with the Rosetta mis- The ability to experience the surrounding (Kramer et al., 2010). sion (European Space Agency, 2014); Universe is inherent and taken for granted NASA’s Earth+ system1 and the Harvard– by those who are sighted, but for those with Sonification, in many situations, is better Smithsonian Centre for Astrophysics’ Star a visual impairment, the night sky remains suited to analysing large datasets for pat- Songs: From X-rays to Music project2. almost entirely impenetrable. Utilising soni- terns and anomalies than visual modalities Furthermore, during the press coverage fication — the use of non-speech audio to (Kramer et al., 2010).
    [Show full text]