DOCSLIB.ORG

Science by Ear. an Interdisciplinary Approach to Sonifying Scientific

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text

Load more

Institute of Electronic Music and Acoustics - IEM, University for Music and Dramatic Arts Graz Science By Ear. An Interdisciplinary Approach to Sonifying Scientific Data Alberto de Campo Dissertation Graz, February 23, 2009 Supervisor: Prof Dr Robert H¨oldrich (IEM/KUG), Prof Dr Curtis Roads (MAT/UCSB) ii Science By Ear. An Interdisciplinary Approach to Sonifying Scientific Data Author: Alberto de Campo Contact: [email protected] Supervisor: Prof Dr Robert H¨oldrich (IEM/KUG), Prof Dr Curtis Roads (MAT/UCSB) Contact: [email protected], [email protected] Dissertation Institute of Electronic Music and Acoustics - IEM, University for Music and Dramatic Arts Graz Inffeldgasse 10, A-8020 Graz, Austria February 23, 2009, 211 pages Abstract Sonification of Scientific Data is intrinsically interdisciplinary: It requires collaboration between experts in the respective scientific domains, in psychoacoustics, in artistic design of synthetic sound, and in working with appropriate programming environments. The SonEnvir project hosted at IEM Graz put this view into practice: in four domain sciences, sonification designs for current research questions were realised. This dissertation contributes to sonification research in three aspects: The body of sonification designs realised within the SonEnvir context is described, which may be reused in sonification research in different ways. The software framework built with and for these sonification designs is presented, which supports fluid experimentation with evolving sonification designs. A theoretical model for sonification design work, the Sonification Design Space Map, was synthesised based the analysis of this body of sonification designs (and a few selected others). This model allows systematic reasoning about the process of creating sonifica- tion designs, and provides concepts for analysing and categorising existing sonifications designs more systematically. Deutsche Zusammenfassung - German abstract Die Sonifikation von wissenschaftlichen Daten ist intrinsisch interdisziplin¨ar:Sie verlangt Zusammenarbeit zwischen ExpertInnen in den jeweiligen wissenschaftlichen Gebieten, in Psychoakustik, in der k¨unstlerischenGestaltung von synthetischem Klang, und in der Arbeit mit geeigneten Programmierumgebungen. Das Projekt SonEnvir, das am IEM Graz stattfand, hat diese Sichtweise in die Praxis umgesetzt: in vier wissenschaftlichen Gebieten (domain sciences) wurden Sonifikations-Designs zu aktuellen Forschungsfragen realisiert. iii Diese Dissertation tr¨agtdrei Aspekte zur Sonifikationforschung bei: Der Korpus der im Kontext von SonEnvir entwickelten Sonification Designs wird detail- liert beschrieben; diese Designs k¨onnen in der Forschungsgemeinschaft in verschiedener Weise Weiterverwendung finden. Das Software-Framework, das f¨urund mit diesen Designs gebaut wurde, wird beschrieben; es erlaubt fliessendes Experimentieren in der Entwicklung von Sonifikationsdesigns. Ein theoretisches Modell f¨urdie Gestaltung von Sonifikationen, die Sonification Design Space Map, wurde auf Basis der Analysen dieser (und ausgew¨ahlteranderer) Designs synthetisiert. Dieses Modell erlaubt systematisches Nachdenken (reasoning) ¨uber den Gestaltungsprozess von Sonifikationsdesigns, und bietet Konzepte f¨urdie Analyse und Kategorisierung existierender Sonifikationsdesigns an. Keywords: Sonification, Sonification Theory, Perceptualisation, Interdisciplinary Re- search, Interactive Software Development, Just In Time Programming iv Acknowledgements First of all, I would like to thank Marianne Egger de Campo for designing several versions of the XENAKIS proposal with me - a sonification project with European partners that eventually became SonEnvir. Then, I would like to thank my research partners in the SonEnvir project: Christian Day´e,Christopher Frauenberger, Kathi Vogt and Annette Wallisch, without whom this work would not have been possible. I would like to thank Robert H¨oldrichfor his collaboration on the grant proposals, and for his contribution to the EEG realtime sonification; and Gerhard Eckel for leading the SonEnvir project for most of its lifetime. I would like to thank the participants of the Science By Ear workshop, who have been very open to a very particular experimental setup in interdisciplinary collaboration, especially for the discussions which eventually led to formulating the concept of the Sonification Design Space Map. A very special thank you is in order for the brave people who were willing to try programming sonification designs just-in-time within this workshop: Till Bovermann, Christopher Frauenberger, Thomas Musil, Sandra Pauletto, and Julian Rohrhuber. For the Spin Models, the following Science By Ear participants also worked on a sonifica- tion design for the Ising model (besides the SonEnvir team): Thomas Hermann, Harald Markum, Julian Rohrhuber and Tony Stockman. Concerning the background in theo- retical physics, we would also like to thank Christof Gattringer, Christian Bernd Lang, Leopold Mathelitsch and Ulrich Hohenester. For the piece Navegar, I would to thank Peter Jakober for researching the detailed timeline, and Marianne Egger de Campo for suggesting the Gini index as an interesting variable. Alberto de Campo Graz, February 23, 2009 Contents 1 Introduction1 1.1 Motivation.................................2 1.2 Scope....................................3 1.3 Methodology................................4 1.4 Overview of this thesis...........................5 2 Psychoacoustics, Perception, Cognition, and Interaction6 2.1 Psychoacoustics..............................6 2.2 Auditory perception and memory.....................8 2.3 Cognition, action, and embodiment.................... 10 2.4 Perception, perceptualisation and interaction............... 11 2.5 Mapping, mixing and matching metaphors................ 12 3 Sonification Systems 13 3.1 Background................................ 14 3.1.1 A short history of sonification................... 14 3.1.2 A taxonomy of intended sonification uses............. 17 3.2 Sonification toolkits, frameworks, applications.............. 18 3.2.1 Historic systems.......................... 18 3.2.2 Current systems.......................... 19 3.3 Music and sound programming environments............... 20 3.4 Design of a new system.......................... 23 3.4.1 Requirements of an ideal sonification environment........ 23 3.4.2 Platform choice.......................... 24 3.5 SonEnvir software - Overall scope..................... 24 3.5.1 Software framework........................ 25 v vi 3.5.2 Framework structure........................ 25 3.5.3 The Data model.......................... 26 4 Project Background 29 4.1 The SonEnvir project........................... 29 4.1.1 Partner institutions and people.................. 29 4.1.2 Project flow............................ 30 4.1.3 Publications............................ 31 4.2 Science By Ear - An interdisciplinary workshop.............. 32 4.2.1 Workshop design.......................... 32 4.2.2 Working methods......................... 32 4.2.3 Evaluation............................. 33 4.3 ICAD 2006 concert............................. 34 4.3.1 Listening to the Mind Listening.................. 34 4.3.2 Global Music - The World by Ear................. 34 5 General Sonification Models 37 5.1 The Sonification Design Space Map (SDSM)............... 38 5.1.1 Introduction............................ 38 5.1.2 Background............................ 38 5.1.3 The Sonification Design Space Map................ 41 5.1.4 Refinement by moving on the map................ 43 5.1.5 Examples from the 'Science by Ear' workshop........... 47 5.1.6 Conclusions............................ 50 5.1.7 Extensions of the SDS map.................... 51 5.2 Data dimensions.............................. 52 5.2.1 Data categorisation........................ 52 5.2.2 Data organisation......................... 52 5.2.3 Task Data analysis - LoadFlow data................ 53 5.3 Synthesis models.............................. 56 5.3.1 Sonification strategies....................... 57 5.3.2 Continuous Data Representation................. 57 5.3.3 Discrete Data Representation................... 61 5.3.4 Parallel streams.......................... 62 vii 5.3.5 Model Based Sonification..................... 63 5.4 User, task, interaction models....................... 64 5.4.1 Background - related disciplines.................. 64 5.4.2 Music interfaces and musical instruments............. 65 5.4.3 Interactive sonification....................... 66 5.4.4 "The Humane Interface" and sonification............. 67 5.4.5 Goals, tasks, skills, context.................... 69 5.4.6 Two examples........................... 71 5.5 Spatialisation Model............................ 74 5.5.1 Speaker-based sound rendering................. 75 5.5.2 Headphones........................... 77 5.5.3 Handling speaker imperfections................. 80 6 Examples from Sociology 81 6.1 FRR Log Player.............................. 82 6.1.1 Technical background....................... 82 6.1.2 Analysis steps........................... 84 6.1.3 Sonification design......................... 86 6.1.4 Interface design.......................... 87 6.1.5 Evaluation for the research context................ 88 6.1.6 Evaluation in SDSM terms.................... 88 6.2 'Wahlges¨ange' - 'Election Songs'..................... 90 6.2.1 Interface and sonification design.................. 91 6.2.2 Evaluation............................. 93 6.3 Social Data Explorer............................ 94 6.3.1 Background..........................
Recommended publications
  • Sonification As a Means to Generative Music Ian Baxter

    Sonification As a Means to Generative Music Ian Baxter

    Sonification as a means to generative music By: Ian Baxter A thesis submitted in partial fulfilment of the requirements for the degree of Doctor of Philosophy The University of Sheffield Faculty of Arts & Humanities Department of Music January 2020 Abstract This thesis examines the use of sonification (the transformation of non-musical data into sound) as a means of creating generative music (algorithmic music which is evolving in real time and is of potentially infinite length). It consists of a portfolio of ten works where the possibilities of sonification as a strategy for creating generative works is examined. As well as exploring the viability of sonification as a compositional strategy toward infinite work, each work in the portfolio aims to explore the notion of how artistic coherency between data and resulting sound is achieved – rejecting the notion that sonification for artistic means leads to the arbitrary linking of data and sound. In the accompanying written commentary the definitions of sonification and generative music are considered, as both are somewhat contested terms requiring operationalisation to correctly contextualise my own work. Having arrived at these definitions each work in the portfolio is documented. For each work, the genesis of the work is considered, the technical composition and operation of the piece (a series of tutorial videos showing each work in operation supplements this section) and finally its position in the portfolio as a whole and relation to the research question is evaluated. The body of work is considered as a whole in relation to the notion of artistic coherency. This is separated into two main themes: the relationship between the underlying nature of the data and the compositional scheme and the coherency between the data and the soundworld generated by each piece.
  • Proceedings-Print.Pdf ISSN: 2175-6759 ISBN: 978-85-76694-75-5

    Proceedings-Print.Pdf ISSN: 2175-6759 ISBN: 978-85-76694-75-5

    Edited by: Flávio Luiz Schiavoni Rodrigo Schramm José Eduardo Fornari Novo Junior Leandro Lesqueves Costalonga ISSN 2175-6759 Ficha catalográfica elaborada pelo Setor de Processamento Técnico da Divisão de Biblioteca da UFSJ Simpósio Brasileiro de Computação Musical (15. : 2015 : Campinas, SP) Anais [recurso eletrônico] do 15º Simpósio Brasileiro de Computação Musical = 15th Brazilian Symposium on Computer Music (SBCM), 23 a 25 de novembro de 2015, Campinas, SP / editado por Flávio Luiz Schiavoni ... [et al.]. – Campinas: UNICAMP, 2015. Disponível em: http://compmus.ime.usp.br/sbcm2015/files/proceedings-print.pdf ISSN: 2175-6759 ISBN: 978-85-76694-75-5 1. Música por computador. 2. Arte e tecnologia. 3. Multimídia (Arte). I. Schiavoni, Flávio Luiz (Ed.). II. Título. CDU: 78:004 SBCM 2015 is organized by University of Campinas (UNICAMP) President: Jos´eTadeu Jorge Vice President for University Coordination: Alvaro´ Penteado Cr´osta Vice President for Research (PRP): Gl´aucia Maria Pastore Coordination of Interdisciplinary Centers (COCEN) Coordinator: Jurandir Zullo Junior Interdisciplinary Center for Studies on Sound Communication (NICS) Coordinator: Adriana do Nascimento Ara´ujo Mendes Art Institute, Department of Music Director: Fernando Augusto de Almeida Hashimoto Chief of the Department: Leandro Barsalini Coordinator of Graduate Studies in Music: Alexandre Zamith Almeida Coordinator of Undergraduate Studies in Music: Paulo J. Siqueira Tin´e Production Center Staff (Ceprod) Visual programming: Ivan Avelar Promotion Brazilian Computer
  • ESA: a CLIM Library for Writing Emacs-Style Applications

    ESA: a CLIM Library for Writing Emacs-Style Applications

    ESA: A CLIM Library for Writing Emacs-Style Applications Robert Strandh Troels Henriksen LaBRI, Université Bordeaux 1 DIKU, University of Copenhagen 351, Cours de la Libération Universitetsparken 1, Copenhagen 33405 Talence Cedex [email protected] France [email protected] David Murray Christophe Rhodes ADMurray Associates Goldsmiths, University of London 10 Rue Carrier Belleuse, 75015 Paris New Cross Road, London, SE14 6NW [email protected] [email protected] ABSTRACT style applications. The central distinguishing feature of an We describe ESA (for Emacs-Style Application), a library Emacs-style application, for our purposes, is that it uses for writing applications with an Emacs look-and-feel within short sequences of keystrokes as the default method of in- the Common Lisp Interface Manager. The ESA library voking commands, and only occasionally requires the user to takes advantage of the layered design of CLIM to provide a type the full name of the command in a minibuffer. A spe- command loop that uses Emacs-style multi-keystroke com- cial keystroke (M-x) is used to invoke commands by name. mand invocation. ESA supplies other functionality for writ- This interaction style is substantially different from the one ing such applications such as a minibuffer for invoking ex- provided by CLIM by default, and therefore required us to tended commands and for supplying command arguments, write a different CLIM top level. Fortunately, CLIM was Emacs-style keyboard macros and numeric arguments, file designed to make this not only possible but fairly straight- and buffer management, and more. ESA is currently used forward, as the implementation of a replacement top level in two major CLIM applications: the Climacs text editor can build on the protocol layers beneath, just as the default (and the Drei text gadget integrated with the McCLIM im- top level is built.
  • 92 Airq Sonification As a Context for Mutual Contribution Between

    92 Airq Sonification As a Context for Mutual Contribution Between

    ARANGO, J.J. AirQ Sonification as a context for mutual contribution between Science and Music Revista Música Hodie, Goiânia, V.18 - n.1, 2018, p. 92-102 AirQ Sonification as a context for mutual contribution between Science and Music Julián Jaramillo Arango (Caldas University Manizales, Colombia) [email protected] Abstract. This paper addresses a high-level discussion about the links between Science and Music, focusing on my own sonification and auditory display practices around air quality. Grounded on recent insights on interdisciplina- rity and sonification practices, the first sections will point out some potential contributions from scientific research to music studies and vice versa. I will propose the concept of mutualism to depict the interdependent status of Mu- sic and Science. The final sections will discuss air contamination as a complex contemporary problem, and will re- port three practice-based-design projects, AirQ jacket, Esmog Data and Breathe!, which outline different directions in facing local environmental awareness. Keywords. Science and Music, Sonification, Environmental Awarness, Sonology. A sonificação do AirQ como um contexto de contribuição mutua entre a Ciência e a Música. Resumo. Este artigo levanta uma discussão geral sobre as relações entre a Ciência e a Música, e se concentra em minhas praticas de sonificação e display auditivo da qualidade do ar. Baseado em reflexões recentes sobre a interdis- ciplinaridade e as práticas de sonificação as primeiras seções identificam algumas contribuições potenciais da pes- quisa científica aos estudos musicais e vice versa. O conceito de mutualismo será proposto para expressar a interde- pendência entre Música e Ciência. As seções finais discutem a contaminação do ar como um problema complexo de nossos dias e descreve três projetos de design-baseado-na-prática, AirQ jacket, Esmog Data e Breathe! que propõem direções diferentes para gerar consciência ambiental no contexto local.
  • Connecting Time and Timbre Computational Methods for Generative Rhythmic Loops Insymbolic and Signal Domainspdfauthor

    Connecting Time and Timbre Computational Methods for Generative Rhythmic Loops Insymbolic and Signal Domainspdfauthor

    Connecting Time and Timbre: Computational Methods for Generative Rhythmic Loops in Symbolic and Signal Domains Cárthach Ó Nuanáin TESI DOCTORAL UPF / 2017 Thesis Director: Dr. Sergi Jordà Music Technology Group Dept. of Information and Communication Technologies Universitat Pompeu Fabra, Barcelona, Spain Dissertation submitted to the Department of Information and Communication Tech- nologies of Universitat Pompeu Fabra in partial fulfillment of the requirements for the degree of DOCTOR PER LA UNIVERSITAT POMPEU FABRA Copyright c 2017 by Cárthach Ó Nuanáin Licensed under Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 Music Technology Group (http://mtg.upf.edu), Department of Information and Communication Tech- nologies (http://www.upf.edu/dtic), Universitat Pompeu Fabra (http://www.upf.edu), Barcelona, Spain. III Do mo mháthair, Marian. V This thesis was conducted carried out at the Music Technology Group (MTG) of Universitat Pompeu Fabra in Barcelona, Spain, from Oct. 2013 to Nov. 2017. It was supervised by Dr. Sergi Jordà and Mr. Perfecto Herrera. Work in several parts of this thesis was carried out in collaboration with the GiantSteps team at the Music Technology Group in UPF as well as other members of the project consortium. Our work has been gratefully supported by the Department of Information and Com- munication Technologies (DTIC) PhD fellowship (2013-17), Universitat Pompeu Fabra, and the European Research Council under the European Union’s Seventh Framework Program, as part of the GiantSteps project ((FP7-ICT-2013-10 Grant agreement no. 610591). Acknowledgments First and foremost I wish to thank my advisors and mentors Sergi Jordà and Perfecto Herrera. Thanks to Sergi for meeting me in Belfast many moons ago and bringing me to Barcelona.
  • SONIFICATION with MUSICAL CHARACTERISTICS: a PATH GUIDED by USER ENGAGEMENT Jonathan Middleton1, Jaakko Hakulinen2, Katariina Ti

    SONIFICATION with MUSICAL CHARACTERISTICS: a PATH GUIDED by USER ENGAGEMENT Jonathan Middleton1, Jaakko Hakulinen2, Katariina Ti

    The 24th International Conference on Auditory Display (ICAD 2018) June 10-15, 2018, Michigan Technological University SONIFICATION WITH MUSICAL CHARACTERISTICS: A PATH GUIDED BY USER ENGAGEMENT Jonathan Middleton1, Jaakko Hakulinen2, Katariina Tiitinen2, Juho Hella2, Tuuli Keskinen2, Pertti Huuskonen2, Juhani Linna2, Markku Turunen2, Mounia Ziat3 and Roope Raisamo2 1 Eastern Washington University, Department of Music, Cheney, WA 99203 USA, [email protected] 2 University of Tampere, Tampere Unit for Computer-Human Interaction (TAUCHI), Tampere, FI-33014 Finland, {firstname.lastname}@uta.fi 3Northern Michigan University, Dept. Psychological Science, Marquette, MI 49855 USA, [email protected] ABSTRACT The use of musical elements and characteristics in sonification has been formally explored by members of Sonification with musical characteristics can engage the International Community for Auditory Display since users, and this dynamic carries value as a mediator the mid 1990’s [4], [5], [6]. A summary of music-related between data and human perception, analysis, and research and software development in sonification can interpretation. A user engagement study has been be obtained from Bearman and Brown [7]. Earlier designed to measure engagement levels from conditions attempts were made by Pollack and Flicks in 1954 [8], within primarily melodic, rhythmic, and chordal yet according to Paul Vickers [9] and other researchers, contexts. This paper reports findings from the melodic such as, Walker and Nees [10], the path for music in portion of the study, and states the challenges of using sonification remains uncertain or unclear. Vickers musical characteristics in sonifications via the recently raised the significant question: “how should perspective of form and function – a long standing sonification designers who wish their work to be more debate in Human-Computer Interaction.
  • The Sonification Handbook Chapter 4 Perception, Cognition and Action In

    The Sonification Handbook Chapter 4 Perception, Cognition and Action In

    The Sonification Handbook Edited by Thomas Hermann, Andy Hunt, John G. Neuhoff Logos Publishing House, Berlin, Germany ISBN 978-3-8325-2819-5 2011, 586 pages Online: http://sonification.de/handbook Order: http://www.logos-verlag.com Reference: Hermann, T., Hunt, A., Neuhoff, J. G., editors (2011). The Sonification Handbook. Logos Publishing House, Berlin, Germany. Chapter 4 Perception, Cognition and Action in Auditory Display John G. Neuhoff This chapter covers auditory perception, cognition, and action in the context of auditory display and sonification. Perceptual dimensions such as pitch and loudness can have complex interactions, and cognitive processes such as memory and expectation can influence user interactions with auditory displays. These topics, as well as auditory imagery, embodied cognition, and the effects of musical expertise will be reviewed. Reference: Neuhoff, J. G. (2011). Perception, cognition and action in auditory display. In Hermann, T., Hunt, A., Neuhoff, J. G., editors, The Sonification Handbook, chapter 4, pages 63–85. Logos Publishing House, Berlin, Germany. Media examples: http://sonification.de/handbook/chapters/chapter4 8 Chapter 4 Perception, Cognition and Action in Auditory Displays John G. Neuhoff 4.1 Introduction Perception is almost always an automatic and effortless process. Light and sound in the environment seem to be almost magically transformed into a complex array of neural impulses that are interpreted by the brain as the subjective experience of the auditory and visual scenes that surround us. This transformation of physical energy into “meaning” is completed within a fraction of a second. However, the ease and speed with which the perceptual system accomplishes this Herculean task greatly masks the complexity of the underlying processes and often times leads us to greatly underestimate the importance of considering the study of perception and cognition, particularly in applied environments such as auditory display.
  • ESA: a CLIM Library for Writing Emacs-Style Applications

    ESA: a CLIM Library for Writing Emacs-Style Applications

    ESA: A CLIM Library for Writing Emacs-Style Applications Robert Strandh Troels Henriksen LaBRI, Université Bordeaux 1 DIKU, University of Copenhagen 351, Cours de la Libération Universitetsparken 1, Copenhagen 33405 Talence Cedex [email protected] France [email protected] David Murray Christophe Rhodes ADMurray Associates Goldsmiths, University of London 10 Rue Carrier Belleuse, 75015 Paris New Cross Road, London, SE14 6NW [email protected] [email protected] ABSTRACT style applications. The central distinguishing feature of an We describe ESA (for Emacs-Style Application), a library Emacs-style application, for our purposes, is that it uses for writing applications with an Emacs look-and-feel within short sequences of keystrokes as the default method of in- the Common Lisp Interface Manager. The ESA library voking commands, and only occasionally requires the user to takes advantage of the layered design of CLIM to provide a type the full name of the command in a minibuffer. A spe- command loop that uses Emacs-style multi-keystroke com- cial keystroke (M-x) is used to invoke commands by name. mand invocation. ESA supplies other functionality for writ- This interaction style is substantially different from the one ing such applications such as a minibuffer for invoking ex- provided by CLIM by default, and therefore required us to tended commands and for supplying command arguments, write a different CLIM top level. Fortunately, CLIM was Emacs-style keyboard macros and numeric arguments, file designed to make this not only possible but fairly straight- and buffer management, and more. ESA is currently used forward, as the implementation of a replacement top level in two major CLIM applications: the Climacs text editor can build on the protocol layers beneath, just as the default (and the Drei text gadget integrated with the McCLIM im- top level is built.
  • Signal Processing for Music Analysis Meinard Müller, Member, IEEE, Daniel P

    Signal Processing for Music Analysis Meinard Müller, Member, IEEE, Daniel P

    IEEE JOURNAL OF SELECTED TOPICS IN SIGNAL PROCESSING, VOL. 0, NO. 0, 2011 1 Signal Processing for Music Analysis Meinard Müller, Member, IEEE, Daniel P. W. Ellis, Senior Member, IEEE, Anssi Klapuri, Member, IEEE, and Gaël Richard, Senior Member, IEEE Abstract—Music signal processing may appear to be the junior consumption, which is not even to mention their vital role in relation of the large and mature field of speech signal processing, much of today’s music production. not least because many techniques and representations originally This paper concerns the application of signal processing tech- developed for speech have been applied to music, often with good niques to music signals, in particular to the problems of ana- results. However, music signals possess specific acoustic and struc- tural characteristics that distinguish them from spoken language lyzing an existing music signal (such as piece in a collection) to or other nonmusical signals. This paper provides an overview of extract a wide variety of information and descriptions that may some signal analysis techniques that specifically address musical be important for different kinds of applications. We argue that dimensions such as melody, harmony, rhythm, and timbre. We will there is a distinct body of techniques and representations that examine how particular characteristics of music signals impact and are molded by the particular properties of music audio—such as determine these techniques, and we highlight a number of novel music analysis and retrieval tasks that such processing makes pos- the pre-eminence of distinct fundamental periodicities (pitches), sible. Our goal is to demonstrate that, to be successful, music audio the preponderance of overlapping sound sources in musical en- signal processing techniques must be informed by a deep and thor- sembles (polyphony), the variety of source characteristics (tim- ough insight into the nature of music itself.
  • The Sonification Handbook Chapter 3 Psychoacoustics

    The Sonification Handbook Chapter 3 Psychoacoustics

    The Sonification Handbook Edited by Thomas Hermann, Andy Hunt, John G. Neuhoff Logos Publishing House, Berlin, Germany ISBN 978-3-8325-2819-5 2011, 586 pages Online: http://sonification.de/handbook Order: http://www.logos-verlag.com Reference: Hermann, T., Hunt, A., Neuhoff, J. G., editors (2011). The Sonification Handbook. Logos Publishing House, Berlin, Germany. Chapter 3 Psychoacoustics Simon Carlile Reference: Carlile, S. (2011). Psychoacoustics. In Hermann, T., Hunt, A., Neuhoff, J. G., editors, The Sonification Handbook, chapter 3, pages 41–61. Logos Publishing House, Berlin, Germany. Media examples: http://sonification.de/handbook/chapters/chapter3 6 Chapter 3 Psychoacoustics Simon Carlile 3.1 Introduction Listening in the real world is generally a very complex task since sounds of interest typically occur on a background of other sounds that overlap in frequency and time. Some of these sounds can represent threats or opportunities while others are simply distracters or maskers. One approach to understanding how the auditory system makes sense of this complex acoustic world is to consider the nature of the sounds that convey high levels of information and how the auditory system has evolved to extract that information. From this evolutionary perspective, humans have largely inherited this biological system so it makes sense to consider how our auditory systems use these mechanisms to extract information that is meaningful to us and how that knowledge can be applied to best sonify various data. One biologically important feature of a sound is its identity; that is, the spectro-temporal characteristics of the sound that allow us to extract the relevant information represented by the sound.
  • Gravity's Reverb: Listening to Space-Time, Or Articulating The

    Gravity's Reverb: Listening to Space-Time, Or Articulating The

    GRAVITY’S REVERB: Listening to Space-Time, or Articulating the Sounds of Gravitational-Wave Detection STEFAN HELMREICH Massachusetts Institute of Technology http://orcid.org/0000-0003-0859-5881 I heard gravitational waves before they were detected.1 I was sitting in a pub in May 2015 with MIT physicists Scott Hughes and David Kaiser, my headphones looped into a laptop. Hughes was readying to play us some interstellar sounds: digital audio files, he explained, that were sonic simulations of what gravitational waves might sound like if, one day, such cosmic undulations arrived at detection devices on Earth. Gravitational waves are tiny but consequential wriggles in space-time, first theorized by Einstein in 1916— vibrations generated, for example, by such colossal events as the collision of black holes or the detonation of supernovae. Listening to Hughes’s .wav representations of black holes spiraling into one another, I heard washing-machine whirs, Ther- emin-like glissandos, and cybernetic chirps—noises reminiscent, I mused, of mid- twentieth century sci-fi movies (see Taylor 2001 on space-age music). Audio 1. Circular inspiral, spin 35.94% of maximum, orbital plane 0Њ,0Њ viewing angle. Created by Pei-Lan Hsu, using code written by Scott Hughes. Audio 2. Generic inspiral, eccentricity e = 0.7, inclination i =25Њ. Created by Pei-Lan Hsu, using code written by Scott Hughes. CULTURAL ANTHROPOLOGY, Vol. 31, Issue 4, pp. 464–492, ISSN 0886-7356, online ISSN 1548-1360. ᭧ by the American Anthropological Association. All rights reserved. DOI: 10.14506/ca31.4.02 GRAVITY’S REVERB These were sounds that, as it turned out, had a family resemblance to the actual detection signal that, come February 2016, would be made public by astronomers at the Laser Interferometer Gravitational-Wave Observatory (LIGO), an MIT-Caltech astronomy collaboration anchored at two massive an- tennae complexes, one in Hanford, Washington, another in Livingston, Louisiana.
  • Comparison and Evaluation of Sonification Strategies for Guidance Tasks Gaetan Parseihian, Charles Gondre, Mitsuko Aramaki, Sølvi Ystad, Richard Kronland Martinet

    Comparison and Evaluation of Sonification Strategies for Guidance Tasks Gaetan Parseihian, Charles Gondre, Mitsuko Aramaki, Sølvi Ystad, Richard Kronland Martinet

    Comparison and Evaluation of Sonification Strategies for Guidance Tasks Gaetan Parseihian, Charles Gondre, Mitsuko Aramaki, Sølvi Ystad, Richard Kronland Martinet To cite this version: Gaetan Parseihian, Charles Gondre, Mitsuko Aramaki, Sølvi Ystad, Richard Kronland Mar- tinet. Comparison and Evaluation of Sonification Strategies for Guidance Tasks. IEEE Transac- tions on Multimedia, Institute of Electrical and Electronics Engineers, 2016, 18 (4), pp.674-686. 10.1109/TMM.2016.2531978. hal-01306618 HAL Id: hal-01306618 https://hal.archives-ouvertes.fr/hal-01306618 Submitted on 25 Apr 2016 HAL is a multi-disciplinary open access L’archive ouverte pluridisciplinaire HAL, est archive for the deposit and dissemination of sci- destinée au dépôt et à la diffusion de documents entific research documents, whether they are pub- scientifiques de niveau recherche, publiés ou non, lished or not. The documents may come from émanant des établissements d’enseignement et de teaching and research institutions in France or recherche français ou étrangers, des laboratoires abroad, or from public or private research centers. publics ou privés. JOURNAL OF LATEX CLASS FILES, VOL. ?, NO. ?, ? 2016 1 Comparison and evaluation of sonification strategies for guidance tasks Gaetan¨ Parseihian, Charles Gondre, Mitsuko Aramaki, Senior Member, IEEE, Sølvi Ystad, Richard Kronland Martinet, Senior Member, IEEE Abstract This article aims to reveal the efficiency of sonification strategies in terms of rapidity, precision and overshooting in the case of a one-dimensional guidance task. The sonification strategies are based on the four main perceptual attributes of a sound (i.e. pitch, loudness, duration/tempo and timbre) and classified with respect to the presence or not of one or several auditory references.