Data Sonification in Creative Practice
Total Page：16
File Type：pdf, Size：1020Kb
University of Plymouth PEARL https://pearl.plymouth.ac.uk 04 University of Plymouth Research Theses 01 Research Theses Main Collection 2019 Data Sonification in Creative Practice Bonet Filella, Nuria http://hdl.handle.net/10026.1/14583 University of Plymouth All content in PEARL is protected by copyright law. Author manuscripts are made available in accordance with publisher policies. Please cite only the published version using the details provided on the item record or document. In the absence of an open licence (e.g. Creative Commons), permissions for further reuse of content should be sought from the publisher or author. DATA SONIFICATION IN CREATIVE PRACTICE by Núria Bonet Filella A thesis submitted to the University of Plymouth in partial fulfilment for the degree of DOCTOR OF PHILOSOPHY School of Humanities & Performing Arts September 2018 Copyright © 2018 This copy of the thesis has been supplied on condition that anyone who consults it is understood to recognise that its copyright rests with its author and that no quotation from the thesis and no information derived from it may be published without the author's prior consent. ii Acknowledgements I have spent a lot of time thinking about what a PhD is and why I should persevere with it. I have concluded that it is an academic apprenticeship which will set me up for my future career. The lessons learnt on this journey are far larger than can be expressed in the thesis and the portfolio. This would not have been possible without many people, some of whom I wish to thank specifically. It has been a privilege to work under Prof Eduardo Miranda who has provided continued support and inspiration. I thank him for his advice, time and last-minute all-important emails. He has allowed me the freedom to develop my practice and ideas while trusting that I would get there in the end. Dr Alexis Kirke has provided the ideal counter-balance as 2nd supervisor and I have sought to understand and at times emulate his practice. His success in bringing ideas alive has inspired me to believe in my own ideas, whether inspired or outrageous. This has given me an edge but also an understanding of the murky world of surviving and thriving in academia and the media. A PhD can be very isolating, so it would have been impossible to survive without the company of the ICCMR crew: Ed Braund, Jared Drayton, Ben Payne, Aurélien Antoine, Satvik Venkatesh, Michael McLoughlin, Federico Visi, Pierre-Emmanuel Largeron, Duncan Williams, Joel Eaton. The proximity to the Theatre and Performance department means that I have also spent many lunchtimes and late evenings in company of many other scholars: Charlotte Storey, Josh Slater, Victor De Ladrón Guevara, Natalie Raven, and many more. I would like to thank Simon Ible for playing my music and taking time to hone my skills. The respect that he and the Ten Tors Orchestra have shown me has encouraged me to persevere with my instrumental music. i I could not have survived the gruelling academic demands without the distraction and love from friends. I would like to thank Fran and Stacey for being true friends and cracking neighbours. All the times that I walked in unannounced, had dinner cooked for or talked over whichever programme we were watching kept me sane. Further down the road, the Fawn Social Club has been my home from home. You have kept me entertained, occupied, accompanied, hydrated, fed, astonished, distracted, and most importantly, accepted. I wish everyone had a Fawn Social Club in their life. Finally, nothing would be possible without Montserrat Filella. She is an endless source of wisdom, encouragement and love. My best decisions can be traced back to her and she is truly a leading light in my life. Gràcies mama. ii Author’s Declaration At no time during the registration for the degree of Doctor of Philosophy has the author been registered for any other University without prior agreement of the Doctoral College Quality Sub-Committee. Work submitted for this research degree at the University of Plymouth has not formed part of any other degree either at the University of Plymouth or at another establishment. The writing-up of the thesis was financially supported by the Bourse Michelle of the FOCUNA foundation (Luxembourg). A record of activity detailing the publications and conference presentations can be found in Chapter 8. Word count of main body of thesis: 46,660 Signed ……………………………… Date…………………………. iii Abstract Data Sonification in Creative Practice Núria Bonet Filella Sonification is the process of data transmission with non-speech audio. While finding increasing acceptance as a scientific method, particularly where a visual representation of data is inadequate, it is still often derided as a ‘gimmick’. Composers have also shown growing interest in sonification as a compositional method. Both in science and in music, the criticism towards this method relates to poor aesthetics and gratuitous applications. This thesis aims to address these issues through an accompanying portfolio of pieces which use sonification as a compositional tool. It establishes the principles of ‘musification’, which can be defined as a sonification which uses musical structures; a sonification organised by musical principles. The practice-as-research portfolio explores a number of data sources, musical genres and science-music collaborations. The main contributions to knowledge derived from the project are a portfolio of compositions, a compositional framework for sonification and an evaluation framework for musification. This thesis demonstrates the validity of practice-as- research as a methodology in sonification research. iv Table of Contents 1. Introduction 1 1.1 Research Context 3 1.2 Research Aims 5 1.3 Research Questions 6 1.4 Research Methods and Strategies 6 1.4.1 Literature Review 6 1.4.2 Practice-as-Research 7 1.4.3 Software 9 1.4.4 Evaluation 10 1.4.5 Between Methodologies 10 1.5 Thesis Overview 11 2. From Sonification to Musification 13 2.1 Sonification 13 2.1.1 Audification 13 2.1.2 Parameter Mapping Sonification (PMSon) 14 2.1.3 Auditory Icons 17 2.1.4 Model-based Sonification 18 2.1.5 Further Techniques 19 2.2 Towards a Definition 19 2.2.1 Auditory Display 21 2.2.2 Functional Sounds 22 2.3 Aesthetics 23 2.3.1 Aesthetics in Sonification 23 2.3.2 Aesthetics and Beauty 24 2.3.3 Psychoacoustics and Human Perception 25 2.4 Sonification and Music 27 2.4.1 ICAD 27 2.4.2 Popularity 28 2.4.3 Further Criticisms 29 2.5 Music 29 v 2.6 Musification 33 2.7 Trends in Musification 36 2.7.1 Natural Organisms 37 2.7.2 Human Body 39 2.7.3 Environment 41 2.7.4 Weather and Climate 44 2.7.5 Astronomy and Particle Physics 46 2.8 Conclusion 50 2.8.1 Musification 50 2.8.2 Challenges 50 2.8.3 Proposed Solutions 51 3. Blyth-Eastbourne-Wembury: A Framework for 52 Musification 3.1 Overview 52 3.1.1 Compositional Process 53 3.2 Data 54 3.2.1 Temperature 54 3.2.2 Salinity 54 3.2.3 Telling Climate Change 55 3.3 Mappings 59 3.3.1 Temperature 59 3.3.2 Salinity 60 3.3.3 Software 61 3.4 Musical Language 61 3.4.1 Sonification 61 3.4.2 Soundscape Composition 62 3.4.3 Musification 63 3.5 Emotional Content 64 3.5.1 Empathetic Sonification 64 3.5.2 Sonification 65 3.5.3 Soundscape 66 3.5.4 Geosonification 68 vi 3.5.5 Sonification as Sound Source 69 3.6 Performance 71 3.7 Framework 74 3.7.1 Data 74 3.7.2 Mappings 75 3.7.3 Musical Language 76 3.7.4 Emotional Content 77 3.8 Conclusions 77 4. The Sonification of Dark Matter: Interdisciplinary 79 Collaboration and the Scientist-Composer 4.1 Overview 79 4.1.1 Aims 80 4.1.2 Dark Matter 79 4.1.3 Challenges 82 4.1.4 Compositional Process 83 4.2 Data 83 4.2.1 First Approach: Data Files 84 4.2.2 Second Approach: Data Extraction from Visualisations 87 4.3 Mappings 88 4.3.1 Visualisation Mappings 88 4.3.2 Primary and Secondary Parameters 89 4.3.3 Hierarchies in Music 90 4.3.4 Mappings: Experimental Approach 91 4.3.5 Mappings: Warm Dark Matter Halo 91 4.3.6 Mappings: Dark Matter Streams 92 4.3.7 Mappings: Dark Universe 92 4.4 Musical Language 92 4.5 Emotional Content 94 4.6 Performance 95 4.7 Collaboration 95 4.7.1 Types of Collaboration 97 4.7.2 Example of Working Relationships in Sonification 97 vii 4.7.3 Working Relationships in The Sonification of Dark Matter 99 4.7.4 Challenges of Collaboration 100 4.7.5 Criticisms of Collaboration 101 4.7.6 Potential Solutions for Problems in Collaborations 102 4.8 Scientist-Composer 104 4.9 Conclusions 106 5. The Voice of the Sea: Echoing our Environment 108 5.1 Overview 108 5.1.1 Compositional Process 110 5.2 Data 110 5.3 Mappings 112 5.3.1 Synthesised Sounds: Beeps 114 5.3.2 Recorded Sounds: Ocean Sounds 116 5.3.3 Recorded Sounds: Pebbles 117 5.3.4 Synthesised Sounds: Bass Beat 119 5.3.5 Recorded Sounds: Boats and Seagulls 120 5.3.6 Two Recorded Sounds: 2d.wave~ and Ocean Sounds 120 5.4 Performance 120 5.5 Musical Language 123 5.5.1 Soundscape Music 123 5.5.2 Spatialisation 124 5.6 Emotional Content 126 5.6.1 (Re-)creating a Soundscape 126 5.6.2 Empathetic Sonification 127 5.6.3 Climate Change 129 5.6.4 A Tradition 129 5.6.5 Sound Buoys 133 5.7 Collaboration 134 5.8 Conclusions 135 6.