Exploring 3D Audio as a New Musical Language Timothy Schmele Autonomous Generation of Soundscapes using Unstructured Sound Databases Submitted for the award of a Master of Science in Sound and Music Computing. Nathaniel Finney Music Technology Group Departament of Information and Communication Technologies Universitat Pompeu Fabra MASTER THESIS UPF / 2009 Master in Sound and Music Computing Master thesis supervisor: JordiSupervisors: Janer Pau Arum´ı,Josep Comacunjosas Department of Information and CommunicationSubmitted Technologies to the Pompeu Fabra University, September, 2011 Universitat Pompeu Fabra, Barcelona Declaration Full Title: Exploring 3D Audio as a New Musical Language Supervisors: Pau Arum´ı Josep Comacunjosas This thesis is presented in partial fulfillment of the requirements for the degree of Master of Science in Sound and Music Computing. It is entirely my own work and has not been submitted to any other university or higher education institution, or for any other academic award in this university. Where use has been made of the work of other people it has been fully acknowledged and fully referenced Signature: Timothy Schmele September 1, 2011 { Barcelona, Catalonia, Spain i Acknowledgements I would like to thank both my supervisors Pau Arum´ıand Josep Comacunjosas for all their help, advice and support. Many thanks go out to Toni Mateos, as well as the whole crew at Barcelona Media for their help and for giving me the opportunity to carry out my work in their facilities. Without the technology they provided and the trust Barcelona Media showed in giving me full access to the studio, this thesis would be unthinkable! Thank you also to the Sound and Music Computing Master program coordinator, Xavier Serra, and all the staff at the Music Technology Group, UPF and ESMUC for sharing their knowledge and providing their competence with the upmost respect. Furthermore, I want to stress my gratitude to a few people who explicitly helped by providing their expertise: Alexandros Katsakos, for an inspiring discussion about the aural perception of blind people, Felix Tutzer & Jan van Balen, for the many five-in-the-morning brainstorms, Imanol Gomez, for his great collaboration audifying brainstorms, Katrin Neue, for her expertise on postmodern dance, Kerry Hagan, for her valuable insights into textural composition, Arthur Sauer, for his help on Wave Field Synthesis, as well as Robert Sazdov and Natasha Barrett, who shared their research without hesitation and especially Maja Trochimczyk, for scanning in and sharing her complete PhD thesis on her own initiative! I am also grateful to the German Academic Exchange Service (DAAD) for selecting and giving me the graduate scholarship for German students studying abroad, without which my whole studies at the Pompeu Fabra University and my life here in Barcelona would not be as fruitful as it was! Special thanks go to all my class mates, the chungos and the mijas, at the Sound and Music Computing Master program for all their support, help and especially for their friendship, with whom I shared some great, memorable times during the creation of this work. It was highly inspiring to work in such an international group of bright and creative minds. Let's have one more after this is finished... or five... Very special thanks and love to my parents and family for all the support and love one could possibly wish for. Although I copy and paste this sentence every time, its meaning is as strong as ever! ii Xenakis! Abstract This thesis explores the artistic potential to make use of three dimensional, physical space as a new musical language. It does so in three ways: It first investigates the greater history of spatialization and traces its epitaxy as an emancipated musical parameter up to recent developments since the new millennium. The second part then brings these historic ideas together into a contemporary context, in the effort to further establish space a prime vessel able to carry a musical message of expression. Several approaches to spatial composition are elaborated, keeping previously mentioned psychoacoustic limitations in mind. Finally, in the practical part of the thesis, two different three-dimensional spatial sound synthesis methods are presented, followed by a qualitative evaluation of their use. Based on the insights { and especially the complications encountered relating to the success of this approach, the thesis concludes by arguing the necessary conditions under which space as a musical parameter might hopefully be established. iv Συνοψη´ 1 Η διατριβή αυτή εξερευνά την καλλιτεχνική δυνατότητα χρησιμοποίησης του τρισδιάστατου φυσικού χώρου σαν μια καινούρια μουσική γλώσσα. Το επιτυγχάνει με τρεις τρόπους: Αρχικά, διερευνά την μακρά ιστορία της χωροδιάστασης και εξετάζει την ανέλιξη της σαν μια ανεξάρτητη μουσική παραμετρος μέχρι και τις τελευταίες επιτεύξεις της νέας χιλιετίας. Το δεύτερο μέρος ενσωματώνει αυτές τις ιστορικές ιδέες σε ενα σύγχρονο πλαίσιο, στην προσπάθεια κατοχύρωσης του χώρου σαν κύριο μέσο μεταφοράς στην έκφραση ενός μουσικού μηνύματος. Παρατίθεται ανάλυση των διαφόρων μεθόδων μουσικής σύνθεσης στον χώρο, έχοντας πρωτίστως υπόψη τους περιορισμούς της ψυχοακουστικής. Τέλος, στο πρακτικό μέρος της διατριβής, παρουσιάζονται δύο διαφορετικές συνθέσεις ήχου σε τρισδιάστατο χώρο, η χρήση των οποίων ακολουθείται από ποιοτική αξιολόγηση. Με βάση τις γνώσεις, και κυρίως τις πολυπλοκότητες που συνδέονται με την επιτυχία της μεθόδου αυτής, η διατριβή ολοκληρώνεται επιχειρηματολογώντας τις αναγκαίες συνθήκες κάτω από τις οποίες ο χώρος θα μπορέσει επιτέλους να κατοχυρωθεί σαν μια μουσική παράμετρος. 1..για την θεία προσωποπρέπεια του! v Contents Declaration i Acknowledgements ii Abstract iv 1 Introduction 1 2 Historical Backgrounds9 2.1 From Antiphony to Theater: Space in Acoustic Ensembles............. 10 2.1.1 Polychorality in Antiphonic Music...................... 11 2.1.2 Space as a Means of Dramaturgy....................... 13 2.1.3 Symbolical Spaces............................... 17 2.2 Technological Developments in Multichannel Speaker Arrays........... 19 2.3 Spatial Parameters in the Electro-acoustic Tradition................ 25 2.3.1 A Space of Moving Objects.......................... 26 2.3.2 Serialization of the Spatial Parameter.................... 29 2.3.3 Musical Space in the Computer Age..................... 34 3 Spatial Composition Methodology 41 3.1 Psychoacoustic Implications.............................. 43 3.1.1 Localization on the Spherical Periphery................... 43 3.1.2 Distance Perception.............................. 48 Experiment I: Assessing the impact of linear signal expansion....... 53 Experiment II: Panning audio into the center of the listening space.... 55 3.1.3 Ecological Psychoacoustics.......................... 58 3.2 On the Notion of Space................................ 62 3.3 Composing the Space................................. 71 3.3.1 Point Source and Trajectorial Approaches.................. 75 Decorrelation effects.............................. 83 3dSpamlePlayer: a Max/MSP spatialization tool.............. 85 3.3.2 Architectural Pieces and Soundscapes.................... 89 3.3.3 Virtual Spaces and Spatial Textures..................... 92 vi 4 Spatial Sound Synthesis 99 4.1 Timbral Spatialization................................. 102 4.1.1 Implementation of an extensible Spectral Splitter.............. 103 4.1.2 Conclusions on the Spectral Splitter..................... 107 4.2 Rapid Panning Modulation Synthesis......................... 109 4.2.1 Implementation of the Sphere Panning Spatial Synthesis Module..... 111 4.2.2 Using Spynth.................................. 114 4.2.3 Gestural Synthesis............................... 117 5 Conclusions 119 5.1 Redefining the Musical Conception of a Whole Culture............... 122 Bibliography 128 vii Chapter 1 Introduction "The space parameter is no more an effect in pitch music, but pitch is only an effect in space music. Space as a finality in music expression." {Leo Kupper1 Establishing the physical space2 as a musical language might alienat those who think of music in terms of pitches and melodies, but if one looks at the history of musical development, spatial aspects and considerations have always been part of the respective periods own respec- tive aesthetics. Space is present in most musical vocabulary, as well as projected into many other musical characteristics and parameters. Traces can be found starting from the deliber- ate separation of ensemble parts to articulate antiphonal compositions in biblical times, and continuing with architecturally motivated compositions over symbolical spaces up to virtual soundscapes. This is possibly due to the recent developments in audio technology in the past 100 years. As pitch disappears, new building blocks such as sine tones, concrete sounds and noises are used. Since melodies and harmonies do not hold anymore, there is a need for a new parameter which brings more depth into the musical expression. Space and the concept of spatialization in electronic music today is a substantiated aspect of the music. The question, asked throughout this thesis, is the effectiveness in how far the spatial considerations in any example given herein constitutes as a true musical language. The way space is used in the electronic music domain is unique and radically new com- pared to any previous acoustic effort is an accepted fact (Harley 1994, Smalley 2007, Norman- deau 2009). As Normandeau(2009) points out, the development of the loudspeaker