Sweet [re]production: Developing sound spatialization tools for musical applications with emphasis on sweet spot and off-center perception Nils Peters Music Technology Area Department of Music Research Schulich School of Music McGill University Montreal, QC, Canada October 2010 A thesis submitted to McGill University in partial fulfillment of the requirements for the degree of Doctor of Philosophy. c 2010 Nils Peters 2010/10/26 i Abstract This dissertation investigates spatial sound production and reproduction technology as a mediator between music creator and listener. Listening experiments investigate the per- ception of spatialized music as a function of the listening position in surround-sound loud- speaker setups. Over the last 50 years, many spatial sound rendering applications have been developed and proposed to artists. Unfortunately, the literature suggests that artists hardly exploit the possibilities offered by novel spatial sound technologies. Another typical drawback of many sound rendering techniques in the context of larger audiences is that most listeners perceive a degraded sound image: spatial sound reproduction is best at a particular listening position, also known as the sweet spot. Structured in three parts, this dissertation systematically investigates both problems with the objective of making spatial audio technology more applicable for artistic purposes and proposing technical solutions for spatial sound reproductions for larger audiences. The first part investigates the relationship between composers and spatial audio tech- nology through a survey on the compositional use of spatialization, seeking to understand how composers use spatialization, what spatial aspects are essential and what functionali- ties spatial audio systems should strive to include. The second part describes the development process of spatializaton tools for musical applications and presents a technical concept. The Virtual Microphone Control (ViMiC) system is an auditory virtual environment that recreates a recording situation through vir- tual sound sources, virtual room properties and virtual microphones. A technical concept is presented to facilitate artistic work with spatial audio systems and to allow the combination of different spatialization tools. The third part investigates the perception of spatialized sounds as a function of the listening positions in multichannel sound systems. Perceptual experiments were designed to ii understand the multidimensional nature of an off-center sound degradation and to propose concepts to improve the listening conditions for larger audiences. This research extends our understanding of spatial audio perception and has potential value to all those interested in spatial audio quality, including designers, creators and specialists in the fields of acoustics, music, technology and auditory perception. iii Abr´eg´e Cette th`esea pour objet l'´etudedes dispositifs de production et de reproduction des sons spatialis´es.Plus particuli`erement, ce travail comporte plusieurs exp´eriences´evaluant l’influence de la position d'un auditeur au sein d'un dispositif de haut-parleur de type \surround" sur la perception d'oeuvres musicales. Au cours des 50 derni`eresann´ees,de nombreuses applications destin´ees`ala mise en espace de sources sonores ont vu le jour. Cependant, deux probl`emesprincipaux persistent. D'une part, il semble que les artistes n'exploitent que tr`essuperficiellement le potentiel de ces nouveaux outils de cr´eation.D'autre part, dans la majeure partie des cas, lorsque les dispositifs de reproduction de sons spatialit´esont utilis´espour un grand nombre de spec- tateurs, seulement une fraction du public peut vraiment percevoir les nuances et subtilit´es de la spatialisation; ces privil´egi´esse situent dans une zone d'´ecouteplus connue sous le nom de \sweet spot", caract´eristique des syst`emes de reproduction du son dans l'espace. La recherche syst´ematiquede solutions `aces deux probl`emes constituent le coeur de ce travail. Cette th`eseest divis´eeen trois parties. Dans la premi`ere,gr^ace`aun sondage effectu´eaupr`esde compositeurs, nous avons essay´e de mieux cerner comment la spatialisation s'int`egredans leur processus de composition, et aussi quels sont leurs exigences vis-a-vis des syst`emesde spatialisation afin de faire l'inventaire des fonctionnalit´esessentielles d'un tel syst`eme. Dans la seconde partie de cette th`ese,le d´eveloppement d'un nouvel outil pour la mise en espace de sources sonores est pr´esent´e: le syst`emeVirtual Microphone Control (ViMiC). Cette application est un environnement auditif virtuel (EAV) qui permet de recr´eerdes conditions d'enregistrement particuli`eresgr^ace`ala mod´elisationde salles, de microphones et de sources sonores. Une approche technique permettant l'utilisation facile de plusieurs outils de spatialisation `ades fins artistiques est aussi pr´esent´ee. Enfin, dans la troisi`emepartie de cet ouvrage, les r´esultatsde diff´erentes ´etudesper- iv ceptives sont d´etaill´espour identifier l’influence de la position des auditeurs lors de per- formances en son \surround". Ces ´etudesont ´et´emises en place afin de caract´eriserde mani`ereapprofondie la d´egradationde la perception sonore en fonction de la position d'un auditeur. Des solutions pratiques permettant de minimiser ces effets n´egatifssont aussi propos´es. Ce travail de recherche permet d'´etendrenotre compr´ehensionde la perception des sons spatialis´es.D'autre part, cette ´etudeest certainement pr´ecieusepour ceux et celles travail- lant `al'am´elioration de la qualit´edes sons spatialis´e,qu'ils soient techniciens, compositeurs, sp´ecialistesen acoustique, en musique ou encore en perception auditive. v Acknowledgments A number of people has been of great importance to this dissertation. First and foremost I would like to thank my advisors Stephen McAdams and Jonas Braasch for bringing me to McGill and for invaluable guidance over the course of this dissertation. I fell lucky to had the opportunity in learning from Marcelo M. Wanderley, Philippe Depalle, Gary Scavone, Vincent Verfaille, Sean Ferguson, Fabrice Marandola, William M. Martens, Martha De Francisco, Catherine Guastavino, Evan Balaban and Yoshio Takane. Of great importance were the members of the Music Perception and Cognition Lab, in particular Bruno L. Giordano, Georgios Marentakis, Bennett Smith and Finn Upham. I am thankful for all the discussions with students from Sound Recording and Music Technology, in particular Sungyoung Kim, Joe Malloch, Mark Marshall, Steve Sinclair, Doug van Nort, Avrum Hollinger, Carmine Casciato, Andrey da Silva, Mark Zadel, Sven-Amin Lembke and Bertrand Scherrer, who also helped me with the Abr´eg´e. Special thanks to Tristan Matthews for supporting me on the ViMiC development. I should not forget to thank Darryl Cameron and the CIRMMT crew Harold Kilian- ski, Yves Methot, Julien Boissinot, Rosemary Marland and Sara Gomez for technical and administrative support. I also want to mention the close collaboration over the last years with Tim Place, Trond Lossius, Jan Schacher, Marlon Schumacher, Alexander Refsum Jensenius, Pascal Baltazar and Th´eode la Hogue in the Jamoma project and the SpatDIF initiative. I should not forget to acknowledge the Canada Council for the Arts (CAA), the Natural Science and Engineering Research Council of Canada (NSERC) and the Centre for Inter- disciplinary Research in Music Media and Technology (CIRMMT) for financial support. I also thank Theresa Leonard from the Banff Centre for a research residency in 2007. Finally, above all, thanks to my family and Sandra for all the support and encourage- ments over the years. vi vii Contribution of Authors The document is formatted as a manuscript dissertation and includes the following publi- cations. • Chapter2: Peters N. , Marentakis G., McAdams S.: Current technologies and com- positional practices for spatialization: A qualitative and quantitative analysis. to appear in Computer Music Journal (CMJ), Spring 2011. • Chapter3: Peters N. , Lossius T., Schacher J., Baltazar P., Bascou C., Place T.: A Stratified Approach for Sound Spatialization. In Proc. of the 6th Sound and Music Computing Conference (SMC), pages 219{224, Porto, Portugal, 2009. • Chapter4: Peters N. , Matthews T., Braasch J., McAdams S.: Spatial Sound Ren- dering in Max/MSP with ViMiC. In Proc. of the International Computer Music Conference (ICMC), pages 755{758, Belfast, Northern Ireland, 2008. • Chapter5: Peters N. , Braasch J., McAdams S.: Learning from Users - Case Studies for Real-time Spatialization with ViMiC. Submitted. • Chapter6: Peters N. , Braasch J., McAdams S.: Auditory perception at off-center listening positions in surround sound environments. Submitted to Journal of the Acoustical Society of America, 2009. currently under revision • Chapter7: Peters N. , McAdams S.: Off-center Sound Degradation Based on Geo- metrical Properties in 5.1 loudspeaker setups. Manuscript prepared for submission. I was responsible for designing and carrying out all experiments, including the data collec- tion, conduction the data analysis, and preparing the manuscripts for all of the above listed publications. As my primary supervisor, Stephen McAdams provided necessary funding, laboratory space and equipment. Further, he generally contributed guidance in the con- ception and interpretation of the experimental data. Jonas Braasch, my secondary advisor, guided me primarily on technical aspects throughout the dissertation and provided the viii initial programming code