TAXONOMY AND NOTATION OF SPATIALIZATION Emile Ellberger Germán Toro Pérez Linda Cavaliero Institute for Computer Music Institute for Computer Music Institute for Computer Music and and Sound Technology and Sound Technology Sound Technology Zurich University of the Arts Zurich University of the Arts Zurich University of the Arts [email protected] [email protected] [email protected] Johannes Schütt Giorgio Zoia Basile Zimmermann Institute for Computer Music Institute for Computer Music Faculty of Humanities and Sound Technology and Sound Technology University of Geneva Zurich University of the Arts Zurich University of the Arts [email protected] [email protected] [email protected] ABSTRACT that include implementation of spatialization have been limited in their capacity to propose adequate notation The SSMN Spatial Taxonomy and its symbols libraries, possibilities to convey spatial information within musical which are the corner stone of the Spatialization Symbolic scores. In spite of the availability of a variety of strategies Music Notation (SSMN) project, emanates from research and tools for spatialization within the context of electroa- into composers’ attitudes in this domain. It was con- coustic music composition, decisions about position and ceived as the basis for the development of dedicated movement of sound in space, or the general question of notation and rendering tools within the SSMN project. virtual space quality remain often a secondary formal The taxonomy is a systematic representation of all rele- issue; in many cases these decisions are left to a post- vant features necessary to specify sound spatiality: shape production stage instead of being fully integrated and acoustic quality of the space, structure, position and throughout the composition process. This situation can movement of sound sources. It is based on single de- marginalize spatialization to an ornamental aspect that scriptors that can be combined in order to define complex can be adapted or reduced without affecting musical spatial configurations. Descriptors can be transformed substance. locally and globally and can be the object of structural On the other hand, performers engaged in the and behavioral operations. The SSMN Spatial Taxonomy interpretation of music involving electroacoustic proposes a corresponding graphic symbolic representa- spatialization (and other kinds of signal processing) find tion of descriptors, operations and other functional ele- mostly a reduced graphic representation of the ongoing ments facilitating the communication of creative ideas to processes in the score [1]. According to the experience of performers and technical assistants. the authors during several years of performance practice This paper focuses on the presentation of the taxonomy the notation of electroacoustic events prioritize mostly and the symbols. Additionally it describes the workflow cue numbers and synchronization events. This limits the proposed for using symbols inside a notation software possibility of a more intimate interaction within the prototype developed within the project. Finally, further performance situation. In addition, the lack of a aspects concerning the actual and future developments of spatialized acoustic feedback while studying prevents SSMN are mentioned. performers from preparing a piece taking into account sound motion. This issue becomes especially relevant when considering the usual restrictions of rehearsal time 1. INTRODUCTION in performance spaces. The need for a graphical representation of spatialization The field of sound representation has undergone continu- within the context of sound diffusion of electroacoustic al development throughout the history of creative arts. music in concert has been also addressed with arguments The issue of sound motion representation, as concerns pro [2] and contra [3]. Nevertheless a generic and composers, has however hardly been studied. Composers practical way to accurately notate spatialization has not have been continually obliged to reinvent strategies for been formulated yet. Even meticulous spatial notation as communicating their ideas of spatial movement of sound, in Stockhausen’s Oktophonie [4] using sequences of performers’ displacement, and description of the perfor- channel numbers instead of symbols –as in the mance space. In fact, even today’s musical software tools introductory notes to the score– is difficult to read for performers. Finally, when audio engineers collaborate with com- Copyright: © 2016 First author et al. This is an open-access article posers preparing compositions within a multi-channel dis- tributed under the terms of the Creative Commons Attribution environment, they have to overcome the difficulties of License 3.0 Unported, which permits unrestricted use, distribution, and interpreting placement of sound in space as imagined by reproduction in any medium, provided the original author and source are credited. the composers, who typically invent a personal system of focuses mainly on spectral structure [21] or develops a graphical or textual explanations. perceptual approach to the analysis of acoustic scenes The aim of SSNM is thus to open a new approach of [22]; Vandergorne’s spatial categories and figures are substantial integration of spatial relationships and spatial specifically concerned with sound diffusion [8]; UST processes in musical thinking as well as in composition, (Unités Sémiotiques Temporelles) [23] are obviously rehearsal and performance practice. For this purpose focused on temporal meaning. In our opinion, a taxono- SSMN has defined a typology of spatial movements and my for notation of spatialization should ideally be univer- designed a library of symbols to represent them. In order sal, generic and based on low-level structural features to enable its use in creative processes, an open source that can be represented through symbols. The terminolo- software tool that integrates this library within a common gy should emanate directly from musical practice and be western musical notation context is being developed, as much as possible self-explaining. The work presented allowing editing and acoustic feedback through a render- in the next section was developed under these premises. ing engine. Composers are thus able to use and edit sym- bols describing spatialization in a notation program and 3. TAXONOMY immediately hear the results. Performers are given full information on spatialization in the score and can hear the 3.1 Preliminary considerations results from the beginning of the studying process. The basic units of the SSMN spatial taxonomy are called 2. SCOPE OF RESEARCH descriptors. There are two kinds of descriptors: room descriptors and descriptors of sound sources. Sound During the preparatory stages of the SSMN Spatial Tax- sources can be physical root sound (RS) like instruments onomy, research has focused on the following: and voices or projected audio signals (PA) like micro- • Musical scores containing verbal or graphical annota- phone signals, audio files and streamed audio. tions of spatial indications, focusing on spatialization Descriptors can be simple or compound and are as- and extended notation in contemporary music since sumed to be perceptually relevant, but definitive percep- 1950 [5] tion depends on the interaction between the actual sound • Artistic performance practice wherein spatial placement and the actual spatial configuration. Although descriptors and/or mobility of live performers is relevant to musical are primarily defined in structural (geometrical, mathe- compositions as well as composers' means of express- matical, acoustical) terms, they have been conceived in ing placement and/or motion in space [6], [7] view of musical practice. • Spatialization in electroacoustic media [8] Simple descriptors are the basic atoms of the SSMN • Extended musical notation [9] spatial taxonomy. They are able to denote all single pri- • 2D and 3D symbolic notation [1] mary features relevant to sound spatiality and can be • Typologies of spatial qualities of sound [10] represented as symbols. Compound descriptors are arrays • Spatial perception of sound placement, sound motion of simple descriptors. They are used to represent more and physical space [11] complex spatial configurations and processes (e.g. pat- • Semiotics and epistemology of notation [12] terns, figures, motives, etc.) and can also be represented • Relevant programming languages, audio and graphic as symbols. design software including Csound, PD, Iannix, Super- Descriptors can have several properties that are finally Collider, Max/MSP, Illustrator, WFS [13] [14], Ambi- defined through names, numeric parameters and flags. sonics, IRCAM’s OpenMusic [15] & Antescofo [16], For instance, the descriptor "Position of loudspeakers" inScore [17] has the entry "labels" to name specific items, the parame- • Spatial notation in other fields, especially dance, aero- ters "position" given as Cartesian or spherical coordinates nautics, geographical migration patterns, and theater and "inclination" (yaw) defined as angle and the flag staging [18] "interior" or "exterior" according to their position inside In a nutshell, the specifications for the SSMN research or outside the room. For reasons of clarity, parameter units as well as some parameters and flags will be omit- project are based on a four-pronged study: (a) spatial ted in this paper. typo-morphology resulting in the SSMN Spatial Taxon- The third part of the