Digital Musical Instruments as Probes: How computation changes the mode-of-being of musical instruments KORAY TAHİ ROĞ LU1 , THOR MAGNUSSON2 , ADAM PARKINSON3 , IRIS GARRELFS4 and ATAU TANAKA4 1Aalto University School of Arts, Design and Architecture, Finland. Email: [email protected] 2University of Sussex, Brighton, UK. Email: [email protected] 3London South Bank University, UK. Email: [email protected] 4Goldsmiths University of London, UK. Emails: [email protected]; [email protected] This article explores how computation opens up possibilities for definition, we focus on those DMIs which have novel new musical practices to emerge through technology design. interfaces and mapping strategies. These DMIs Using the notion of the cultural probe as a lens, we consider the are increasingly decoupled from the established digital musical instrument as an experimental device that yields relationships we have with more traditional musical findings across the fields of music, sociology and acoustics. instruments. Such DMIs are no longer like traditional As part of an artistic-research methodology, the instrumental object as a probe is offered as a means for artists to answer musical instruments, such as the saxophone, sitar or questions that are often formulated outside semantic language. piano, for which many different compositions have This article considers how computation plays an important role been written and which have established themselves in the authors’ personal performance practices in different as cultural icons, references and constants. Rather ways, which reflect the changed mode-of-being of new musical than being a generic type, these DMI move towards instruments and our individual and collective relations becoming a different category of musical instrument; with them. often embodying certain theoretical approaches and affording specific practices. Today’s technological context makes it easier for composers and performers 1. INTRODUCTION alike to develop their own new instruments and systems, tailored to each new musical context, be Computation has been central to the development that a single composition, a band or an ensemble, or of new interfaces for musical expression, and this an interactive installation or software-based work. has been followed with new musical practices. During the last half-century, we have witnessed devel- In this, computation is an inherent part of the DMI, opments in the field of digital musical instrument much as electricity is part of rock instrumentation, (DMI) design and practice. However, much of the and mechanical buttons and reeds are fundamental knowledge and many of the methods that have to woodwind instruments. Computation therefore emerged in experimental music practices have been shapes our relationship with DMIs and also transforms fragmentary, often responding to individual and spe- our musical norms, habits, language and intentions; it is ’ cific artistic and musical problems. These fragments the DMI s unique mode-of-being in a new performance partially revolve around academic communities such practice. Furthermore, computation impacts widely as those assembled for NIME conferences since upon the nature of musical activities; how music is 2002, and institutions such as STEIM, established performed, experienced, shared and distributed. in the Netherlands in 1959. Whilst this largely hetero- In this article we seek to provide a new examination geneous assemblage of practices and ideas among of our relationships with such new musical instruments. connected communities has addressed different musical, Our primary questions are: How has computation social and technological elements of DMI culture, how changed the mode-of-being of musical instruments? computation has changed the mode-of-being of such Can we think of DMIs as having an embodied quality musical instruments more generally has been less widely given their digital materiality? What do today’s realised or discussed. instruments inspire us to do differently? Can we apply A DMI has been defined as a musical instrument the idea of a cultural probe (Gaver, Dunne and Pacenti where the digital sound generator is separate from 1999) to enquire further into their agency as far as the control interface, the two relatable via mappings musical composition and performance are concerned? (Malloch, Birnbaum, Sinyor and Wanderley 2006: 49). Through considering the computational nature of new Whist commercial digital keyboards can fit this musical instruments, we question what has changed in Organised Sound 25(1): 64–74 © The Author(s) 2020. This is an Open Access article, distributed under the terms of the Creative Commons Attribution licence (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted re-use, distribution, and reproduction in any medium, provided the original work is properly cited. doi:10.1017/S1355771819000475 Downloaded from https://www.cambridge.org/core. IP address: 170.106.202.58, on 27 Sep 2021 at 21:15:26, subject to the Cambridge Core terms of use, available at https://www.cambridge.org/core/terms. https://doi.org/10.1017/S1355771819000475 Digital Musical Instruments as Probes: How computation changes the mode-of-being of musical instruments 65 our relationships with musical instruments, composi- concept and apply it to DMIs. Gaver et al (2004:53) tions, performances and musical experiences. This note that the probes demonstrated that ‘research perspective gives us, the current authors as practising questions could be packaged as multiple, rich, and sonic artists, a lens through which to explain how engaging tasks that people could engage with by computation influences the methods and processes choice and over time’. What research questions, then, of our musical work(s). In the following sections, we can be asked through DMIs and the contexts in which explore these questions through a phenomenology they are used? of embodied musical performance with the conceptual When performing with a DMI, we likely depart to tool of ‘cultural probes’. This article is based on ref- some degree from traditional musical performance lections around our creative processes when working practice. This can begin a dialogue about what consti- with computational devices, and are the outcome of a tutes music and performance, and open up discussions workshop held at Goldsmiths, University of London, about what ideologies we want to embed in the instru- in early 2019. ment and inscribe in its computation. This might range from the extremes of being told (hopefully without too much cruelty or aggression) that what we are doing 2. PROBING: AN INTERCONNECTED ‘isn’t music’ or that our chosen tool ‘isn’t a musical ECOSYSTEM instrument’, through to more positive responses, for example that a performance or instrument is Bill Gaver, Tony Dunne and Elena Pacenti (Gaver ‘expressive’. Whereas Gaver et al.’s ‘probe returns’ et al. 1999) introduced cultural probes as a tool for Human Computer Interaction (HCI) research and constituted feedback in terms of photographs, anno- development in the late 1990s. Gaver and his team tated maps and postcards, the returns we receive had been tasked to design projects for various commu- from our probes come in the form of bewildered faces, nities, and order to initiate a dialogue with the users, dancing bodies, applause, heckles, camera phone the team created toolkits that enabled people to footage and perhaps the occasional piece of rotten document their lives in a creative and lively fashion. fruit. For the most part, we informally digest and As Gaver (1999: 22) describes: ‘The cultural probes— process this feedback, and respond to it in (future) these packages of maps, postcards, and other performances. materials—were designed to provoke inspirational In this way we can consider DMIs as cultural probes responses from elderly people in diverse communities.’ insofar as they are a means by which we, as musicians The ‘probe returns’ of postcards, maps and other arte- and researchers, seek to provoke and gather responses facts were used to start a dialogue so that the designers from audiences and listeners in a dialogue about what felt they were almost ‘designing for friends’ (Gaver, constitutes music and musical performance (Figure 1). Boucher, Pennington and Walker 2004: 54). The idea To design a DMI is to probe musical history and to of a cultural probe began to resonate throughout ask what musical ideas and ideologies we subscribe to. the HCI community, and over the next few years this In many instances, the end result of the design idea was adopted by various designers and researchers, process surrounding a DMI is best understood as becoming a valuable tool in co-design and related the design of a musical experience. In this way, the design practices. musical concert itself can be conceived of as a cultural We propose that cultural probes can also be used probe. It refers not just to a single performance with as a theoretical toolkit to explore how we relate with a DMI, but also to the larger context within which per- DMIs. This is not to say that DMIs are necessarily formers, promoters and audiences are conceptualising cultural probes per se, but rather that thinking through and experiencing musical performances. This includes DMIs and the greater context in which DMIs are used everything from the stage, lighting, the type of venue in terms of cultural probes can be fruitful in discussing or soundsystem, to whether people are seated or the changed mode-of-being of these musical instru- dancing. This can be conceived of as an ecosystem ments. It should be noted that cultural probes, as within which musical experiences involving the DMI used by Gaver et al., refer to specific packages of mate- are collectively imagined and instantiated. Concerts rials. For our purposes here, we would like to imagine are the means by which performers, curators and audi- a probe and its returns to have broader media and ences interrogate a new instrument and the performance materiality. Furthermore, the idea of cultural probes practice around it.