Digital Musical Instruments As Probes: How Computation Changes the Mode-Of-Being of Musical Instruments

Digital Musical Instruments as Probes: How computation changes the mode-of-being of musical instruments

KORAY TAHİ ROĞ LU1 , THOR MAGNUSSON2 , ADAM PARKINSON3 , IRIS GARRELFS4 and ATAU TANAKA4

1Aalto University School of Arts, Design and Architecture, Finland. Email: [email protected] 2University of Sussex, Brighton, UK. Email: [email protected] 3London South Bank University, UK. Email: [email protected] 4Goldsmiths University of London, UK. Emails: [email protected]; [email protected]

This article explores how computation opens up possibilities for definition, we focus on those DMIs which have novel new musical practices to emerge through technology design. interfaces and mapping strategies. These DMIs Using the notion of the cultural probe as a lens, we consider the are increasingly decoupled from the established digital musical instrument as an experimental device that yields relationships we have with more traditional musical findings across the fields of music, sociology and acoustics. instruments. Such DMIs are no longer like traditional As part of an artistic-research methodology, the instrumental object as a probe is offered as a means for artists to answer musical instruments, such as the saxophone, sitar or questions that are often formulated outside semantic language. piano, for which many different compositions have This article considers how computation plays an important role been written and which have established themselves in the authors’ personal performance practices in different as cultural icons, references and constants. Rather ways, which reflect the changed mode-of-being of new musical than being a generic type, these DMI move towards instruments and our individual and collective relations becoming a different category of musical instrument; with them. often embodying certain theoretical approaches and affording specific practices. Today’s technological context makes it easier for composers and performers 1. INTRODUCTION alike to develop their own new instruments and systems, tailored to each new musical context, be Computation has been central to the development that a single composition, a band or an ensemble, or of new interfaces for musical expression, and this an interactive installation or software-based work. has been followed with new musical practices. During the last half-century, we have witnessed devel- In this, computation is an inherent part of the DMI, opments in the field of digital musical instrument much as electricity is part of rock instrumentation, (DMI) design and practice. However, much of the and mechanical buttons and reeds are fundamental knowledge and many of the methods that have to woodwind instruments. Computation therefore emerged in experimental music practices have been shapes our relationship with DMIs and also transforms fragmentary, often responding to individual and spe- our musical norms, habits, language and intentions; it is ’ cific artistic and musical problems. These fragments the DMI s unique mode-of-being in a new performance partially revolve around academic communities such practice. Furthermore, computation impacts widely as those assembled for NIME conferences since upon the nature of musical activities; how music is 2002, and institutions such as STEIM, established performed, experienced, shared and distributed. in the Netherlands in 1959. Whilst this largely hetero- In this article we seek to provide a new examination geneous assemblage of practices and ideas among of our relationships with such new musical instruments. connected communities has addressed different musical, Our primary questions are: How has computation social and technological elements of DMI culture, how changed the mode-of-being of musical instruments? computation has changed the mode-of-being of such Can we think of DMIs as having an embodied quality musical instruments more generally has been less widely given their digital materiality? What do today’s realised or discussed. instruments inspire us to do differently? Can we apply A DMI has been defined as a musical instrument the idea of a cultural probe (Gaver, Dunne and Pacenti where the digital sound generator is separate from 1999) to enquire further into their agency as far as the control interface, the two relatable via mappings musical composition and performance are concerned? (Malloch, Birnbaum, Sinyor and Wanderley 2006: 49). Through considering the computational nature of new Whist commercial digital keyboards can fit this musical instruments, we question what has changed in

Organised Sound 25(1): 64–74 © The Author(s) 2020. This is an Open Access article, distributed under the terms of the Creative Commons Attribution licence (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted re-use, distribution, and reproduction in any medium, provided the original work is properly cited. doi:10.1017/S1355771819000475

Downloaded from https://www.cambridge.org/core. IP address: 170.106.202.58, on 27 Sep 2021 at 21:15:26, subject to the Cambridge Core terms of use, available at https://www.cambridge.org/core/terms. https://doi.org/10.1017/S1355771819000475 Digital Musical Instruments as Probes: How computation changes the mode-of-being of musical instruments 65

our relationships with musical instruments, composi- concept and apply it to DMIs. Gaver et al (2004:53) tions, performances and musical experiences. This note that the probes demonstrated that ‘research perspective gives us, the current authors as practising questions could be packaged as multiple, rich, and sonic artists, a lens through which to explain how engaging tasks that people could engage with by computation influences the methods and processes choice and over time’. What research questions, then, of our musical work(s). In the following sections, we can be asked through DMIs and the contexts in which explore these questions through a phenomenology they are used? of embodied musical performance with the conceptual When performing with a DMI, we likely depart to tool of ‘cultural probes’. This article is based on ref- some degree from traditional musical performance lections around our creative processes when working practice. This can begin a dialogue about what consti- with computational devices, and are the outcome of a tutes music and performance, and open up discussions workshop held at Goldsmiths, University of London, about what ideologies we want to embed in the instru- in early 2019. ment and inscribe in its computation. This might range from the extremes of being told (hopefully without too much cruelty or aggression) that what we are doing 2. PROBING: AN INTERCONNECTED ‘isn’t music’ or that our chosen tool ‘isn’t a musical ECOSYSTEM instrument’, through to more positive responses, for example that a performance or instrument is Bill Gaver, Tony Dunne and Elena Pacenti (Gaver ‘expressive’. Whereas Gaver et al.’s ‘probe returns’ et al. 1999) introduced cultural probes as a tool for Human Computer Interaction (HCI) research and constituted feedback in terms of photographs, anno- development in the late 1990s. Gaver and his team tated maps and postcards, the returns we receive had been tasked to design projects for various commu- from our probes come in the form of bewildered faces, nities, and order to initiate a dialogue with the users, dancing bodies, applause, heckles, camera phone the team created toolkits that enabled people to footage and perhaps the occasional piece of rotten document their lives in a creative and lively fashion. fruit. For the most part, we informally digest and As Gaver (1999: 22) describes: ‘The cultural probes— process this feedback, and respond to it in (future) these packages of maps, postcards, and other performances. materials—were designed to provoke inspirational In this way we can consider DMIs as cultural probes responses from elderly people in diverse communities.’ insofar as they are a means by which we, as musicians The ‘probe returns’ of postcards, maps and other arte- and researchers, seek to provoke and gather responses facts were used to start a dialogue so that the designers from audiences and listeners in a dialogue about what felt they were almost ‘designing for friends’ (Gaver, constitutes music and musical performance (Figure 1). Boucher, Pennington and Walker 2004: 54). The idea To design a DMI is to probe musical history and to of a cultural probe began to resonate throughout ask what musical ideas and ideologies we subscribe to. the HCI community, and over the next few years this In many instances, the end result of the design idea was adopted by various designers and researchers, process surrounding a DMI is best understood as becoming a valuable tool in co-design and related the design of a musical experience. In this way, the design practices. musical concert itself can be conceived of as a cultural We propose that cultural probes can also be used probe. It refers not just to a single performance with as a theoretical toolkit to explore how we relate with a DMI, but also to the larger context within which per- DMIs. This is not to say that DMIs are necessarily formers, promoters and audiences are conceptualising cultural probes per se, but rather that thinking through and experiencing musical performances. This includes DMIs and the greater context in which DMIs are used everything from the stage, lighting, the type of venue in terms of cultural probes can be fruitful in discussing or soundsystem, to whether people are seated or the changed mode-of-being of these musical instru- dancing. This can be conceived of as an ecosystem ments. It should be noted that cultural probes, as within which musical experiences involving the DMI used by Gaver et al., refer to specific packages of mate- are collectively imagined and instantiated. Concerts rials. For our purposes here, we would like to imagine are the means by which performers, curators and audi- a probe and its returns to have broader media and ences interrogate a new instrument and the performance materiality. Furthermore, the idea of cultural probes practice around it. Therefore, the way in which we has been an interesting ‘probe’ into the HCI commu- choose to frame the presentation and reception of the nity itself, stimulating debates around methodology, instrument can do as much ‘probing’ as the DMI itself uncertainty and the role of ‘the scientific process’ and can be a way of packaging a research question. in design, something addressed by Gaver himself When instruments are put in the hands of per- (Gaver et al. 2004; Hutchinson et al. 2003). We believe formers, a further probing occurs. Musicians often that it is in the spirit of cultural probes to adapt the use musical instruments in ways that the original

Downloaded from https://www.cambridge.org/core. IP address: 170.106.202.58, on 27 Sep 2021 at 21:15:26, subject to the Cambridge Core terms of use, available at https://www.cambridge.org/core/terms. https://doi.org/10.1017/S1355771819000475 66 Koray Tahiroğlu, Thor Magnusson, Adam Parkinson, Iris Garrelfs and Atau Tanaka

Figure 1. In 4 Hands Iphone performance series, Atau Tanaka and Adam Parkinson re-appropriate the iPhone and its advanced technical capabilities to transform the consumer and iconic object into an expressive DMI.

designers never intended, probing for hidden affordan- come into play. Through the instrument as a probe ces. An oft-cited example is the Roland TB 303, a given to the world, to other users, the designers of the bassline synthesiser that was designed to provide back- DMI are able to explore the character of the instrument ing tracks for bass players. It turned out to be awkward in important aspects that are not available when using to programme and essentially ‘failed’ at what the design- the instrument themselves, as they are too familiar with ers wanted it to do. However, a few years after its its functions. It is only when the instrument is released, release, DJ Pierre was experimenting with one of these or set free, that we begin to see its alternative potentials. devices and accidentally stumbled upon the ‘squelchy’ One criticism of DMIs is that, in many instances, sound that came to characterise, and effectively they are always being updated and evolving, prevent- invented, the Acid House genre which spread across ing the performer from acquiring the sort of virtuosity North America and the UK (Reynolds 2013). Taking we associate with instruments of the past. However, another example, saxophonists such as Evan Parker if we see DMIs as probes, it makes sense that they and John Butcher reveal ‘hidden affordances’ (Gaver are always in development, and that they are part of 1991) that were never deliberately designed into the an ongoing design process, forming one element of instrument, using extended techniques to produce multi- a multi-sided conversation. The relationship between phonics and ‘unintended’ sounds from the instruments audience feedback and instrument design is not such that have come to define the style of these musicians. that the DMI performer necessarily responds directly There are many similar cases, all of which undermine, to every criticism uttered, much in the way that Gaver or at least problematise, the notion that even if an instru- et al. (1999: 29) note ‘Although the probes were central ment can be designed according to a specified set of to our understanding of the sites, they didn’t directly criteria, it can never be fully limited to those definitions. lead to our designs. They were invaluable in making DMIs as cultural probes have specific functions in us aware of the detailed texture of the sites, allowing this context; by giving an undefined musical technology, us to shape proposals to fit them.’ Likewise, this probe albeit from a specific cultural context, to an instrumen- becomes part of the DMI’s performative ecosystem to talist the designer is able to observe the musician’s which we respond. response and how ideas emerge through the use of The design of DMIs is often rooted in rich, multimodal the instrument. This has been explored by instrument and conversational design processes that resonate with makers, such as Ulfarsson (2019), whose halldorophone the cultural probe’s methodology, as opposed to a more instrument has been adopted by various performers ‘scientific’ process. Attempts to assess the ‘success’ around the world (Hildur Guðnad´ottir plays the halldor- or ‘failure’ of a new DMI can prove to be difficult. ophone on the Sunn O))) album Life Metal). The results Knowing that a DMI enables a performer to reach a dis- have been interesting: the instrument shapes the musi- tant octave in a short time period may not be as valuable cian as much as the musician shapes the music, but as knowing that a DMI made people dance or enjoy a this is not a simple picture and innumerable parameters concert, but the latter is likely to be harder to measure

and quantify. Having considered the DMI as a probe in a the Hands (1993) demonstrates. For Sudnow, learning larger ecosystem, next we consider how phenomenology the piano late in his life became an investigation into has looked at the ways we probe our world through a phenomenological condition between human and technology. instrument, and writing the book became a reporting mechanism, akin to the postcards and artefacts with Gaver’s subjects with which the journey and its 3. PHENOMENOLOGY IN DMI insights are captured. By bringing together insights PERFORMANCE from De Souza (2017) on the embodied cognition view of musical instrument practice with Dourish’s Phenomenology is a wide-ranging philosophical (2004) phenomenological understanding of human– movement that studies human experience. It looks computer interaction, we can ask how does the at how we make sense of the world and takes an computational nature of DMIs reframe the phenome- approach that rejects Cartesian dualism. At the begin- nological probe? What does it mean to ‘think’ with a ning of phenomenological thought, Husserl proposes probe, to utilise an instrument that might change ‘ ascientific, systematic enquiry into our conscious men- or evolve over time? Are there modes of feedback – tal processes without regard to their non-mental causes visual, haptic or sonorous – that can strengthen the ’ and consequences (Inwood 1999: 2). Heidegger breaks performer’s focus such that new insights emerge? ’ with Husserl s neo-Kantian view and considers the (Figure 2). philosophical question of being, and extends the word However, a phenomenology of digital musical ‘ ’ Dasein to think about ways of being-in-the-world . instrument performance cannot reside solely in the He coins the terms vorhanden (present-at-hand) and performer relationship with the instrument. If the per- zuhanden (ready-to-hand) to distinguish whether an formance takes place in a public setting, the experience object is the focus of attention or is transparent to the is not complete until it is seen and heard by an audi- action that is carried out through it (Inwood 1999). ence. In the The Meaning of the Body, Mark Johnson These concepts have been taken up by HCI researchers, describes forms of intersubjectivity that are useful for including Dourish (2004). us to consider how a performer’s experience is commu- Different aspects of phenomenology have been nicated (Johnson 2008). Using phenomenology to applied to studies of sound and music, and of musical probe a DMI performance, we can study the experi- instruments. Cox (2001) takes an embodied cognition ences of performers and audiences. They can then approach to studying musical meaning and proposes a be combined to form a holistic understanding of per- mimetic hypothesis where a listener naturally internally formance and reception of a concert taking place imitates a performance they experience. De Souza on digital musical instrument technology. The non- (2017), in his Music at Hand combines music theory dualist orientation of post-Husserlian phenomenology and phenomenology to study ways in which traditional helps us to consider visceral and mental musical musical instruments become creative prostheses, technol- experience together as an intertwined whole. Studying ogies that condition and involve the body in otherwise performer and audience together may allow us to seemingly cerebral tasks such as notated composition. understand the experiential space that is afforded by Instrumentalists are familiar with the experience of sens- interactive instrument design. Beyond the instrument, ing the unique character of an instrument, as if each one it may help us to understand the ways in which the is a probe into unknown musical possibilities. performance experience is transfused through different Ihde (2007) carries out a phenomenological study bodies – the performer’s body, the audience’sbodies of sound and uses the philosophical frameworks put and the social body through which a moment is shared. forth by Husserl, Heidegger and Merleau-Ponty to unwrap the act of listening. By emphasising phenomenology not as philosophy, but as an activity, 4. DIFFERENT BODIES we might liken Idhe’s gesture to Christopher Small’s transformation of music into the act of musicking In thinking about phenomenological probes and (Small 1998). If doing phenomenology becomes a bodies, how do we as active participants in the ecology way to study our experiences of the world, we can of sonic-making relate to this, especially in a perfor- deploy DMIs as scientific instruments or probes in mance setting? How do we feature as bodies in this which we further seek to understand our human becoming-practice? What does this ecosystem of body condition through active musical perception (Noë and computation in digital instruments have to say 2004). Phenomenology serves here as the theoretical about the music that is being produced? foundation of the instrumental probes we have in our From a phenomenological perspective, Ihde speaks methodological toolbox. about the significance of ‘listening to the voiced Instruments can function as phenomenological character of the sounds of the World’ (Ihde 2007: probes, as the example of David Sudnow’s Ways of 147), equally inner imagined sounds and auditory

Downloaded from https://www.cambridge.org/core. IP address: 170.106.202.58, on 27 Sep 2021 at 21:15:26, subject to the Cambridge Core terms of use, available at https://www.cambridge.org/core/terms. https://doi.org/10.1017/S1355771819000475 68 Koray Tahiroğlu, Thor Magnusson, Adam Parkinson, Iris Garrelfs and Atau Tanaka

Figure 2. Atau Tanaka performing the piece Myogram that explores the digital music performance with sensors, networks, mobility and performer body.

phenomena experienced in hearing. In performance In Western art music, composition and performance this relates to the sounds we create, and how we imag- have become seen as separate activities: according to ine our audience to experience them. This probing-as- this model a composer encodes her ideas in musical listening connects the performer’s body with the notation, and the performer then executes this score instrumental body, the body of the audience, the archi- in sound. Later, some of these separate activities began tectural body and, crucially, the sonic body that to overlap as sounds could be ‘sculpted’ according to emerges from their interplay. The instrumental body the composers wishes (Ihde 2007: 262); for example, in this configuration is an interesting one with far- via editing and manipulating tapes (and later desktop reaching computational implications on the sound computers), guided by the probing ears. Further down that is being created, and how performers and audien- the digital timeline, and in regard to the augmented ces might relate to them by active listening. In the voice pioneer Pamela Z, the ‘body becomes redefined twentieth century, the encounter between new technol- as both a source of data and as a bidirectional, perme- ogies and new philosophies gave us new musics and able membrane of transference’ (Lewis 2007: 77). new listening perspectives, such as musique concrète In short, any envisaged ‘instrumentation’ becomes and reduced listening. Ihde surmised that ‘perceptions part of the composition process, imagination activated are concretely situated within an emerging metaphor, by the body and vice versa. This ‘instrumentation’ also a newly oriented system’ (Ihde 2007: 233). If we hear activates us and contributes to the music that unfolds. differently it is because we exist in the world differ- And so, while we make music with/through an instru- ently. At the same time, within both the creation of ment, each instrument we engage with also leads us to and the listening to the sonic body, past memories create certain sounds, or at least favour some (Figure 3). emerge and are encoded continually, both culturally This applies to both haptic properties and computa- (Castells 1996) and personally speaking (Demers tional parameters, and their joint ‘workflow’ (Brown, 2010); the same applies to the design of DMIs. Eldridge and McCormack 2009).Atthecoreofthe Regardless of how the past influences auditory percep- composition process is a system which ties together tion, it always takes place in the present. Even if the performer, instrument, audience, performance space technology shapes our listening and making of music, and sound. However, this system is never completely these are clearly deeply embodied and intertwined pro- fixed, and to boot many of these demarcation lines have cesses. For DMIs to work fluidly there needs to be an become blurred. It requires listening as a probe to retain immediacy of interaction between body and interface. balance in flux. A short response time between imagining sound, execut- Thinking back to Gaver’s notion of probes, we ing sound and listening to the sound made, enables the can extend this to the probing of spaces. Considered performer’s body to feel intimately connected with the in conjunction with Castells’s observation that spaces sonic output of the instrument played. express society (Castells 1996), perhaps we also need

Figure 3. An example for improvised augmented voice performance by Iris Garrelfs. Performance took place at the Barbican during the 2013 HackTheBarbican event, curated by Music Hackspace.

to re-think the spaces that we want to inhabit as musi- mode-of-being of musical instruments and change our cians and sounding artists, or how we inhabit them. relationships with them. In order to do this, we will look For instance, by creating mobile performances or at human–technology relations through Don Ihde’s four using wearable speaker systems that allow sounds to phenomenological modes of technological mediation: emanate from the body that produces it, by which embodiment relations, hermeneutic relations, alterity sound is re-combined with body, re-focusing schizo- relations and background relations (Ihde 1990). phonic experience. The complexity of contemporary In the first mode, embodiment relations, technology music technology, and how it influences composition, remains perceptually transparent representing a kind also includes aspects of co-creation where generative of isomorphism between our bodily actions and processes contribute to the music and offer the performer perceptions (Ihde 1990: 82). We can think of this as an opportunity for response. Probing-as-listening then being consistent with Heidegger’s zuhanden. In this also connects the performer’s body with this computa- perspective the user of the technology is fully engaged tional body, accessed through sound. with interactions with technological artefacts where the artefact itself slips into the background, disappear- ing from conscious thought. In this mode, technology 5. MULTIDIMENSIONAL RELATIONS becomes part of life, not distinct from it. It becomes an WITH DMIS extension of our body and we use it focusing on the The above contexts represent the diverse interrelations content and not the technology. When music becomes of bodies and the materiality of sound, offering us mediated through an instrument in an embodiment compelling possibilities for rediscovering our relation- relation, we do not focus on the instrument as a ships with DMIs. These relations are mediated distinct object but rather on the music itself. by digital technologies and are centred around the Another aspect of embodiment relations is that tech- different dimensions in which the changed nature of nology brings into perception that which was not embodiment may become manifest (Ihde 2010: 41). previously apparent. Examples given by Ihde are the The word ‘dimensions’ is used here to mean the range magnifying glass or the telescope, because these scien- of musical activities we undertake with surrounding tific instruments mediate perception, illuminating that technologies and our interactions with them. They which was invisible to the human eye in earlier times are the ways in which our embodied relationships (Ihde 1990: 49). By mediating perception, technology with DMIs may vary and take on diversity. The shapes and transforms our relationship with the world features of DMIs have traditionally been developed around us. Ihde points out that once the sight of the by novel technological configurations, at times embrac- mountains of the moon became visible, visual technol- ing design constraints. Here, we explore how different ogy transformed the moon’s earlier existing spiritual aspects of human–technology interaction alter the context in human history into a visual perception that

Downloaded from https://www.cambridge.org/core. IP address: 170.106.202.58, on 27 Sep 2021 at 21:15:26, subject to the Cambridge Core terms of use, available at https://www.cambridge.org/core/terms. https://doi.org/10.1017/S1355771819000475 70 Koray Tahiroğlu, Thor Magnusson, Adam Parkinson, Iris Garrelfs and Atau Tanaka

Figure 4. Koray Tahiroğlu performing the piece No More Together written for the PESI system and three-musicians. PESI provides co-located, embodied and spatial opportunities for musical exploration with on-body mobile instruments and in-space surround speakers with motion-tracking modules.

has changed our relationship with it. The moon is not upon the ‘engaged phenomena’ that we encounter in part of the heavens any more, but it is an astronomical the form of technology. Any technology could chal- object, a body that orbits the earth. lenge us in a similar way to how an artwork might Digital technologies result in deep transformations change our view of the world – when we look at a great in embodied musical practices, enhancing the differ- painting the world opens up as we act on it (Noë 2012:2). ences in comparison to acoustic musical instruments. There are times in embodiment relations where the Computation brings new methods of generating music technology becomes too easy, too transparent, and with our bodies and new methods of producing sound. even prescribes actions through the abilities it bestows The quantity of differences in terms of the speed or upon us. Designers can remove the possibility of accuracy of the control that we can have evolves challenging users to critically respond to a technology. into qualitative differences in being able to specify at By making everything ready at hand (vorhanden), various levels how sound develops in a composition ready to use and easy to facilitate, the technological (Rowe 2008). In this way, computation reveals new world empowers itself. Contrary to such a relation- possibilities for sound. This results in digital tech- ship, computational technologies can give us great nologies transforming the musical behaviour of new technical advancements when designing musical musical instruments. Digital technologies magnify instruments; as the user can redesign it through pro- our listening behaviour, and result in the acquisition gramming or hardware hacking. Such a relationship of new skills in understanding the music, ultimately might empower the user in their use (and misuse) allowing us to experience new aesthetics in music. of technology, allowing us to discuss the changed DMIs become the probes that bring in a new mode-of-being of a musical instrument through other embodied player–instrument relation with computa- relations we have with technology. This is what Ihde tionally enhanced variations. Figure 4 illustrates the calls hermeneutic relations (Ihde 1990: 80). performance setup of the Notion of Participative and In hermeneutic relations, users of a technology rely Enacting Sonic Interaction – PESI interactive music on their interpretive capabilities to ‘read’ some aspect system (Tahiroğlu, Correia and Espada 2013). This is of the world through that technology. In this type of a an example of a DMI that enables embodied musical relation, the technology itself ‘becomes the object of exploration in co-located collaborative performance, perception while simultaneously referring beyond itself encouraging reflections about space and movements to what is not immediately seen’ (Ihde 1990: 82). (Goddard and Tahiroğlu 2013; Parkinson and The computational nature of the DMI brings in Tahiroğlu 2013). such hermeneutic relations, another type of digital The strength of our embodied interaction with transformation, that takes our interpretive sense the world around us gives us the capacity to reflect of embodiment (perception) to refer to the possible ways

Figure 5. Thor Magnusson live coding at a festival in Bristol’s Arnolfini. The code is written in real time and presented on the screen, which opens ups for further interpretation by the audience.

in which we interpret sound and music. Such interpreta- behaviours are applied to music generation; for exam- tion appears as a common method of practice in ple, with machine learning and artificial intelligence building and rebuilding DMIs through which we reflect tools (Briot and Pachet 2017; Tahiroğlu, Svedström and think about computational tools (Magnusson and Wikström 2015). These advanced computational 2009). In this hermeneutic relation with computational technologies have been applied to some DMIs, tools, DMIs present themselves as probes through forming technologies that are alternative to human the ways musical sound is transformed into musical musicians and to commonly practised musical instru- object. Ihde (1990: 96) calls this transformation ‘instruments. The computational features of automatic mentation’, comprising our actions through technology, response behaviours potentially turn a DMI into something that also exists in our relationship with an entity, generating relevant musical actions to be acoustic instruments and is therefore common to performed, monitoring the music space, performing all musical instruments (Vasquez, Tahiroğlu, and musical events that would challenge the human musi- Kildal 2017). cal discourse (Tahiroğlu et al. 2015). These advanced The computational features of DMIs represent automation features on musical responses bring hermeneutic qualities that are new to the domain of independent decision-making technologies into our musical instruments. Figure 5 shows the ‘live-coding’ relationship with musical instruments. One could performance in practice, in which coding becomes an argue over whether the otherness of the DMI is less embodied hermeneutic technic, resulting in interpreta- strong than the otherness found in an alternative tions of music not only perceived by the performer but human musician. The new relationship with DMIs also by the audience. Taking Ihde’s ideas further, we provides such alterity relations with an equal sense could argue that certain sound-generation features in of interacting with something other than me as dis- computer-generated music, such as random processes cussed by Ihde, not in the sense of a competitor but applied to audio synthesis algorithms, or generative more in line with a dialogue in the co-creation sense. algorithms in musical composition, further unfold into The otherness for alterity relations with DMIs probes sets of relations that turn the DMI into a probe and the possibilities of alternatives to the relationship we cause it to become the other, constituting a phenome- have with musical instruments that are designed as nological mode Ihde terms as alterity relations (Ihde tools to serve musical objectives. The DMI might 1990: 97). come to be regarded as having its own objectives The alternative and otherness relationship (Ihde and self-purpose within itself: ‘having a life of its own’. 1990: 98) can take place where there exists a unique These relationships we have with technology affect type of otherness in which autonomous and intelligent to different degrees the ways in which music made with

Downloaded from https://www.cambridge.org/core. IP address: 170.106.202.58, on 27 Sep 2021 at 21:15:26, subject to the Cambridge Core terms of use, available at https://www.cambridge.org/core/terms. https://doi.org/10.1017/S1355771819000475 72 Koray Tahiroğlu, Thor Magnusson, Adam Parkinson, Iris Garrelfs and Atau Tanaka

DMIs is experienced. Some phenomenological modes following a technological normatisation in composition, may bring certain effects to the fore more directly than performance and mixing-mastering techniques (Percino, other relations. Ihde introduces the term technological Klimek and Thurner 2014). texturing in background relations to describe the ways Mark Fisher recently asked, how might it be, with that technological interference is linked to our lived the powerful music technologies we have available, experience (Ihde 1990: 109). Computational technolo- that so much of today’s music has become so homog- gies texture music performance environments in such enous and similar to the music of the 1990s (Fisher a way that an ‘absent presence’ (Ihde 1990:112)oftech- 2014). We can here consider digital music technologies nology in background relations may transform music as consisting of two strands: the first one is a simula- perception. For example, surround sound systems tion of historical technologies (notation, piano rolls, afford a particular type of musical performance and mixers, tape, outboard devices) and aimed at music perception that opens new dimensions of human– production. In the design of these tools, we have technology relations. Concert spaces in their ‘absent abstracted technologies and practices and implemented presence’ role where their acoustics have a prominent them in the digital domain as software. Through this effect upon the way music is experienced but it is abstraction process, processes and musical objects are rarely the object of focus. The environment in which amplified, transformed, reduced, rejected, or simply the music is presented reflects our background relations forgotten. Many of the techniques that were possible with music – something that can be studied by the use of with a multitrack tape machine cannot be done with probes in real-time experiments. a DAW, and some of those were never intentionally design, or indeed part of the device’s manual. In the world of acoustic and electronic instruments, the acci- 6. KNOWLEDGE IN DMIS dental was often embraced and there was a strong The previous sections explored our relationships with quality of discoverability in these systems, a quality that our instruments, and how we form bonds and interpret is reduced in digital systems. them as extensions of our physical body, as an aug- The other strand of digital musical design could mented body-image. We can also explore the qualities be conceived of as one of performance (as opposed that bestow instruments with this potential. What is it to production). Here digital materials (MIDI cont- that gives instruments their character, depth and person- rollers, DIY electronics and coding) are applied as ality? Asking such a question will quickly take us to sources for unique assemblages of musical purpose. a realisation that the materialities of our twenty-first- Installations, software, apps, instruments, tools and century instruments are heterogeneous and resisting other systems are made for specific ends in mind, often ’ clear organological categorisations (Magnusson 2017). uniquely shaped to the creator s interests to the degree We find that acoustic instruments embody certain that the digital system becomes a piece, a composition qualities of reliable behaviour and continuity, offering on its own. Here, what previously could be character- standards in performance and composition. We also ised as composing work has now transformed into discover the incredible flourishing of experimentation inventing a system (Magnusson 2019). With the pleth- in instrument design and musical media brought with ora of diverse ingredients available (sensors, motors, electricity, electronic and computational technologies. controllers, code libraries, etc.) an approach to Here, our instruments bring new modes of thinking, composition with these materials as compositional performing and understanding sound and music. elements becomes more diverse and less streamlined Digital technologies are a revolution in this sense, as in alignment with the tools of the commercial software algorithmic control makes design flexible, fluid and houses. redefinable through code. Hardware is here written, like Considering how these two strands of design functional poetry. approaches have influenced today’s music, we see Conceiving of the development of instruments how the production software strand has homogenised in these three distinct, yet overlapping, material music (Fisher 2014), but in the performance real-time substrata of the acoustic, electric and digital, we also strand a new world of musical creativity is emerging. find that the instruments incorporate and embody Fisher’s quest for new music and new musical practices certain musical qualities. Tunings, scales, time signatures, is less likely to be rewarded through music streamed tempi and other musical elements are increasingly written from musical subscription providers, but rather to be into our tools. The 16-step paradigms of electronic experienced in live settings across the world in new sequencers and drum-machines, or the 4/4 time signature music events, festivals, hacklabs and art contexts. and 120 beats-per-minute setups of most DAWs, implic- New music embraces the performative, liveness, materi- itly direct the way we conceive of music-making in ways ality and system design and these new musical practices that acoustic instruments never did. The result is that do not travel as well as an MP3 bitstream on the music has become more homogeneous and standardised, internet.

What, then, are these new qualities of digital mate- been transformed with computational technologies rials? How, and to what degree, can we define them as and networked computers, and we must consider containing an epistemic dimension: one that functions our instrument design as real-time experiments that differently from the phenomenological perspective sometimes succeed and sometimes fail in shaping the than acoustic instruments? Any designer of a new nature of future music. The text has described how DMI will be familiar with how the process of design- we, as active practitioners in the sonic arts, reflect ing the instrument inevitably involves musical upon the new computational materiality of the tech- decisions. These might involve deciding upon a tuning nologies we work with and how they pose a new system, a scale, a metric system and other musical mode of being in the performing arts. The figures in parameters that will affect the music performed. The the article demonstrate the diverse practices of the five violin and the Theremin do not have a scale or a fixed authors, which supports the individual sections. tuning, but in a digital system any continuous dimen- We have applied the notion of cultural probes in sion could easily be reticulated into discrete steps. the context of instrument design, as we find that the Furthermore, those discrete steps (e.g., a scale) could design process is a cultural experiment where we give be changed by the press of a button, thus completely something out into the world, whether that be an transforming the quality of the instrument. This can, instrument, installation, code library or the music of course, also be done with acoustic instruments. itself, and in that process we observe how the work We have movable frets on a sitar and we can use alter- is received and gain invaluable feedback or return. native tunings on the guitar, but these are relatively The instrument as a cultural probe is equally a test major operations that are rarely done, at least in the into the potential of new musical expression, body– case of the guitar. It is hard to change the tuning of instrument relations, sound in space technology, a flute, clarinet or oboe. The digital brings a certain performer–audience relationship and many more fluidity and sculptability to our instruments: through aspects of contemporary musicking in a network of algorithmic definitions we can transform the function reciprocal relationships. These methods can be formal, of the material object with a press of a button. as often happens in NIME development at academic The openness and lack of definition of digital mate- levels, or they can equally be implicit in the work pro- rials (a sensor can equally be used in military, sport, cesses only rarely emerging explicitly, yet informally, games and music equipment) means that anything is through discussion in clubs and hacklabs. These inten- possible and that, at least this point in time, most sive methods of personalising technology through approaches are characteristically unique to the author iterative design approach can be beneficial to other or the designer. This blank technological space offers fields of product design, as music presents a unique a method of probing musical possibilities through new and highly intensive performance form (in terms of instruments making new music in new contexts. There numbers of parameters, bodily training, timeliness, is a lot of inventiveness in the field of new musical embodiment, etc.). We have long-term relationships interfaces and it is not limited to instrument design. with musical instruments. To fully appreciate the role Rather, what is happening is that musical composition of these designed objects in performance may require is moving from operating with symbolic writing of durational research studies. By studying DMIs in this dots on staff lines or signal writing on phonographic way, we can think of them as computational sonic media to new digital systems that merge the externali- entities, as the objects that are fluid, never resting sation of thought in material, symbolic and signal and continually opening themselves up for new defini- technologies. With the democratisation of hardware tions and usage. and software design through open source and maker culture, we also open up new performance contexts and cultural spaces for new music to emerge in. Inevitably this design is of such high level of function Acknowledgements that it involves new definition of musical spaces, where This work is supported by the Academy of Finland technology sets the constraints of the possible, but (project 319946) and the AHRC (UK Arts and at a heightened musical dimension compared with Humanities Research Council - AH/R002657/1). the constraints we are accustomed to with acoustic instruments. REFERENCES 7. CONCLUSION Briot, J. P. and Pachet, F. 2017. Music Generation by Deep Learning-Challenges and Directions. arXiv preprint In this article we have pointed to how DMIs can be arXiv:1712.04371. used as probes into the nature of new music and Brown, O., Eldridge, A. and McCormack, J. 2009. Under- related practices. The context of music-making has standing Interaction in Contemporary Digital Music:

Downloaded from https://www.cambridge.org/core. IP address: 170.106.202.58, on 27 Sep 2021 at 21:15:26, subject to the Cambridge Core terms of use, available at https://www.cambridge.org/core/terms. https://doi.org/10.1017/S1355771819000475 74 Koray Tahiroğlu, Thor Magnusson, Adam Parkinson, Iris Garrelfs and Atau Tanaka

From Instruments to Behavioural Objects. Organised Magnusson, T. 2017. Musical Organics: A Heterarchical Sound 14(2): 188–96. Approach to Digital Organology, Journal of New Music Castells, M. 1996. The Rise of the Network Society. Oxford: Research 46(3): 286–303. Blackwell. Magnusson, T. 2019. Sonic Writing: Technologies of Cox, A. 2001. The Mimetic Hypothesis and Embodied Material, Symbolic, and Signal Inscriptions. New York: Musical Meaning. Musicae scientiae 5(2): 195–212. Bloomsbury Academic. Demers, J. 2010. Listening through the Noise: The Aesthetics Malloch, J., Birnbaum, D., Sinyor, E. and Wanderley, of Experimental Electronic Music. Oxford: Oxford M. M. 2006. Towards a New Conceptual Framework University Press. for Digital Musical Instruments. Proceedings of the De Souza, J. 2017. Music at Hand: Instruments, Bodies, and 9th International Conference on Digital Audio Effects. Cognition. Oxford: Oxford University Press. Montreal, Canada, 49–52. Dourish, P. 2004. Where the Action Is: The Foundations of Noë, A. 2004. Action in Perception. Cambridge, MA: Embodied Interaction. Cambridge, MA: MIT Press. MIT Press. Fisher, M. 2014. Ghosts of my LIFE: Writings on Depression, Noë, A. 2012. Varieties of Presence. Cambridge, MA: Hauntology and Lost Futures. Winchester and Washington, Harvard University Press. DC: Zero Books. Parkinson, A. and Tahiroğlu, K. 2013. Composing Social Gaver, B., Dunne, T. and Pacenti, E. 1999. Design: Cultural Interactions for an Interactive-Spatial Performance Probes. Interactions 6(1): 21–9. System. Proceedings of the 2013 International Sound and Gaver, W. W. 1991. Technology Affordances. Proceedings of Music Computing Conference. Stockholm: KTH Royal the SIGCHI Conference on Human Factors in Computing Institute of Technology, 347–52. Systems: New Orleans, Louisiana: ACM, 79–84. Percino, G., Klimek, P. and Thurner, S. 2014. Gaver, W., Boucher, A., Pennington, S. and Walker, B. Instrumentational Complexity of Music Genres and 2004. Cultural Probes and the Value of Uncertainty. Why Simplicity Sells. PloS one 9(12): e115255. Interactions-Funology 11(5): 53–6. Reynolds, S. 2013. Energy Flash: A Journey through Rave Goddard, C. and Tahiroğlu, K. 2013. Situating the Music and Dance Culture. London: Faber & Faber. Performer and the Instrument in a Rich Social Context Rowe, R. 2008. Personal Communication. Doctoral with PESI Extended System. Proceedings of the Sound defence, 12 December. and Music Computing Conference. Stockholm, Sweden, Small, C. 1998. Musicking: The Meanings of Performing and 368–75. Listening. Middletown, CT: Wesleyan University Press. Hutchinson, H., Mackay, W., Westerlund, B., Bederson, B. B., Sudnow, D. 1993. Ways of the Hand: The Organization of Druin, A., Plaisant, C., et al. 2003. Technology Probes: Improvised Conduct. Cambridge, MA: MIT Press. Inspiring Design for and with Families. Proceedings of Tahiroğlu, K., Correia, N. N. and Espada, M. 2013. PESI the SIGCHI Conference on Human Factors in Computing Extended System: In Space, On Body, with 3 Musicians. Systems. Ft. Lauderdale, Florida: ACM, 17–24. Proceedings of the 2013 International New Interfaces Ihde, D. 1990. Technology and the Lifeworld: From garden to for Musical Expression Conference. Daejeon, Republic of Earth (no. 560). Indianapolis: Indiana University Press. Korea, 35–40. Ihde, D. 2007. Listening and Voice: Phenomenologies Tahiroğlu, K., Svedström, T. and Wikström, V. 2015. Noisa: of Sound, 2nd edn. New York: State University of New A Novel Intelligent System Facilitating Smart Interaction. York Press. Proceedings of the 33rd Annual ACM Conference Extended Ihde, D. 2010. Embodied Technics. Copenhagen: Automatic Abstracts on Human Factors in Computing Systems.Seoul, Press/VIP. Republic of Korea, 279–82. Inwood, M. (ed.) 1999. A Heidegger Dictionary. Oxford: Ulfarsson, H. 2019. Feedback Mayhem. Compositional Wiley-Blackwell. Affordances of the Halldorophone Discussed by its Users. Johnson, M. 2008. The Meaning of the Body: Aesthetics of Proceedings of the 2019 International Computer Music Human Understanding. Chicago: University of Chicago Press. Conference. New York: ICMC/NYCEMF. Lewis, G. 2007. The Virtual Discourses of Pamela Z. Journal Vasquez, J., Tahiroğlu,K.andKildal,J.2017.Idiomatic of the Society for American Music 1(1): 57–77. Composition Practices for New Musical Instruments: Magnusson, T. 2009. Of Epistemic Tools: Musical Instru- Context, Background and Current Applications. Pro- ments as Cognitive Extensions. Organised Sound 14(2): ceedings of the International Conference on New Interfaces 168–76. for Musical Expression. Copenhagen, Denmark, 174–9.