Digital Musical Instruments as Research Products Robert H. Jack Jacob Harrison Andrew P. McPherson Centre for Digital Music Centre for Digital Music Centre for Digital Music Queen Mary University of Queen Mary University of Queen Mary University of London, E1 4NS London, E1 4NS London, E1 4NS [email protected] [email protected] [email protected] ABSTRACT musical context and their influence on a performance ecosys- In the field of human computer interaction (HCI) the limita- tem [22]. This division is key to what we outline in this tions of prototypes as the primary artefact used in research paper as research prototype and research product. have been noted. Prototypes often remain open in their de- We argue that a significant number of the interfaces, in- sign, are partially-finished, and have a focus on a specific struments and artefacts created as part of the research con- aspect of interaction. Previous authors have proposed ‘re- ducted at NIME are imagined as research prototypes: their search products’ as a specific category of artefact distinct deployment and evaluation encourages people to reflect on from both research prototypes and commercial products. the qualities of the instrument and to suggest limitations or The characteristics of research products are their holistic possible improvements to the instrument. Prototypes direct completeness as a design artefact, their situatedness in a people to think about the artefact as a demonstration of a specific cultural context, and the fact that they are evalu- principle that points towards a future revision [13]. ated for what they are, not what they will become. This Odom et al. define ‘research products’ [19] as an extension paper discusses the ways in which many instruments created of the prototype which allow us to better focus on the com- within the context of New Interfaces for Musical Expression plex factors that are brought together in human-computer (NIME), including those that are used in performances, of- relationships. In short, research products are a type of arte- ten fall into the category of prototype. We shall discuss why fact in HCI and design research, which match the level of research products might be a useful framing for NIME re- fidelity often seen in commercially available and everyday search. Research products shall be weighed up against some objects, as opposed to the explicitly ‘unfinished’ nature of of the main themes of NIME research: technological inno- a prototype. This property allows research products to be vation; musical expression; instrumentality. We conclude taken ‘for what they are, rather than what they might be- this paper with a case study of Strummi, a digital musical come’, and for researchers to investigate ‘complex matters instrument which we frame as research product. of human-technology relations that often involve messy, in- timate and contested aspects of every day life’. In this paper, we introduce the concept of research prod- Author Keywords uct, and evaluate how it can be applied to NIME research. Research products, in-the-wild, prototypes, research probes, We shall conclude this paper with a reflection on Strummi, design theory a digital musical instrument (DMI) which we characterise as research product. CCS Concepts 2. CONTEXT •Applied computing → Sound and music comput- Prototypes play an important role in both HCI and DMI ing; •Information systems → Performing arts; Music research. The prototype can allow researchers to focus on retrieval; a particular aspect of interaction, isolating it from its sur- rounding context, and can inform developments of a tech- 1. INTRODUCTION nological system that improve its usability and a users’ ex- Much NIME research is driven by the exploration of novel perience. Prototypes are also often quick and cheap ways technologies applied to the musical context. One interpreta- of testing an idea. tion of the ‘new’ in NIME is that it represents a belief in new 2.1 Prototype, Probe, Product technologies opening up new means of musical expression, and leading to the creation of new music. This largely de- Prototypes play a crucial role in the crafting of an artefact: pends on advancements in the core technology that under- certain questions about an artefact’s design, particularly pin instrument making, and hence an instrument’s function- those that relate to tangible experience, are best answered ality and ability to create music. Another main strand of through observing interaction with a semi-finished artefact NIME considers the agency of musical instruments within a that has enough of its design in place as to guide the in- teraction of the user. Based on observation of this interac- tion the designer can hone their future decisions to better cater for the kind of experiences that they want to encour- age through their design. In this sense, prototypes serve to Licensed under a Creative Commons Attribution elicit feedback from the people who use them with the goal 4.0 International License (CC BY 4.0). Copyright of improving their design in the future [19]. The role of the remains with the author(s). prototype is often understood as a means of helping gather NIME’20, July 21-25, 2020, Royal Birmingham Conservatoire, requirements that will inform future designs, whether these Birmingham City University, Birmingham, United Kingdom. requirements are specifically known or not [13]. 446 Where prototypes may fall short is in their ability to ad- be asked [19]. Research products are emphatically distinct dress more complex issues about how a technological device from commercial products - we are not interested in their intervenes in everyday life and how the device itself is con- commercial potential or how many copies have been made: ditioned and appropriated by its surrounding context. Pro- they are only designed to fulfil a research inquiry. Any simi- totypes vary in their fidelity. Some are component parts larities with commercial products would only arise from the of a bigger system, others are a full system but are low- shared goals of producing ‘finished’ artefacts that can be fi or approximate in certain ways: differing in size, finish, easily imagined being adopted in real-world settings. behaviour and feel from a potential final product. 2.2 Prototypes in NIME 2.1.1 Technology Probes There is a focus in much DMI research on the functional el- When considering the process of co-designing technologies ements of an instrument as research contributions (sensors, with users the method of the ‘technology probes’ by Hutchin- mappings, sound model) even though there are many other son et al. [10] provides a useful extension of the prototype. aspects of a design that affect performer experience. Due Building upon Gaver et al.’s ‘cultural probes’ [5], technology to this functional focus, the prototype is the most common probes are defined as lo-fi devices, whose main technological form of research artefact at NIME. Prototype instruments hurdles are solved, which can be deployed in an everyday sit- often stand in as a demonstration of a principle and the uation to gather information about technology use in that types of evaluation conducted with these instruments often situation. They are created in order to serve three goals: encourages people to reflect on the qualities of the instru- the design goal of inspiring users and researchers to think ment and to suggest limitations or possible improvements. about new technologies, the engineering goal of field-testing Specifically, the iterative nature of NIME design and evalua- the technology, and the social science goal of understanding tion can lead to a focus amongst designers, performers, com- the needs and desires of users in a real-world setting. posers and audience on how an instrument might change or Probes get their name from their ability to ‘probe’ a cer- evolve, rather than be treated as a finished artefact. tain situation and gather information on it. They at once Prototypes are essential research tools for many areas of disrupt an everyday situation and then monitor what hap- enquiry relating to DMIs. In this paper we are interested pens after that disruption. Technology probes are not de- in how we can best categorise the research motivations of signed to be neutral or invisible devices that can gather instrument building which includes the creation of the arte- objectively without interfering in an everyday situation. In fact itself, its design process, deployment and evaluation. fact the power of the technology probe methodology comes In a recent paper by Morreale et al. [18] some of these in the recognition of the disruptive impact of a design, and motivations were surveyed in the NIME community asking in using this disruption to inspire users to reflect on the questions about uptake, number of performances and initial technology and their situation and to imagine possible fu- reasons for creating an instrument. 32% of the surveyed ture technologies. In this way technology probes are similar instruments were created with potential commercialisation to the ‘counterfactual artefacts’ as proposed by Wakkary et in mind with the rest intended to answer specific research al. [21]. Equally, as a research methodology which focuses questions, test new technologies or meet an artistic desire on deploying design artifacts in-the-wild in order to learn of the creator or collaborator. something about that environment, technology probes have The evaluation of a prototype is often focused on the arte- much in common with research products. fact itself rather than the environment in which the artefact is deployed: NIME evaluation methodologies in general bor- 2.1.2 Research Products row heavily from HCI experimental design where a single Odom et al.’s ‘research products’ [19] is a response to the variable is tested in a controlled situation. However when use of ‘unfinished’ prototypes in design research: dealing with a cultural form as rich and complicated as a musical instrument finding the right evaluation framework ‘The complexities and challenges in researching is a difficult task [1].