Exposing the Scaffolding of Digital Instruments with Hardware-Software Feedback Loops
Total Page:16
File Type:pdf, Size:1020Kb
Proceedings of the International Conference on New Interfaces for Musical Expression, Baton Rouge, LA, USA, May 31-June 3, 2015 Exposing the Scaffolding of Digital Instruments with Hardware-Software Feedback Loops Andrew P. McPherson Victor Zappi Centre for Digital Music Media and Graphics Interdisciplinary Centre Queen Mary University of London University of British Columbia London, United Kingdom Vancouver, BC, Canada [email protected] [email protected] ABSTRACT transistors produce roughly equivalent results in linear am- The implementation of digital musical instruments is of- plification, but their sound when pushed into distortion is ten opaque to the performer. Even when the relationship noticeably di↵erent, such that tube amplifiers remain dom- between action and sound is readily understandable, the inant despite cost and practical shortcomings. internal hardware or software operations that create that Similarly, in the 19th century, the high-tension steel strings relationship may be inaccessible to scrutiny or modifica- in the piano were the means to an end of creating loud tion. This paper presents a new approach to digital in- and sustained tones, but the design also makes possible strument design which lets the performer alter and subvert John Cage’s prepared piano, the bowed ensemble pieces of the instrument’s internal operation through circuit-bending Stephen Scott, and electromagnetic actuation techniques [3, techniques. The approach uses low-latency feedback loops 14]. A hypothetical digital piano which perfectly emulated between software and analog hardware to expose the in- the sound and feel of an acoustic grand would nonetheless ternal working of the instrument. Compared to the stan- be more restrictive than the original, since these and other dard control voltage approach used on analog synths, al- interventions would be impossible. terations to the feedback loops produce distinctive and less 1.1 Implementation of Digital Instruments predictable changes in behaviour with original artistic ap- The internal implementation of a digital musical instrument plications. This paper discusses the technical foundations of (DMI) is often a black box. Even when the nominal rela- the approach, its roots in hacking and circuit bending, and tionship between action and sound is obvious, the software case studies of its use in live performance with the D-Box and hardware processes that create that relationship may be “hackable instrument”. inaccessible to the performer. This inaccessibility can pre- vent the kinds of creative misuses and modifications which Author Keywords are common on acoustic instruments [20]. Feedback, circuit bending, hacking, analog circuits, low la- This paper presents a new method of exposing the inter- tency, embedded hardware nal operations of DMIs, allowing performers to modify or subvert the original design for creative purposes. The aim ACM Classification is not to explain or demonstrate the instrument’s nominal H.5.5 [Information Interfaces and Presentation] Sound and mapping to the performer or audience, but rather to make Music Computing—Methodologies and techniques, C.3 [Special- the implementation of the instrument open to repurposing. Purpose and Application-Based Systems] Real-time and em- 1.2 Appropriation, Modification and Hacking bedded systems, H.5.2 [Information Interfaces and Presen- In an earlier study [21], we showed how even very simple tation] User Interfaces—Input devices and strategies. instruments were appropriated by performers for use in un- expected ways. Surprisingly, our results suggested that per- 1. INTRODUCTION fomers saw instruments with fewer degrees of freedom as The implementation of an instrument matters. For the de- richer, and that tight constraints provided greater incen- signer, the exact selection of materials, circuits or code may tives to develop creative use cases. be primarily a means to an end, a way of achieving a partic- This techniques in this paper widen our study of appro- ular sound. For the performer, though, the subtle idiosyn- priation to include modifications to the instrument’s inter- cracies and limitations of how the instrument behaves can nal operation, while retaining the essential simplicity of the become powerful sources of creative inspiration, giving rise instrument. Specifically, our techniques aim to: to new playing techniques and even new styles of music. Expose the internal state of an instrument’s software For example, in early guitar amplifiers, distortion was an • engineering limitation until guitarists discovered the mu- (not just sensor inputs) to modification in hardware. sical value in pushing the device beyond its limits. The Do not limit the allowable modifications to choosing • amplifier’s internal construction matters just as much as from predefined options created by the designer. its nominal action of making the guitar louder: tubes and Make modifications interesting: the instrument should • not crash or go silent, and the behaviour when modi- fied should include unusual but repeatable e↵ects. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are Our approach is inspired by circuit bending practice (Sec- not made or distributed for profit or commercial advantage and that copies tion 2.4). The technical implementation uses low-latency bear this notice and the full citation on the first page. To copy otherwise, to feedback loops between software and hardware which pro- republish, to post on servers or to redistribute to lists, requires prior specific duce unique characteristic e↵ects when altered. To state permission and/or a fee. NIME’15, May 31-June 3, 2015, Louisiana State Univ., Baton Rouge, LA. the goal more compactly: Make the instrument simple, Copyright remains with the author(s). but make it break in interesting ways. 162 Proceedings of the International Conference on New Interfaces for Musical Expression, Baton Rouge, LA, USA, May 31-June 3, 2015 2. BACKGROUND Most familiar instruments have undergone many gener- The concept of modularity has been an important factor ations of testing and refinement, often incorporating what in the development of electronic instruments, but it is not were originally unexpected usage techniques (e.g. fuzz boxes). the only approach to modifiability. This section examines The limitations opaque DMIs impose on the performer may analog control voltages (CVs) and their influence on DMI thus end up a↵ecting their future development. mapping before considering limitations and alternatives. 2.4 Modularity versus Circuit Bending 2.1 Dominance of the Control Voltage Circuit bending [6] rejects the black box model of digital electronic devices, working directly with the circuits and Hugh LeCaine’s Electronic Sackbut (1945-48) is widely seen signals inside. Modularity is irrelevant, since any trace on as the first voltage-controlled synthesizer [19]. Its keyboard the board that is physically accessible can become part of a features pitch and volume control from lateral and down- modification. Some objects become prized circuit bending ward pressure, respectively, while a set of pressure sensors targets precisely because of their implementation. The TI played with the left hand control various aspects of timbre. Speak and Spell (1978) has been used in dozens of hacks The modular synths of Moog and Buchla in the 1960’s and to produce new sound e↵ects [9]. A smartphone emulation 70’s established control voltages as the dominant paradigm of the device would hold no interest for circuit bending; for analog synthesis [17]. arbitrary modifications would lead to a crash or damage to The advantage of the CV lies in its conceptual clarity the device rather than creating interesting e↵ects. and interconnectability: by describing each musical param- It is rare to see a circuit bender working with a latest- eter with a single continuous variable, any signal source can generation DMI, other than one they have created. We conceivably be attached to any destination with predictable thus argue for a new approach to DMI design which min- results. Physical patch cords for routing CVs are a staple imises opaque software processes and exposes as much of of analog modular synths, but modular approaches can be the substrate as possible to modification by the performer. found whether or not cords are literally present, for example in modulation matrices or switches for reassigning control 2.5 Feedback in Musical Instruments 1 parameters. Recent kits such as littleBits and Patchblocks Feedback in performance dates back to the 1960’s [18], with have also featured interconnectable hardware modules. examples including amp-to-pickup on electric guitar, Tu- dor’s Microphone (1973) and other works, Di Scipio’s Au- 2.2 Digital Musical Instrument Mapping dible Ecosystems [7], Collins’ 1974 Pea Soup, Nakamura’s Mapping in DMIs refers to the assignment of control (sen- 1997 No-Input Mixing Board [18], and Bowers and Haas’s sor) inputs to instrument outputs (audio, visual, haptic). “hybrid resonant assemblages” [4]. Feedback through skin Many mapping strategies have been proposed, including conductance features in Waisvisz’s Cracklebox.2 one-to-one and many-to-many relationships [10], multi-layer In comparison to previous work, this paper explores feed- approaches [1], and dynamic performer-adjustable mappings back as a control mechanism within a larger DMI rather [12]. Machine learning is increasingly used for creating map- than as a direct audio output. It also demonstrates the pings [8, 11, 5]. Recent e↵orts have also focused on making construction of loops between software and analog electron- mappings understandable to the audience [2, 16]. ics whose e↵ects could not be created with either modality Mapping, particularly in its more direct forms, can be alone. Finally, it demonstrates a circuit-bending methodol- seen as a digital extension of the CV: musical parameters ogy for modifying feedback loops to alter the behaviour of are associated with one or more continuous or discrete vari- a primarily software-driven instrument. ables, which are then assigned to various sensors or other signal generators.