Toward User-Friendly, Sophisticated New Musical Instruments
Total Page:16
File Type:pdf, Size:1020Kb
Sergi Jorda` FMOL: Toward Music Technology Group, Audiovisual Institute Pompeu Fabra University Passeig de la Circumval•lacio´8 User-Friendly, 08003 Barcelona, Spain [email protected] Sophisticated New Musical Instruments The design of new instruments and controllers for New Musical Instruments and New performing live computer music is an exciting field Music-Making Paradigms of research that can lead to truly new music- making paradigms. The ever-increasing availability The conception and design of new musical inter- of sensing technologies that enable virtually any faces is a burgeoning multidisciplinary area where kind of physical gesture to be detected and tracked technological knowledge (sensor technology, sound has indeed sprouted a wealth of experimental in- synthesis and processing techniques, computer pro- struments and controllers with which to explore gramming, etc.), artistic creation, and a deep under- new creative possibilities. However, the design of standing of musicians’ culture must converge to these new controllers is often approached from an create new interactive music-making paradigms. If essentially technical point of view in which the new musical interfaces can be partially responsible novelty of the sensing technologies deployed over- for shaping future music, these new musical para- shadows the attainable musical results. Although digms should not be left to improvisation. We are new instruments are not constrained to the physi- not asserting that the design of new instruments cal restrictions of traditional ones, an integrated ap- must follow any given tradition, but when trying to proach in which the instrument is designed as a define new models, we cannot ignore existing ones. whole and not as a combination of arbitrary input A wide knowledge of these, together with the per- and output devices can lead to the creation of more sonal beliefs and intuitions of each designer, should rewarding and more musical instruments. provide orientation about what could be changed This article describes an attempt at an integrated and what could be kept in incoming designs. conception, called F@ust Music On-Line (FMOL), Prior knowledge may originate from different which is a simple mouse-controlled instrument fields or practices, but we could roughly consider that has been used on the Internet by hundreds of that new instruments designers have three major musicians during the past four years. FMOL has backgrounds to deal with: the first is a millennial been created with the complementary goals of in- one, as old and rich as the history of music- troducing the practice of experimental electronic making, while the two others are less than one music to newcomers while trying to remain attrac- half-century-old. We are talking about computer tive to more advanced electronic musicians. It has music and the realm of human-computer interac- been used by musicians of diverse skills for the col- tion (HCI). What we are suggesting is that new mu- lective composition of the music of two important sical controllers must obviously include new shows, including one opera of the Catalan theatre capabilities that have been unavailable and even group La Fura dels Baus. It is also being played in unimaginable in traditional mechanical instru- live concerts. ments. However, they should also try to recover es- sential components from the acoustic legacy that have been left out during the recent decades of computer music owing to the former technical lim- itations in digital technologies. Moreover, they could benefit as well from all the existing corpus of knowledge and research in non-musical areas of Computer Music Journal, 26:3, pp. 23–39, Fall 2002 HCI (Wanderley 2001; see also www.csl.sony.co.jp ᭧ 2002 Massachusetts Institute of Technology. /ϳpoup/research/chi2000wshp/papers/orio.pdf). Jorda` 23 Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/014892602320582954 by guest on 30 September 2021 Musical Interfaces Are Not Musical Instruments provements while remaining within MIDI’s con- straints. A recent study toward that direction was Musical instruments transform the actions of one offered by Goebl and Bresin (2001), who investi- or more performers into sound. An acoustic instru- gated the measuring and reproducing capabilities of ment consists of an excitation source under the a Yamaha Disklavier MIDI piano. They concluded control of the performer(s) and a resonating system that the weakest point of that system does not lie that couples the vibrations of the excitation to the in the recorded MIDI information but in how this surrounding atmosphere. This affects, in turn, the information is used for reproduction. precise patterns of vibration. In most acoustic in- Our complaint is much more banal but at the struments (apart from organs and other keyboard same time frequently overlooked, if we consider instruments) the separation between the input and the number of generic new musical controllers de- the excitation subsystems is unclear. Digital musi- signed today. Is it possible to design nonspecific cal instruments, on the other hand, can always be controllers, to develop highly sophisticated inter- divided into a gestural controller or input device faces without a profound assumption of how the that takes control information from the per- sound or music generators will work? Apart from former(s) and a sound generator that plays the role the truism that not all combinations of controllers– of the excitation source. Digital instruments may generators are equally satisfying (nothing will also include a virtual resonating system, but real match a MIDI wind-controller when driving a resonators are usually missing and substituted by a physical model synthesizer of a woodwind instru- digital-to-analog converter (DAC) that feeds an am- ment), the final expressiveness and richness of any plifier connected to a speaker system. Often, when musical interface (controller) cannot be indepen- discussing new digital instruments, we are referring dent of its generator and the mapping applied be- in fact only to an input device component. In that tween them. sense, the Theremin is a complete musical instru- We firmly believe that a parallel design between ment, while a MIDI Theremin is only a controller controllers and generators is needed to create new or an interface, and not necessarily a musical one. sophisticated musical instruments and music- The separation between gestural controllers and making paradigms. Both design processes should sound generators as standardized by MIDI led to therefore be treated as a whole. A complementary the creation and development of many new alterna- approach to this separation problem is proposed by tive controllers with which to explore new creative Ungvary and Vertegaal (2000), creators of the possibilities (e.g., Chadabe 1996; Wanderley and SensOrg Cyberinstrument, a modular and highly Battier 2000). However, this separation should not configurable assembly of input/output hardware always be seen as a virtue or an unavoidable tech- devices designed for optimal musical expression. nical imposition. The most frequent criticisms of this division deal with the limitations of the proto- col that connects these two components of the in- strument chain. Are today’s standard music Mapping and Feedback Strategies communication protocols (e.g., MIDI) flexible enough, or are the potential dialogues between con- Whether following a parallel design or maintaining trollers and generators limited or narrowed by two independent approaches, the physical and logi- these existing protocols? MIDI does indeed have cal separation of the input device from the sound many limitations (low bandwidth, low resolution, production necessitates multiple ways of process- inaccurate timing, half-duplex communications, ing and mapping the information coming from the etc.) (Moore 1988), but we optimistically believe input device. Mapping becomes therefore an essen- that there is still room for experimentation and im- tial element in designing new instruments. 24 Computer Music Journal Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/014892602320582954 by guest on 30 September 2021 Mappings in acoustical instruments are often Duplex communications allow the existence of multidimensional and slightly nonlinear. Blowing haptic and force feedback manipulations, which harder on many wind instruments not only affects may bring richer dialog loops between the instru- dynamics but also influences pitch, and in these ment and the performer (Bongers 1994; Gillespie control difficulties lie much of their expressive- 1999). However, force feedback is not the only ness. As Johan Sundberg suggests, the voice, possi- feedback loop present in traditional music playing. bly the most expressive instrument ever, can also An acoustical instrument resonates, affecting its be considered to be one of the most poorly designed patterns of vibration, and this acoustic feedback from an engineering point of view (Sundberg 1987), that takes place inside the instrument is funda- taking into account the complex relationship be- mental to the overall sound production process. tween articulator movement and formant fre- We could metaphorically imagine that by this feed- quency changes that makes articulation/formant back, the instrument is being ‘‘conscious’’ of the frequency a one-to-many system. sound it is producing and that this information is For the sake of simplicity, many electronic in- also given back to the performer. struments use basic