A Hybrid Keyboard-Guitar Interface Using Capacitive Touch Sensing and Physical Modeling

A Hybrid Keyboard-Guitar Interface using Capacitive Touch Sensing and Physical Modeling Christian Heinrichs Andrew McPherson Queen Mary University of London Queen Mary University of London [email protected] [email protected] ABSTRACT are interested in encouraging creative exploration of these parameters, and also in transferring the experience of gui- Physical modeling allows realistic guitar synthesis incor- tar playing to musicians with little or no guitar training. porating many expressive dimensions commonly employed Traditional keyboard interfaces, even when coupled to so- by guitarists, including pluck strength and location, plec- phisticated synthesizers, largely fail to capture this expe- trum type, hand damping and string bending. Often, when rience: techniques as foundational as strumming have no a physical model is used in performance, most control di- keyboard analog, the distinction between pitch selection mensions go unused when the interface fails to provide and note activation does not exist, and expressive dimen- a way to intuitively control them. Techniques as foun- sions including pluck location and string bending are diffi- dational as strumming lack a natural analog on the MIDI cult to achieve. On the other hand, GUI interfaces lack the keyboard, and few digital controllers provide the indepen- physicality of instrumental playing. dent control of pitch, volume and timbre that even novice To provide both physicality and multidimensionality, we guitarists achieve. This paper presents a hybrid interface control the physical model using a touch-sensitive musical based on a touch-sensing keyboard which gives detailed keyboard we recently developed [2] which measures the expressive control over a physically-modeled guitar. Most location and contact area of fingers on the key surfaces. dimensions of guitar technique are controllable polyphon- Mapping between touch data and physical model param- ically, some of them continuously within each note. Map- eters thus becomes a primary focus for study, with the pings are evaluated in a user study of keyboardists and gui- goal of providing intuitive control to guitarists and non- tarists, and the results demonstrate its playability by per- guitarists alike. Our goal is not to replicate the guitar; formers of both instruments. rather, we seek to explore the creative possibilities of a hybrid instrumental technique while providing an instrument 1. INTRODUCTION non-guitarists can use to produce common guitar techniques. In the remainder of this paper, we examine related work Physical modeling allows detailed simulation of familiar and present our physical model, with a focus on the rele- acoustic instruments and the creation of novel instruments, vance of the control parameters to instrumental technique. even physically impossible ones. But in performance, a We then discuss how these parameters are controlled by good model is only half the challenge: the control inter- physical gestures extending traditional keyboard technique. face and the mapping from gesture to sound strongly in- Our results are evaluated in a user study of 10 keyboard and fluence the resulting interaction. Gelineck and Serafin [1] guitar players. propose a set of directives for controlling physical models with the goal of encouraging creativity and exploration. 1.1 Related Interfaces Among these are a balance between the model’s sonic di- versity and plausibility, the control of physical models with Our work connects to a long tradition of extended key- physical gestures, and experimentation with the interplay board interfaces. The keys of the early 20th-century On- between instantaneous and continuous gestures. They de- dioline [3] could be moved horizontally to create vibrato scribe a modular system in which, among other things, ges- effects. Moog and Rhea created a keyboard which sensed tures from one instrumental technique can be coupled to the position of fingers on the key surface, much like our sound sets associated with another instrument, producing present design [4]. Other recent extended or abstracted new hybrid musical systems. keyboards includes the Continuum [5], the Seaboard [6], In this paper we describe the development of one such the Hyperkeys 1 and the Endeavour Evo 2 . cross-instrumental interface. A physical model of a guitar Our goal of simple, expressive guitar-like sounds also has is proposed whose control parameters intuitively relate to a venerable history. The autoharp and its electronic suc- the actions of the left and right hands in guitar playing. We cessors including the Suzuki Omnichord are expressly designed to simplify chordal playing. More broadly, many Copyright: c 2012 Christian Heinrichs et al. This is graphical and physical interfaces, including the Ghost con- an open-access article distributed under the terms of the troller [7] and Apple’s GarageBand for iPad, are designed Creative Commons Attribution 3.0 Unported License, which permits unre- around the principle of separating chord selection and string stricted use, distribution, and reproduction in any medium, provided the original 1 http://www.hyperkeys.com/ author and source are credited. 2 http://www.endeavour.de/ activation. The Kalichord [8] achieves a wide tonal range a guitar body it is not a physical simulation. This means by driving physical string models with signals from piezo that in order to adjust any sounds resulting from the guitar tines plucked by the performer. The performer’s other hand body (e.g. resonances at particular frequencies) new sets of controls a series of buttons selecting the string tunings. impulse responses need to be recorded. In our implemen- In comparison to previous work, this paper seeks a more tation we convolved these wavetables with freely available explicit hybrid of control gestures from keyboard and gui- room impulse responses as an efficient way of achieving tar. It also offers many degrees of expressive control in the reverberation. context of a detailed physical model whose parameters are intuitive for musicians to understand. 2.1.4 Coupling On real instruments, vibrations from one string will pro- 2. PHYSICALLY-MODELED GUITAR STRING duce sympathetic resonances in the others. The model uses a two-dimensional waveguide, and we use the output from 2.1 Core Synthesis the vertical SDL of each string to resonate the horizontal The aim was to create a realistic and computationally ef- SDL of each other string, thereby avoiding feedback prob- ficient model of a guitar that is controllable by parame- lems. This leaves us with six inputs and six outputs for ters that have an intuitive relationship to the player. The each string. These can be routed to each other by a 6x6 ma- model developed for this study is based on Karjalainen, trix. The volume of each output scales with the consonance Valim¨ aki¨ and Tolonen’s extensions of the Karplus-Strong of the frequency ratio between strings. The frequency ra- algorithms [9], shown in Figure1. tio between driving and observed strings is calculated; the closer it is to a multiple of 0.5, the higher the amount of consonance. All couplings can be globally scaled as an adjustable parameter, with small coupling values producing the best results. This is an efficient albeit simplified way of recreating string coupling effects without relying on precise but computationally expensive techniques. For more detailed im- Figure 1. Signal chain of guitar model. plementations see [10] and [11]. 2.1.5 String Variance 2.1.1 Single Delay-Loop model Our model makes further use of vertical and horizontal The propagation of energy along a string in both directions waveguides by adding a slight random variance to the cut- can be simplified to a single delay-loop (SDL) model. This off frequency, feedback level and pitch of the respective consists of a simple delay line with a low pass filter inte- SDLs. The resulting sound qualities are a less predictable, grated into the loop circuit, of which the Karplus-Strong more exponential decay envelope and a subtle beating ef- algorithm can be seen to be a special case. Two SDLs lie fect caused by the two frequencies that are slightly out of at the center of the model corresponding to the vertical and tune. The amount of variance is an adjustable parameter in horizontal motion along the string. our model. 2.1.2 Harmonics 2.2 High-Level Parameters and Control Two further delay lines (DL1 and DL4) are integrated into 2.2.1 Changing Notes the signal chain to simulate excitation position and left- hand harmonics. To simulate changing notes on a string without introducing The frequency is controlled by calculating the exact posi- artifacts in the output, feedback values of the delay-lines tion on the string, which, in the case of the finger harmonic, are ramped down and back up again as the frequency is relates to the corresponding note that would sound if the changed. Simultaneously, the transition between frequen- finger were held down firmly. The held note can be ob- cies must occur with sufficient amount of low-pass filtering tained by taking the inverse of the string length (1/L) and to avoid both abrupt artifacts and undesired portamento. multiplying this by the fundamental frequency. The har- Suitable parameters were chosen by trial and error and il- monic is obtained by calculating the ratio from the string lustrated in Figure2. length between the finger and the nut to the entire string 2.2.2 Pitch Bend length and multiplying the inverse by the frequency. The resonant frequency of the delay lines can be changed 2.1.3 Excitation to bend a note’s pitch. Low-pass filtering of the frequency To reduce computation, wavetables from recorded impulse parameter is used during a bend to avoid zipper noise and responses are used to excite the signal chain. Six impulse other interference. responses were recorded at the bridge, the body and the fin- 2.2.3 Finger Pressure gerboard of an acoustic guitar.

Load more