Proceedings of the 2008 Conference on New Interfaces for Musical Expression (NIME08), Genova, Italy Slide guitar synthesizer with gestural control Jyri Pakarinen Vesa Valim¨ aki¨ Tapio Puputti Department of Signal Department of Signal Helsinki University of Processing and Acoustics Processing and Acoustics Technology Helsinki University of Helsinki University of P.O. Box 3000 Technology Technology FI-02015 TKK, Finland P.O. Box 3000 P.O. Box 3000 tapio.puputti@tkk.fi FI-02015 TKK, Finland FI-02015 TKK, Finland jyri.pakarinen@tkk.fi vesa.valimaki@tkk.fi ABSTRACT x(n) y(n) This article discusses a virtual slide guitar instrument, re- -L -L cently introduced in [7]. The instrument consists of a novel gc f I CSG Hl(z) z z physics-based synthesis model and a gestural user interface. Contact sound Loop filter Energy Fractional Integer delay The synthesis engine uses energy-compensated time-varying generator compensation delay digital waveguides. The string algorithm also contains a parametric model for synthesizing the tube-string contact sounds. The real-time virtual slide guitar user interface em- Figure 1: The signal flow diagram of the slide guitar ploys optical gesture recognition, so that the user can play string synthesizer. The energy compensation block this virtual instrument simply by making slide guitar play- compensates for the artificial energy losses due to ing gestures in front of a camera. the time-varying delays. The contact sound genera- tor (see Figure 2) simulates the handling noise due to the sliding tube-string contact. Keywords Sound synthesis, slide guitar, gesture control, physical mod- eling played by wearing the slide tube on one hand and the ring on the other, and by making guitar-playing gestures in front of the camera. The user’s gestures are mapped into synthe- 1. INTRODUCTION sis control parameters, and the resulting sound is played The term slide- or bottleneck guitar refers to a specific back through the loudspeaker in real-time. More informa- traditional playing technique on a steel-string acoustic or tion on gestural control of music synthesis can be found e.g. electric guitar. When playing the slide guitar, the musician in [8] and [16]. wears a slide tube on the fretting hand. Instead of pressing From the control point of view, the VSG can be seen as the strings against the fretboard, she or he glides the tube a successor of the virtual air guitar (VAG) [1] developed at on the strings while the picking hand plucks the strings in Helsinki University of Technology a few years ago. The ma- a regular fashion. This produces a unique, voice-like tone jor difference between these gesture-controlled guitar syn- with stepless pitch control. Although the tube is usually thesizers is that like in the real slide guitar, the VSG allows slid along all six strings, single-note melodies can be played a continuous control over the pitch, and also sonifies the by plucking just one string and damping the others with contact sounds emanating from the sliding contact between the picking hand. The slide tube, usually made of glass the slide tube and the imaginary string. or metal, also generates a squeaking sound while moving The VSG uses digital waveguides [11, 12] for synthesiz- along on the wound metal strings. In most cases, the slide ing the strings. A model-based contact sound generator guitar is tuned into an open tuning (for example the open is added for simulating the friction-based sounds created G tuning: D2,G2,D3,G3,B3, and D4 starting from the by the sliding tube-string contact. More information on thickest string). This allows the user to play simple chords physics-based sound synthesis methods can be found in [14]. just by sliding the tube into different positions on the guitar neck. The player usually wears the slide tube on the pinky or ring finger, and the other fingers are free to fret the 2. STRING SYNTHESIS strings normally. A single-delay loop (SDL) digital waveguide (DWG) model A virtual slide guitar (VSG) [7, 4] is described in this [2] with time-varying pitch forms the basis of the slide gui- paper. The VSG consists of an infra-red (IR) camera, IR- tar synthesis engine, as illustrated in Fig. 1. The string reflecting slide tube and a ring, a computer running a physics- model consists of a feedback delay loop with an additional based string algorithm, and a loudspeaker. The VSG is loop filter, an energy scaling coefficient, and a contact sound generator block. The fractional delay filter in Fig. 1 allows for a smooth transition between pitches, and also enables the correct tuning of the string. There are several tech- Permission to make digital or hard copies of all or part of this work for niques for implementing fractional delay filters, a thorough personal or classroom use is granted without fee provided that copies are tutorial being found in [3]. For the purpose of this work, not made or distributed for profit or commercial advantage and that copies a fifth-order Lagrange interpolator was found to work suf- bear this notice and the full citation on the first page. To copy otherwise, to ficiently well. It must be noted that both the integer delay republish, to post on servers or to redistribute to lists, requires prior specific line length and the fractional delay filter are time-varying, permission and/or a fee. NIME08, Genoa, Italy i.e. the user controls the total loop delay value and thus Copyright 2008 Copyright remains with the author(s). also the pitch during run-time. 49 Proceedings of the 2008 Conference on New Interfaces for Musical Expression (NIME08), Genova, Italy (a) Filter 1-g The basis of the synthetic contact sound for wound strings bal is produced in the noise pulse train generator (Fig. 2, block (a)). It outputs exponentially decaying noise pulses at the (d) Filter Waveshaper g given firing rate. In addition, the type of the string deter- TV mines the decay time and duration of an individual pulse. gbal g For enhancing the harmonic structure of the contact noise (b) (c) user L(n) Smooth |x| on wound strings, the lowest time-varying harmonic is em- - nw fc(n) phasized by filtering the noise pulse train with a second- z-1 order resonator (block (b)), where the firing rate controls the resonators center frequency. The higher harmonics are Figure 2: The contact sound generator block. The produced by distorting the resonators output with a suit- sliding velocity controlled by the user commands able nonlinear waveshaper (block (c)). A scaled hyperbolic the synthetic contact noise characteristics. The tangent function is used for this. Hence, the number of sub-blocks are (a) the noise pulse generator, (b) a higher harmonics can be controlled by changing the scaling resonator creating the first harmonic of the time- of this nonlinear function. varying noise structure, (c) a static nonlinearity A 4th-order IIR filter (block (d)) is used for simulating generating the upper time-varying harmonics, and the static longitudinal string modes and the general spec- (d) an IIR filter simulating the general spectral tral shape of the contact noise. As the noise characteristics characteristics of the noise. depend on the tube material and string type, different filter parameters are used for different slide tube and string con- figurations. In Fig. 2, the scaling coefficient gbal controls The loop filter is a one-pole lowpass filter that simulates the ratio between the time-varying and static contact sound the vibrational losses of the string. Different filter parame- components. Finally, the total amplitude of the synthetic ters are used depending on the length and type of the string, contact noise is controlled by the slide velocity fc(n), via a as suggested in [15]. Also, when changing the length of a scaling coefficient gTV. Parameter guser allows the user to DWG string during run time, the signal energy is varied [5]. control the overall volume of the contact sound. For plain, In practice, this can be heard as an unnaturally quick decay i.e. unwound strings, the contact sound synthesis block is of the string sound. A time-varying scaling technique, in- simplified by replacing the noise burst generator (block (a) troduced in [5], was used as a compensation. This results in in Fig. 2) with a white noise generator, and by omitting an additional scaling operation inside the waveguide loop, blocks (b), (c), and (d). as illustrated in Fig. 1. 3. REAL-TIME IMPLEMENTATION 2.1 Contact Sound Synthesis Since the user controls the pitch of the VSG in a continu- The handling sounds created by the sliding tube-string ous manner, it is important that there is not a large latency contact are very similar to the handling sounds between a between the user’s action and the resulting sound. Thus, a sliding finger-string contact. A recent study [6] revealed high frame rate (120 fps) infra-red (IR) camera is used for that these squeaky sounds consist mainly of lowpass-type detecting the users hand locations. The camera operates by noise with both static and time-varying harmonic compo- lighting the target with IR-LEDs and sensing the reflected nents. The lowpass-cutoff frequency, frequencies of the time- IR light. A real slide tube coated with IR reflecting fabric is varying harmonics, and the overall magnitude of the contact used for detecting the users fretting hand. For recognition noise are controlled by the sliding velocity. of the picking hand, a small ring of IR reflecting fabric is For synthesizing the handling sounds, we chose a noise worn on the index finger.