Recent Advances in Real-Time Musical Effects, Synthesis, and Virtual Analog Models
Total Page：16
File Type：pdf, Size：1020Kb
Hindawi Publishing Corporation EURASIP Journal on Advances in Signal Processing Volume 2011, Article ID 940784, 15 pages doi:10.1155/2011/940784 Review Article Recent Advances in Real-Time Musical Effects, Synthesis, and Virtual Analog Models Jyri Pakarinen,1 Vesa Valim¨ aki,¨ 1 Federico Fontana,2 Victor Lazzarini,3 and Jonathan S. Abel4 1 Department of Signal Processing and Acoustics, Aalto University School of Electrical Engineering, 02150 Espoo, Finland 2 Department of Mathematics and Computer Science, University of Udine, 33100 Udine, Italy 3 Sound and Music Technology Research Group, National University of Ireland, Maynooth, Ireland 4 CCRMA, Stanford University, Stanford, CA 94305-8180, USA Correspondence should be addressed to Jyri Pakarinen, jyri.pakarinen@tkk.fi Received 8 October 2010; Accepted 5 February 2011 Academic Editor: Mark Kahrs Copyright © 2011 Jyri Pakarinen et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. This paper reviews some of the recent advances in real-time musical effects processing and synthesis. The main emphasis is on virtual analog modeling, specifically digital emulation of vintage delay and reverberation effects, tube amplifiers, and voltage- controlled filters. Additionally, adaptive effects algorithms and sound synthesis and processing languages are discussed. 1. Introduction synthesis can be found in articles [3]and[4], respectively. A tutorial on virtual analog oscillator algorithms, which are ff Real-time musical e ects processing and synthesis play a not tackled in this paper, has been written by Valim¨ aki¨ part in nearly all musical sounds encountered in the con- and Huovilainen [5]. Also, musical synthesis and effects temporary environment. Virtually all recorded or electrically applications for mobile devices have been reported in [6]. In ff amplified music in the last few decades uses e ects process- order to conveniently fit in a single journal article, a selection ing, such as artificial reverberation or dynamic compression, of some of the most active subtopics under this exciting and synthetic instrument sounds play an increasingly larger research field have been chosen for presentation here. part in the total musical spectrum. Furthermore, the vast The organization of this review is as follows: adaptive ff majority of these e ects are presently implemented using effects processing algorithms, such as the adaptive FM digital signal processing (DSP), mainly due to the flexibility technique, are reviewed in Section 2. Section 3 discusses the and low cost of modern digital devices. For live music, real- emulation of vintage delay and reverberation effects, while ff time operation of these e ects and synthesis algorithms is recent advances in tube amplifier emulation are studied obviously of paramount importance. However, also recorded in Section 4. Real-time simulation of an interesting analog music typically requires real-time operation of these devices effects device, the voltage-controlled filter, is reviewed in and algorithms, because performers usually wish to hear the Section 5, and recent advances in sound synthesis and final, processed sound of their instrument while playing. processing languages are discussed in Section 6.Finally, The purpose of this article is to provide the reader with Section 7 concludes the review. an overview of some of the recent advances in this fascinating and commercially active topic. An exhaustive review of all novel real-time musical effects processing and synthesis 2. Adaptive Effects Processing would fill a book. In fact, an earlier review on digital audio effects can be found in the book [1] and in a recent book [2], Many adaptive effects processing algorithms suitable for a while reviews of virtual analog modeling and digital sound general input signal have been introduced during the past few 2 EURASIP Journal on Advances in Signal Processing Mod. Bias depth Mod. depth x(n) Pitch tracker Sin osc x(n) Low pass Delay y(n) Low pass Delay y(n) (a) (b) Mod. Mod. depth depth x(n) Low pass x(n) Low pass All pass y(n) SDF y(n) (c) (d) Mod. depth x(n) Low pass Delay High pass y(n) (e) Figure 1: Recent adaptive effects processing structures: (a) self-modulating FM [7], (b) adaptive FM [8], (c) coefficient-modulated all-pass filter [9], (d) coefficient-modulated spectral delay filter (SDF) [10], and (e) brassification [11]. years. The idea of an adaptive audio effect is not entirely new: (AdFM). Poepel and Dannenberg had proposed a basic it has been possible for many years to control parameters modified FM synthesis method in which the modulator is of an algorithm with a feature measured from the signal. replaced with the input audio signal [7]. Lazzarini et al. [8] Still, it was found useful to give the name “Adaptive DAFx” reversed the roles of the modulator and the carrier so that to this class of methods a few years ago [12], and since they use the input signal as the carrier. It is advantageous then many papers belonging to this category have been to low-pass filter the carrier signal before modulating it, published. In this section, we briefly review some recent since the spectrum of the signal will expand because of methods belonging to this category of real-time musical frequency modulation and the output sound will otherwise signal processing algorithms. become very bright. The pitch of the input signal, however, Audio-driven sound synthesis introduced by Poepel and is used to control the modulation frequency. In AdFM, Dannenberg [7] is an example of a class of adaptive effects, the modulation is implemented by moving the output tap which goes so far as almost being a synthesis method rather of a delay line at the modulation frequency, as shown in than a transformation of the input signal. In one example Figure 1(b). A fractional delay filter is required to obtain application of this idea, Poepel and Dannenberg show how smooth delay variation [14]. The FM modulation index then FM (frequency modulation) synthesis can be modified by controls the width of this variation. An advantage of the deriving the modulation signal frequency by tracking the AdFM effect is that it retains the character of the input signal. pitch of an input signal. In this case, the input signal is In one extreme, when the modulation depth is set to zero, assumed to be a monophonic signal, such as a trumpet sound the output signal will be identical to the (low-pass filtered) picked up by a microphone. Poepel and Dannenberg also input signal. By increasing the modulation index, the method describe an algorithm, which they call self-modulating FM. distorts the input signal so that it sounds much like an FM- In this method, the low-pass filtered input signal is used synthesized tone. as both modulation and carrier signal. The modulation is Extensions to these methods were presented in [15], realized by varying the delay-line length with the scaled low- where the FM sidebands were split in four separate groups pass filtered input signal, see Figure 1(a) [13]. (in combinations of upper/lower and even/odd), and in [16] Lazzarini and his colleagues [8] extended the basic idea where asymmetric-spectra FM methods were introduced. of audio-driven synthesis to what they call adaptive FM Finally, in [17] a modified FM version was presented EURASIP Journal on Advances in Signal Processing 3 M allpass filters highpass filter is used as postprocessing. Similar methods have previously been used in waveguide synthesis models Optional to obtain interesting acoustic-like effects, such as generic amplitude-dependent nonlinear distortion [20], shock waves x(n) AP AP··· AP EQ y(n) in brass instruments [21–23], and tension-modulation in Figure 2: A spectral delay filter consists of a cascade of all-pass string instruments [24, 25]. These methods aim at imple- filters (AP) and an optional equalizing filter (EQ) [18]. menting a passive nonlinearity [26]. All these nonlinear effects are implemented by controlling the fractional delay with values of the signal samples contained in the delay line. (a variant of FM based on modified Bessel coefficients). This In the practical implementation of the brassification was complemented by an algorithm that allows transitions method, the input signal propagates in a long delay line and between modified, asymmetrical, and classic FM for adaptive the output is read with an FIR interpolation filter, such as applications. linear interpolation or fourth-order Lagrange interpolation. An adaptive effect of a similar spirit as the audio-driven The input signal can be optionally low-pass filtered prior to approach and adaptive FM was introduced by Pekonen [9]. the delay-line input to emphasize its low-frequency content In his method, presented in Figure 1(c),theaudiosignalis and the output signal of the delay line can be high-pass filtered with a first-order all-pass filter and the coefficient filtered to compensate the low-pass filtering, as shown in of that all-pass filter is simultaneously varied with scaled Figure 1(e). and possibly low-pass filtered version of the same input signal. This technique can be seen as signal-dependent phase modulation and it introduces a distortion effect, but does not 3. Vintage Delay and Reverberation Effects require a table lookup, like waveshaping, or pitch tracking, Processor Emulation like AdFM. Digital emulation of vintage electronic and electromechani- It was shown recently by Lazzarini et al. [19] that the cal effects processors has received a lot of attention recently. choice of the all-pass filter structure affects considerably the While their controls and sonics are very desirable, and output signal in the time-varying case. It was found that the the convenience of a software implementation of benefit, direct form I structure has smaller transients with the same these processors often present signal processing challenges input and coefficient modulation signals than two alternative making real-time implementation difficult.