Recent Advances in Real-Time Musical Effects, Synthesis, and Virtual Analog Models

Hindawi Publishing Corporation EURASIP Journal on Advances in Signal Processing Volume 2011, Article ID 940784, 15 pages doi:10.1155/2011/940784

Review Article Recent Advances in Real-Time Musical Effects, Synthesis, and Virtual Analog Models

Jyri Pakarinen,1 Vesa Valim¨ aki,¨ 1 Federico Fontana,2 Victor Lazzarini,3 and Jonathan S. Abel4

1 Department of Signal Processing and Acoustics, Aalto University School of Electrical Engineering, 02150 Espoo, Finland 2 Department of Mathematics and Computer Science, University of Udine, 33100 Udine, Italy 3 Sound and Music Technology Research Group, National University of Ireland, Maynooth, Ireland 4 CCRMA, Stanford University, Stanford, CA 94305-8180, USA

Correspondence should be addressed to Jyri Pakarinen, jyri.pakarinen@tkk.ﬁ

Received 8 October 2010; Accepted 5 February 2011

Academic Editor: Mark Kahrs

Copyright © 2011 Jyri Pakarinen et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

This paper reviews some of the recent advances in real-time musical effects processing and synthesis. The main emphasis is on virtual analog modeling, specifically digital emulation of vintage delay and reverberation effects, tube amplifiers, and voltage- controlled filters. Additionally, adaptive effects algorithms and sound synthesis and processing languages are discussed.

1. Introduction synthesis can be found in articles [3]and[4], respectively. A tutorial on virtual analog oscillator algorithms, which are ff Real-time musical e ects processing and synthesis play a not tackled in this paper, has been written by Valim¨ aki¨ part in nearly all musical sounds encountered in the con- and Huovilainen [5]. Also, musical synthesis and effects temporary environment. Virtually all recorded or electrically applications for mobile devices have been reported in [6]. In ff amplified music in the last few decades uses e ects process- order to conveniently fit in a single journal article, a selection ing, such as artificial reverberation or dynamic compression, of some of the most active subtopics under this exciting and synthetic instrument sounds play an increasingly larger research field have been chosen for presentation here. part in the total musical spectrum. Furthermore, the vast The organization of this review is as follows: adaptive ff majority of these e ects are presently implemented using effects processing algorithms, such as the adaptive FM digital signal processing (DSP), mainly due to the flexibility technique, are reviewed in Section 2. Section 3 discusses the and low cost of modern digital devices. For live music, real- emulation of vintage delay and reverberation effects, while ff time operation of these e ects and synthesis algorithms is recent advances in tube amplifier emulation are studied obviously of paramount importance. However, also recorded in Section 4. Real-time simulation of an interesting analog music typically requires real-time operation of these devices effects device, the voltage-controlled filter, is reviewed in and algorithms, because performers usually wish to hear the Section 5, and recent advances in sound synthesis and final, processed sound of their instrument while playing. processing languages are discussed in Section 6.Finally, The purpose of this article is to provide the reader with Section 7 concludes the review. an overview of some of the recent advances in this fascinating and commercially active topic. An exhaustive review of all novel real-time musical effects processing and synthesis 2. Adaptive Effects Processing would fill a book. In fact, an earlier review on digital audio effects can be found in the book [1] and in a recent book [2], Many adaptive effects processing algorithms suitable for a while reviews of virtual analog modeling and digital sound general input signal have been introduced during the past few 2 EURASIP Journal on Advances in Signal Processing

Mod. Bias depth Mod. depth

x(n) Pitch tracker Sin osc x(n) Low pass

Delay y(n) Low pass Delay y(n)

(a) (b)

Mod. Mod. depth depth

x(n) Low pass x(n) Low pass

All pass y(n) SDF y(n)

x(n) Low pass Delay High pass y(n)

(e)

Figure 1: Recent adaptive effects processing structures: (a) self-modulating FM [7], (b) adaptive FM [8], (c) coefficient-modulated all-pass filter [9], (d) coefficient-modulated spectral delay filter (SDF) [10], and (e) brassification [11]. years. The idea of an adaptive audio effect is not entirely new: (AdFM). Poepel and Dannenberg had proposed a basic it has been possible for many years to control parameters modified FM synthesis method in which the modulator is of an algorithm with a feature measured from the signal. replaced with the input audio signal [7]. Lazzarini et al. [8] Still, it was found useful to give the name “Adaptive DAFx” reversed the roles of the modulator and the carrier so that to this class of methods a few years ago [12], and since they use the input signal as the carrier. It is advantageous then many papers belonging to this category have been to low-pass filter the carrier signal before modulating it, published. In this section, we briefly review some recent since the spectrum of the signal will expand because of methods belonging to this category of real-time musical frequency modulation and the output sound will otherwise signal processing algorithms. become very bright. The pitch of the input signal, however, Audio-driven sound synthesis introduced by Poepel and is used to control the modulation frequency. In AdFM, Dannenberg [7] is an example of a class of adaptive effects, the modulation is implemented by moving the output tap which goes so far as almost being a synthesis method rather of a delay line at the modulation frequency, as shown in than a transformation of the input signal. In one example Figure 1(b). A fractional delay filter is required to obtain application of this idea, Poepel and Dannenberg show how smooth delay variation [14]. The FM modulation index then FM (frequency modulation) synthesis can be modified by controls the width of this variation. An advantage of the deriving the modulation signal frequency by tracking the AdFM effect is that it retains the character of the input signal. pitch of an input signal. In this case, the input signal is In one extreme, when the modulation depth is set to zero, assumed to be a monophonic signal, such as a trumpet sound the output signal will be identical to the (low-pass filtered) picked up by a microphone. Poepel and Dannenberg also input signal. By increasing the modulation index, the method describe an algorithm, which they call self-modulating FM. distorts the input signal so that it sounds much like an FM- In this method, the low-pass filtered input signal is used synthesized tone. as both modulation and carrier signal. The modulation is Extensions to these methods were presented in [15], realized by varying the delay-line length with the scaled low- where the FM sidebands were split in four separate groups pass filtered input signal, see Figure 1(a) [13]. (in combinations of upper/lower and even/odd), and in [16] Lazzarini and his colleagues [8] extended the basic idea where asymmetric-spectra FM methods were introduced. of audio-driven synthesis to what they call adaptive FM Finally, in [17] a modified FM version was presented EURASIP Journal on Advances in Signal Processing 3

M allpass filters highpass filter is used as postprocessing. Similar methods have previously been used in waveguide synthesis models Optional to obtain interesting acoustic-like effects, such as generic amplitude-dependent nonlinear distortion [20], shock waves x(n) AP AP··· AP EQ y(n) in brass instruments [21–23], and tension-modulation in Figure 2: A spectral delay filter consists of a cascade of all-pass string instruments [24, 25]. These methods aim at imple- filters (AP) and an optional equalizing filter (EQ) [18]. menting a passive nonlinearity [26]. All these nonlinear effects are implemented by controlling the fractional delay with values of the signal samples contained in the delay line. (a variant of FM based on modified Bessel coefficients). This In the practical implementation of the brassification was complemented by an algorithm that allows transitions method, the input signal propagates in a long delay line and between modified, asymmetrical, and classic FM for adaptive the output is read with an FIR interpolation filter, such as applications. linear interpolation or fourth-order Lagrange interpolation. An adaptive effect of a similar spirit as the audio-driven The input signal can be optionally low-pass filtered prior to approach and adaptive FM was introduced by Pekonen [9]. the delay-line input to emphasize its low-frequency content In his method, presented in Figure 1(c),theaudiosignalis and the output signal of the delay line can be high-pass filtered with a first-order all-pass filter and the coefficient filtered to compensate the low-pass filtering, as shown in of that all-pass filter is simultaneously varied with scaled Figure 1(e). and possibly low-pass filtered version of the same input signal. This technique can be seen as signal-dependent phase modulation and it introduces a distortion effect, but does not 3. Vintage Delay and Reverberation Effects require a table lookup, like waveshaping, or pitch tracking, Processor Emulation like AdFM. Digital emulation of vintage electronic and electromechani- It was shown recently by Lazzarini et al. [19] that the cal effects processors has received a lot of attention recently. choice of the all-pass filter structure affects considerably the While their controls and sonics are very desirable, and output signal in the time-varying case. It was found that the the convenience of a software implementation of benefit, direct form I structure has smaller transients with the same these processors often present signal processing challenges input and coefficient modulation signals than two alternative making real-time implementation difficult. In this section, structures and, thus, this expression is recommended for use we focus on recent signal processing techniques for real- in the future time implementation of vintage delay and reverberation y(n) = x(n − 1) − a x(n) − y(n − 1) , (1) effects. We first consider techniques to emulate reverberation chambers, and spring and plate reverberators, and then focus where x(n)andy(n) are, respectively, the input and output on tape delay, bucket brigade delays, and the Leslie speaker. signals of the all-pass filter and a is the all-pass filter coefficient. Kleimola and his colleagues [10] combined and 3.1. Efficient Low-Latency Convolution. Bill Putnam, Sr. is expanded further the idea of the signal-adaptive modulation credited with introducing artificial reverberation to record- utilizing all-pass filters. In this coefficient-modulated ing [27]. The method involved placing a loudspeaker and method, the input signal is fed through a chain of many microphone in a specially constructed reverberation cham- identical first-order all-pass filters while the coefficients ber made of acoustically reflective material and having a are modulated at the fundamental frequency of the input shape lacking parallel surfaces. The system is essentially signal. The chain of all-pass filters cascaded with an linear and time invariant and, therefore, characterized by optional equalizing filter, as shown in Figure 2, is called its impulse response. Convolving the input signal with the aspectraldelayfilter[18]. A pitch tracking algorithm or chamber impulse response is a natural choice, as the synthe- low-pass filtered input signal may be used as a modulator, sized system response will be psychoacoustically identical to see Figure 1(d). The modulation of the common all-pass that of the measured space. filter coefficient introduces simultaneously frequency and However, typical room impulse responses have long amplitude modulation effects [10]. decay times and a real-time implementation cannot afford The “brassifier” effect proposed by Cooper and Abel [11] the latency incurred using standard overlap-add processing is another new technique that is closely related to the pre- [28]. Gardner [29]andMcGrath[30] noted that if the vious ones. It has been derived from the nonlinear acoustic impulse responses were divided into two sections, the effect that takes place inside brass musical instruments, when computation would be nearly doubled, but the latency would the sound pressure becomes very large. In the “brassification” be halved. Accordingly, if the impulse response head is algorithm, the input signal is scaled and is used to control recursively divided in two so that the impulse response a fractional delay, which phase modulates the same input section lengths were [L, L,2L,4L,8L, ...], the initial part of signal. It can be seen that the brassification method differs the impulse response would provide the desired low latency, from the self-modulating FM method of Figure 1(a) in while the longer blocks comprising the latter portion of the its computation of the delay modulation and in that a impulse response would be efficiently computed. 4 EURASIP Journal on Advances in Signal Processing

Garcia in [31] noted that processors could efficiently onset duration was given by the time at which the measured implement the needed multiply-accumulate operations, but and FDN impulse responses achieve the same NED. that the addressing involved slowed FFT operations for Regarding controlling the energy profiles of the onset longer block sizes. If a number of blocks were of the same and FDN components during the transition between the length, then they could all share the input signal block two, reference [36] suggests unrolling the loop of the FDN forward transform. For example, if the impulse responses several times so that its impulse response energy onset is were divided into sections of identical block lengths, only one gradual. The convolutional response c(t)isthenwindowed forward transform and only one inverse transform would so that when it is summed with the FDN response d(t), be needed for each block of input signal block processed. the resulting smoothed energy profile matches that of the Garcia showed that dividing the impulse response into a few measured impulse response. While this method is very sections, each of which is divided into equal-length blocks, effective, additional computational and memory resources produces great computational savings while maintaining a are used in unrolling the loop. In [37], a constant-power low latency. cross-fade is achieved by simply subtracting the unwanted Finally, it should be pointed out that an efficient method portion of the FDN response d(t) from the convolutional for performing a low-latency convolution, dividing the response c(t). impulse response into equal-length blocks and using a two- The EMT 140 plate reverberator is a widely used dimensional FFT, was introduced by Hurchalla in [32]. electromechanical reverberator, first introduced in the late 1950s. The EMT 140 consists of a large, thin, resonant 3.2. Hybrid Reverberation Methods. Reverberation impulse plate mounted under tension. A driver near the plate responses contain a set of early arrivals, often including a center produces transverse mechanical disturbances which direct path, followed by a noise-like late field. The late field is propagate over the plate surface, and are detected by pickups characterized by Gaussian noise having an evolving spectrum located toward the plate edges. A damping plate is positioned P(ω, t)[33, 34] near the signal plate and is used to control the low-frequency t decay time (see, e.g., [36]). P(ω, t) = q(ω)2 exp − ,(2) τ(ω) Bilbao [38] and Arcas and Chaigne [39] have explored the physics of plates and have developed finite difference where the square magnitude |q(ω)|2 being the equalization schemes for simulating their motion. However, there are at time t = 0, and τ(ω) defining the decay rate as a function settings in which these schemes are impractical, and for of frequency ω. This late field is reproduced by the feedback real-time implementation as a (linear, time-invariant) rever- delay network (FDN) structure introduced by Jot in the early beration effect, an efficient hybrid reverberator is useful. 1990s [34]. There, a signal is delayed by a set of delay lines Here, the convolutional portion of the hybrid reverberator of incommensurate lengths, filtered according to the desired captures the distinctive whip-like onset of the plate impulse decay times τ(ω), mixed via an orthonormal mixing matrix, response, while the FDN reproduces the late-field decay, and fed back. fixing reverberation time as a function of the damping plate However, when modeling a particular room impulse setting. response, the psychoacoustically important impulse response ffi onset is not preserved. To overcome this di culty, Stewart 3.3. Switched Convolution Reverberator. Both convolutional and Murphy [35] proposed a hybrid structure. A short and delay line-based reverberators have significant memory convolutional section exactly reproduces the reverberation requirements, convolutional reverberators needing a 60 dB onset, while an efficient FDN structure generates the late decay time worth of samples and delay network reverberators field with a computational cost that does not depend on the requiring on the order of a second or two of sample memory. reverberation decay time. Stated mathematically, the hybrid A comb filter structure requires little memory and may reverberator impulse response is the sum of that of the easily be designed to produce a pattern of echos having convolutional section c(t) and that of the FDN section d(t) the desired equalization and decay rate. However, it has a h(t) = c(t) + d(t). (3) constant, small echo density of one arrival per delay line length. This may be improved by adding a convolution The idea is then to adjust c(t)andd(t) so that the system with a noise sequence to the output. The resulting structure impulse response h(t) psychoacoustically approximates the produces the desired echo density and impulse response measured impulse response m(t). The convolutional section spectrogram, and uses little memory—on the order of a few may be taken directly from the measured impulse response tenths of a second. The difficulty is that its output contains onset, and the equalization and decay rates of the FDN an undesired periodicity at the comb filter delay line length. designed to match those of the measured late field response. As proposed in [40] and developed in [41], the periodicity In doing this, however, two issues arise: one having to do with may be reduced significantly by changing or “switching” the duration of the convolutional onset, and the other with the noise filter impulse response over time. Furthermore, by the cross-fade between the convolutional and FDN sections. using velvet noise—a sequence of {+1, 0, −1} samples [40]— A quantity measuring closeness to Gaussian statistics called an efficient time-domain implementation is possible. the normalized echo density (NED) has been shown to A hybrid switched convolution reverberator was devel- predict perceived echo density. In [36], the convolutional oped in [42]forefficiently matching the psychoacoustics of EURASIP Journal on Advances in Signal Processing 5 a measured impulse response. As above, the convolutional or noise in the system. While the feedback loop electronics portion of the system is taken from the impulse response includes a saturating nonlinearity, much of the sonic charac- onset. However, here, the switched convolution reverberator ter of these units arises from the tape transport mechanism, noisesequencesaredrawnfromthemeasuredimpulse which produces both quasiperiodic and stochastic compo- response itself. In this way, the character of the measured late nents, as described in [55, 56]. Finally, it should be pointed field is retained. out that the Echoplex uses a moveable record head to control the delay. The record head is easily moved faster than the tape speed, resulting in a “sonic boom”. In [55], an interpolated 3.4. Spring Reverberators. Springs have been long used to write using a time-varying FIR antialiasing filter is proposed delay and reverberate audio-bandwidth signals. Hammond to prevent aliasing of this infinite-bandwidth event. introduced the spring reverberator in the late 1930s to Bucket brigade delay lines have been widely used in enhance the sound of his electronic organs [43], and, chorus and delay processors since the 1970s. A sample and since the 1960s with the introduction of torsionally driven hold was used with a network of switched capacitors to tensioned springs [44], they have been a staple of guitar delay an input signal according to an externally applied amps. clock. However, as the charge representing the input signal is Modern spring reverbs consist of one or more springs transferred from one capacitor to the next, a small amount held under tension, and they are driven and picked up bleeds to adjacent capacitors, and the output acquires a torsionally from the spring ends. Spring mechanical distur- mild low-pass characteristic. In addition, while the charge is bances propagate dispersively, and the primary torsional and propagating through the delay line, it decays to the substrate. longitudinal modes propagate low frequencies faster than In this way, louder signals are distorted. A physical model of high ones. Bilbao and Parker [45] have developed finite the device is presented in [57]. difference methods based on Wittrick’s treatment of helical coils [46], generating accurate simulations. An efficient approximation, using the dispersive filter design method 4. Tube Amplifier Emulation described in [47]ispresentedin[48]. There, a bidirectional waveguide implements the attenuation and dispersion seen Digital emulation of tube amplifiers has become an active by torsional waves travelling along the spring. A similar area of commercial and academic interest during the last structure was used in [49] to model wave propagation along a fifteen years [58]. The main goal in tube emulation is to Slinky. In addition, an FDN structure was proposed in which produce flexible and realistic guitar amplifier simulation each delay line was made dispersive. algorithms, which faithfully reproduce the sonic characteristics and parametric control of real vintage and modern This model does not include the noise-like “wash” guitar amplifiers and effects. Furthermore, these digital component of the impulse response, which may be the result models should be computationally simple enough so that of spring imperfections. In [50], an efficient waveguide-type several instances could be run simultaneously in real-time. model is described in which a varying delay generates the A recent review article [58] made an extensive survey desired “wash.” Additionally, a simple, noniterative design of the existing digital tube amplifier emulation methods. of high-order dispersive filters based on spectral delay filters The objective of the present section is to summarize the was proposed in [50]. emulation approaches published after the aforementioned review. 3.5. Delay Effects. The Leslie speaker, a rotating horn housed in a small cabinet [51–54], was often paired with a Ham- 4.1. Custom Nonlinear Solvers. Macak and Schimmel [59] mond B3 organ. As the horn rotates, the positions of the simulate the diode limiter circuit, commonly found in direct path and reflections change, resulting in a varying many guitar distortion effects. They start with devising a timbre and spreading of the spectral components, due to first-order IIR filter according to the linearized equivalent Doppler shifts. Approaches to emulating the Leslie include circuit, after which the nonlinear effects are introduced by separately modeling each arrival with an interpolated write allowing the variation of the filter coefficients. The implicit according to the horn’s varying position, and a biquad nonlinear relation between the filter coefficients and its representing the horn radiation pattern [52]. In another output is tackled using two alternative approaches. In the first recent approach [54], impulse responses are tabulated as a approach, an additional unit delay is inserted into the system function of horn rotation angle. As the horn rotates, a time- by evaluating the filter coefficients using the filter output varying FIR filter is applied to the input, with each filter tap at the previous sample. Obviously, this creates a significant drawn from a different table entry according to the horn’s error when the signal is changing rapidly, as can happen evolving rotational state. Rotation rates well into the audio at high input levels, resulting in saturation. Thus, the first bands were produced. approach needs a high sampling rate to perform correctly, so Tape delays, including the Maestro Echoplex and Roland that the signal value and system states do not change much SpaceEcho, are particularly challenging to model digitally. between successive samples. The second approach is to solve Their signal flow is simple and includes a delay and feedback. the implicit nonlinearity using the Newton-Raphson method The feedback is often set to a value greater than one, causing and use the previous filter output only as an initial estimate the unit to oscillate, repeatedly amplifying the applied input for the solver. Additionally, a nonlinear function is added for 6 EURASIP Journal on Advances in Signal Processing

Stage 2 avoid certain computability problems associated with the K Stage 1 Input (load) method. The nonlinear equations are solved during run- time using Newton-Raphson iteration. In practice, with the Stage 3 Stage 2 (load) DK method, the designer can obtain a digital model of a circuit simply by writing its netlist—a well-known textual Stage 3 To the rest representation of the circuit topology—and feeding it to the of the circuitry model generator. Figure 3: The preamplifier structure used in [60]. The interstage Interestingly, the DK method allows the separation of loading effects are approximated by simulating a pair of amplifier the nonlinearity from the memory elements, removing the stages together and reading the output in between them. Thus, the need for run-time iteration and thus allowing an efficient latter stage of this pair acts simply as a load for the first stage and real-time implementation using look-up tables. However, does not produce output. for the memory separation to work properly, the circuit parameters should be held fixed during the simulation, saturating the estimate in order to accelerate the convergence thus disabling run-time control of the knobs on the virtual of the iteration. system. Alternatively, control parameter variations can be In a later article [60], Macak and Schimmel introduce incorporated into the static nonlinearity by increasing its an ordinary differential equation- (ODE-) based real-time dimension, which makes the look-up table interpolation model of an entire guitar amplifier. Although some parts more challenging. of the amplifier are clearly oversimplified (ideal output A variation to the DK method has been introduced by transformer, constant power amplifier load, ideal impedance Dempwolf et al. [67]. In their paper, the system equations are derived manually, leading to more compact matrix repre- divider as the cathode follower), it is one of the most ff complete real-time amplifier models published in academic sentations. Also the discretization procedure is di erent. As works. The ODEs for the tube stages are discretized using the a result, the method described in [67] is computationally less backwards Euler method, and the implicit nonlinearities are expensive than the DK method, but the model generation approximated using the present input value and the previous cannot be automated. The Marshall JCM900 preamp circuit state. The individual tube nonlinearities are modeled using is used as a case study in [67], and the simulation results show Koren’s equations [61], and the tone stack is implemented as a good graphical and sonic match to measured data. reported in [62]. The algorithm is reportedly implemented Another state-space representation for the 12AX7 triode ´ as a VST-plugin. stage is proposed by Cohen and Helie [68], along with a comparison of the traditional static model and a novel The correct modeling of the mutual coupling between dynamic model for the triode tube. In particular, Koren’s amplifier stages is important for realistic emulation, but ff ffi ffi static tube model [61] is augmented by adding the e ect of e cient real-time simulation of this is a di cult task. On stray capacitance between the plate and the grid, a source of the one hand, a full circuit simulation of the amplifier the Miller effect in the amplifier circuit. An implicit numeri- circuitry provides a very accurate, although computationally ffi cal integration scheme is used for ensuring convergence, and ine cient model. On the other hand, a block-based cascade the algorithm is solved using the Newton-Raphson method. model with unidirectional signal flow can be implemented The preamplifier model has been implemented as a real-time very efficiently, but is incapable of modeling the coupling ff VST-plugin. A single-ended guitar power amplifier model e ects. using a linear output transformer has been reported in [69]. An interesting hybrid approach has been used in [60], The pentode tube is simulated using Koren’s equations [61], where the mutual coupling between the preamp triode stages and the same state-space approach as in [68]ischosenfor is simulated by considering each pair of cascaded stages modeling. Also in [69], the simulation is implemented in separately. For example, the output of stage 1 is obtained by real-time as a VST-plugin. simulating the cascaded stages 1 and 2 together, while the output is read in between the stages, as illustrated in Figure 3. Thus, the output of stage 2 is not used at this point, and 4.3. Wave-Digital Filter Models. The usability of wave digital the stage 2 is only acting as a load for stage 1. The output filters (WDFs) in the virtual analog context is discussed by of stage 2 is similarly obtained by simulating the cascade De Sanctis and Sarti in [70]. Importantly from the viewpoint of stages 2 and 3 and reading the output between them. of amp emulation, different strategies for coping with Interestingly, a similar modeling approach has been used in multiple nonlinearities and global feedback are reviewed. a recent commercial amplifier emulator [63]. Traditionally, implementing a circuit with multiple nonlinear elements using WDFs requires special care. In [70], it is 4.2. State-Space Models. A promising state-space modeling suggested that the part of the circuit containing multiple technique for real-time nonlinear circuit simulation, the DK nonlinearities would be implemented as a single multiport method, has been presented by Yeh and colleagues [64, 65]. nonlinearity, and the computability issues would be dealt It is based on the K method [66] introduced by Borin and inside this multiport block, for example using iterative tech- others in 2000, and augments it by automating the model niques. This would essentially sacrifice some of the modular- creation. Furthermore, the DK method discretizes the state ity of the WDF representation for easier computability. The elements prior to solving the system equations in order to consolidation of linear and time-invariant WDF elements EURASIP Journal on Advances in Signal Processing 7 as larger blocks for increasing computational efficiency is The tool has five built-in analysis functions for measuring suggested already in an earlier work by Karjalainen [71]. different aspects of nonlinear distortion, including the A new WDF model of a triode stage has been introduced exponential sweep and dynamic intermodulation distortion in [72]. In contrast to the previous WDF triode stage [73], analysis [84], and additional user-defined analysis techniques this enhanced real-time model is capable of also simulating can be added using Matlab. The software is freely available the triode grid current, thus enabling the emulation of phe- at http://www.acoustics.hut.fi/publications/papers/DATK. nomena such as interstage coupling and blocking distortion Finally, the use of nonlinear signal shaping algorithms [74]. The plate-to-cathode connection is simulated using has also been re-evaluated in view of modern analysis and a nonlinear resistor implementing Koren’s equations [61], modeling methods in [85]. Here the technique of phaseshap- while the grid-to-cathode connection is modeled with a tube ing is studied as an alternative to the more traditional non- diode model. The implicit nonlinearities are solved by the linear waveshaping algorithms. Employing a recent spectral insertion of unit delays, so that there is no need for run- analysis method, the Complex Spectral Phase Evolution time iteration. Although the artificial delays theoretically (CSPE) algorithm, the distortion characteristics of overdrive compromise the modeling accuracy and model stability, in effects are analyzed [86] and polynomial descriptions of practice the simulation results show an excellent fit to SPICE phase and wave shaping functions are obtained from phase simulations, and instability issues are not encountered for and amplitude harmonic data. The method outlined in that realistic signal levels. work is capable of reproducing distortion effects both in The output chain of a tube amplifier, consisting of a terms of their spectrum and waveform outputs. single-ended-triode power amplifier stage, output transformer and a loudspeaker, is modeled using WDFs in [75]. The power amplifier stage uses the same triode model 5. Digital Voltage-Controlled Filters as in [72], thus allowing the simulation of the coupling between the power amp and loudspeaker. Linear equivalent The voltage-controlled filter (VCF) is a famous paradigm in circuits are devised for the transformer and loudspeaker, real-time sound processing. Not only has it been recognized and the component values are obtained from datasheets and as a milestone in the history of electronic music, but in electrical measurements. The simulation is implemented as an attempt to reformulate the challenging solutions in its a computationally efficient fully parametric real-time model architecture in the digital domain, the various discrete-time using the BlockCompiler software [76], developed by Matti models that were proposed in the last fifteen years to simulate Karjalainen. the VCF have given rise to a curious thread of interesting realizations. Developed originally by Moog [87], the VCF is composed 4.4. Distortion Analysis. Since tube amplifier emulators are of an RC ladder whose resistive part is formed by four tran- designed to mimic the sonic properties of real amplifiers ff ff sistor pairs in a di erential configuration. These transistors and e ects units, it is important for the system designer are kept forward biased by a current source, which sets the to be able to carefully measure and analyze the distortion cutoff frequency of the filter. The ladder’s output is fed back behavior of real tube circuits. Although comparisons are to its input via a high-impedance amplifier in a way that typically done by subjective listening, objective methods generates, in the cutoff region, oscillations whose amplitude for distortion analysis in tube amp emulation context have and persistence depend on a variable resistance that controls recently been reported [77–80]. In [77], the parameter the amount of feedback. In the limit of maximum feedback, variations on a highly simplified unidirectional model of the VCF becomes an oscillator ringing at the cutoff frequency a tube amp with static nonlinearities were studied using irrespective of the input. the exponential sweep analysis [81, 82]. In particular, the Both the bias current and the variable resistance are shape of the static nonlinear curves and filter magnitude user controls in Moog synthesizers, the former provided by ff responses were individually varied, and the resulting e ects DC signal generators and low-frequency oscillators, as well on the output spectra with up to nine harmonic distortion as by external signal sources, the latter by simply varying components were analyzed. the resistance through a knob. Sometimes musicians have In [79, 80, 83], Novak´ and colleagues use the exponential controlled the filter behavior by feeding musical signals of sweep analysis in creating nonlinear polynomial models of sufficient amplitude that the bias current is affected and the audio devices. More specifically, the nonlinear model, called cutoff is varied in a complex interplay between synthesis and the generalized polynomial Hammerstein structure, consists control. An analogous effect is produced when the injected of a set of parallel branches with a power function and a currents contain frequency components that are high enough linear filter for each harmonic component. In [79], an audio to reach the filter output. limiter effect is simulated, while two overdrive effects pedals Finally, the VCF response is affected by the input are analyzed and simulated in [80]. Reference [83]modelsan amplitude due to the many solid-state components in the overdrive pedal using a parallel Chebyshev polynomial and circuitry. Large amplitude signals are in fact distorted by filtering structure. the transistors, giving rise to the characteristic nonlinear In [78], a software tool for distortion analysis is behavior of this filter. A similar, but not identical, behavior presented and a VOX AC30 guitar amplifier together with was exhibited by a VCF clone on board the EMS synthesizers, two commercial simulations are analyzed and compared. employing diodes instead of transistors in the RC ladder [89]. 8 EURASIP Journal on Advances in Signal Processing

In conclusion, the VCF has compelling ingredients the nonlinear components, such as transistors or diodes, are that make its simulation in the discrete-time especially modeled, the simulation can be developed starting from a interesting: (i) nonlinear behavior and (ii) two-dimensional plethora of VCF circuit approximations. The final choice continuous control, exerted by both parameter changes (i.e., often ends up on a mathematically tractable subset of such the variable resistance governing the oscillatory behavior) components, each modeled with its own degree of detail, and control signals (the bias current setting the cutoff point). allowing to establish a nonlinear differential state-space As a result, it established a paradigm in virtual analog representation for the whole system. modeling. Furthermore, different techniques exist to solve the nonlinear differential problem. Concerning the VCF, two 5.1. Linear Digital VCFs. Even the reproduction of the VCFs fundamental paradigms have been followed: the functional linear behavior has to deal with the two-dimensional control paradigm, relying on Volterra expansions of the nonlineari- space and its effects in the output. The problem can be ties, and the circuit-driven paradigm, based on the algebraic further simplified by collapsing the current-based control solution of the nonlinear circuit. Both such paradigms yield mechanism into a scalar parameter of cutoff frequency. Such solutions that must be integrated numerically. By solving simplifications lead to the following transfer function: simplified versions of the VCF in a different way, both of them are prone to various types of inaccuracies. {G s }4 H s = ( ) = 1 Huovilainen, who chose to use a circuit-driven approach ( ) 4 4 ,(4) 1+k{G(s)} k + {1+s/ωc} [93], was probably the first to attempt a nonlinear solution of the VCF. He derived an accurate model of the transistor- in which frequency variable ωc sets the cutoff frequency basedRCladderaswellasofthefeedbackcircuit.Onthe and feedback gain k determines the oscillatory behavior other hand, while proposing a numerical solution to this (i.e., resonance). The function G(s) = ωc/(ωc + s)models model, he kept a fictitious unit delay in the resulting digital every single step of the ladder. Figure 4 shows, in dashed structure to avoid costs and complications of an implicit lines, typical magnitude responses of the analog Moog VCF procedure for the feedback loop computation. The extra unit obtained by plotting |H(jω)| in audio frequency as by (4) delay in the feedback loop creates an error in parameter and, in solid lines, spectra of discrete-time impulse responses accuracy, which increases with frequency. Huovilainen then after bilinear transformation of H(s)intoH(z) at 44.1 kHz, uses low-order polynomial correction functions for both the respectively, for gains k equalto1,2,3,and4.Allresponses cut-off frequency and the resonance parameter, thus still have been plotted for cutoff frequencies fc = ωc/(2π)equal reaching a desired accuracy [92]. to 0.1, 1, and 10 kHz [88]. Figure 6 shows a simplified version of Huovilainen’s Stilson and Smith, in their pioneering approach to the nonlinear digital Moog filter, in which only one nonlinear linear modeling of the VCF [90], showed that the accurate function is used [92]. Huovilainen’s full Moog ladder model real-time computation of (4) in discrete time is problematic. contains five such functions: one for each first-order section In fact, k and ωc merge into a bidimensional nonlinear and one for the feedback circuit. A hyperbolic tangent map while moving to the digital domain. On the other is used as the nonlinear function in [93]. In a real-time hand, approximations of H(s) aiming at maintaining such implementation, this would usually be implemented using parameters decoupled in the discrete-time domain lead to a look-up table. Alternatively, another similar smoothly inaccurate responses as well as to mismatches of the cut-off saturating function, such as a third-order polynomial, can frequency and persistence of the oscillations compared to the be used instead. Huovilainen’s nonlinear Moog filter self- analog case. oscillates, when the feedback gain is set to a value of one A step ahead in the linear VCF modeling has been or larger. The saturating nonlinear function ensures that achieved by Fontana, who directly computed the delay- the system cannot become unstable, because signal values free loop VCF structure arising from (4) and illustrated cannot grow freely. The simplified model of Figure 6 behaves in Figure 5 for convenience [88]. By employing a specific essentially in the same manner as the full model, but small procedure for the computation of delay-free loop filter differences in their output signals can be observed. It remains networks [91], the couple (ωc, k) in fact could be preserved in as an interesting future study to test with which input signals the discrete-time domain without mixing the two parameters and parameter setting their minor differences could be best together. In practice, this procedure allows to serialize the heard. computation of the four identical transfer functions G(z) The Volterra approach was proposed by Helie´ in 2006 obtained by the bilinear transformation of G(s), indepen- [95]. This approach requires a particular ability to manip- dently of the feedback gain k. Three look-up tables and a few ulate Volterra kernels and to manage their instability in multiplications and additions are needed to obtain the feed- presence of heavy distortion. Indeed, high distortion levels back signal and the state variable values for each sampling can be generated by the VCF when fed large amplitude inputs interval. This way, an accurate response, an independent and and for high values of k, that is, when the filter is set to continuous parametric control, and real-time operation are operate like a selective resonator or like an oscillator. In all achieved at a fairly low computational cost. a more recent development proposed by the same author [96], sophisticated ad-hoc adaptations of the Volterra kernels 5.2. Nonlinear Digital VCFs. The introduction of nonlin- were set in an aim to model the transistors’ saturation on a earities complicates the problem to a large extent. When sufficiently large amplitude range. EURASIP Journal on Advances in Signal Processing 9

k = 1 k = 2 40 40 30 30 20 20 10 10 0 0 −10 −10 −20 −20 Magnitude (dB) Magnitude (dB) −30 −30 −40 −40 −50 −50 −60 −60 101 102 103 104 101 102 103 104 Frequency (Hz) Frequency (Hz) (a) (b) k = 3 k = 4 40 40 30 30 20 20 10 10 0 0 −10 −10 −20 −20 Magnitude (dB) Magnitude (dB) −30 −30 −40 −40 −50 −50 −60 −60 101 102 103 104 101 102 103 104 Frequency (Hz) Frequency (Hz) (c) (d)

Figure 4: Magnitude responses of the analog linear Moog VCF (dashed line) and its digital version obtained by bilinear transformation at 44.1 kHz (solid line). Cut-oﬀ frequencies equal to 0.1, 1, and 10 kHz are plotted in each diagram [88].

+ + x n G z y n x(t) G(s) G(s) G(s) G(s) y(t) ( ) Nonlinearity G(z) ( ) G(z) G(z) ( ) − − Fictitious delay k k z−1

Figure 5: Delay-free loop filter structure of the VCF. Figure 6: A simplified version of Huovilainen’s nonlinear digital Moog filter [92].

In 2008, Civolani and Fontana devised a nonlinear state- any analog-to-digital transformation preserving passivity space representation of the diode-based EMS VCF out of [94]. The delay-free loops in the resulting digital network an Ebers-Moll approximation of the driving transistors [97]. were finally computed by repeatedly circulating the signal This representation could be computed in real time by means along the loop until convergence, in practice implementing of a fourth-order Runge-Kutta numerical integration of the a fixed-point numerical scheme. nonlinear system. The model was later reformulated in terms Figure 7 provides examples of responses computed by of a passive nonlinear analog filter network, which can readily the EMS VCF model when fed a large amplitude impulsive be turned into a passive discrete-time filter network through input [94]. On the left, the impulse responses for increasing 10 EURASIP Journal on Advances in Signal Processing

80 80

60 60

40 40

20 20

0 0

−20 −20 Magnitude (dB) Magnitude (dB) −40 −40

−60 −60

−80 −80 101 102 103 104 101 102 103 104 Frequency (Hz) Frequency (Hz) (a) (b)

Figure 7: Magnitude responses of the EMS VCF model for a 1 V impulsive input and cutoff frequency set to 0.1, 1, and 10 kHz. (a) k = 0 (bold), 8 (thin solid). (b) k = 11. Sampling frequency set at 176.4 kHz [94]. values of k are illustrated at cut-off frequencies equal to Moreover, this generalization may provide a powerful 0.1, 1, and 10 kHz. On the right, the system behavior is control for musicians who appreciate the effects illustrated with the same cut-off frequencies and a very high resulting from this interplay. feedback gain. Comparison with Figure 4 helps to appreciate (ii) Although designed to have low or no interference the contribution of the distortion components to the output, with the RC ladder, the feedback circuit has a non- as well as their amount for changing values of the feedback negligible coupling effect with the feedforward chain. gain parameter. Also for reasons that are briefly explained at As we could directly assess on a diode-based VCF the end of this section, Figure 7 does not include magnitude on board an EMS synthesizer during a systematic spectra of output signals measured on a real EMS VCF, due measurement session, the leaks affecting the feedback to the gap that still exists between the virtual analog model amplifier in fact give rise to responses that are and the reference filter. often quite far from the “ideal” VCF behavior. Even when the feedback gain is set to zero, this 5.3. Current Issues. The current Java implementation for circuit exhibits a nonnull current absorption that the PureData real-time environment [98] of the aforemen- changes the otherwise stable fourth-order low-pass tioned delay-free loop filter network, obtained by bilinear RC characteristics. Techniques aiming at improving transformation of the state-space representation of the EMS the accuracy of the feedback amplifier would require analog circuit [94], represents a highly sophisticated non- to individually model at least some of its transistors, commercial realization of a VCF software architecture in with consequences on the model complexity and terms of accuracy, moderate computational requirement, computation time that cannot be predicted at the continuous controllability of both ωc and k,andinter- moment. operability under all operating systems capable of running PureData and its pdj libraries communicating with the Java The next generation of virtual analog VCFs may provide Virtual Machine. In spite of all these desirable properties, the answers to the above open issues. implementation leaves several issues open. Especially some among such issues ask for a better 6. Synthesis and Processing Languages understanding and consequent design of the digital filter. Languages for synthesis and processing of musical signals (i) The bias current has been modeled so far in terms have been central to research and artistic activities in com- of a (concentrated) cut-off frequency parameter. As puter music since the late 1950s. The earliest digital sound it has been previously explained, the analog VCF synthesis system for general-purpose computers is MUSIC cutoff is instead biased by a current signal that flows I by Max Mathews (1959), a very rudimentary program across the filter together with the musical signal. The written for the IBM 704 computer [99], capable of generating subtle, but audible nuances resulting from the con- a single triangular-shaped waveform. This was followed in tinuous interplay between such two signals, can be quick succession by MUSIC II and III, introducing some reproduced only by substituting the cut-off frequency of the typical programming structures found in all of parameter in the state-space representation with one today’s systems, such as the table look-up oscillator [100]. more system variable, accounting for the bias current. These developments culminated in MUSIC IV (1963), which EURASIP Journal on Advances in Signal Processing 11 provided many of the elements that are found in modern types of messages with given methods. Messages are passed systems. One of the most important ideas introduced by this through wire connections between objects. For audio, a program was the concept of modular synthesis components, special type of UG is required, which will allow for audio which can be used to construct instruments for computer input and/or output connections and also provide a method music performance defined in a score code. In particular, the for a DSP message. This enables the object to register principle of the unit generator (UG), on which all modern its processing routine with the systems audio processing systems are based, was introduced in this system. UGs are scheduler, so that it is included in the DSP graph. As with the building blocks of synthesis environments, implementing SC3, UGs can be added to PD as dynamic modules that the basic operations that are responsible for digital audio are either loaded at startup or, in certain cases, on demand. generation. Given this relatively simple means of language extension, Another major step in the development of languages for PD has also been adopted as system for the implementation synthesis and processing was the adoption of portable high- and demonstration of new algorithms, as for instance in level programming languages for their implementation. An [108]. Finally, we should note that PD has a closed-source, early example of this was seen in MUSIC IVBF, a version commercially available, equivalent alternative, MAX/MSP of MUSIC IV written in FORTRAN at Princeton University [109]. in 1964. But it is Mathews’ MUSIC V [101], also based on Of these three systems, Csound is the longest in existence, FORTRAN, that occupies a central place in the development having been developed in 1985 and released publicly in of computer music for its widespread adoption, providing a 1996. It has, however, been fundamentally modified for model on which modern systems would be based. version 5, first released in 2006, which has brought its code Modern descendants from these systems include Csound base up-to-date with more recent programming practices. 5[102], Pure Data (PD) [103], and SuperCollider 3 (SC3) Effectively, Csound is a programming library, which can [104] as the most established and commonly used open- be used from various languages, for example, C/C++, Java, source systems. Other currently supported languages include Python, Lua, Tcl, and so forth. As a synthesis system, it Nyquist [105], also used for plugin scripting in the Audacity provides a text language for the creation of instruments from sound editor, and PWGL [106] which supports a com- UGs and a compiler to generate DSP graphs from these. prehensive computer-aided composition system. Most of It has also a separate basic score language, which can be these environments are designed for real-time applications, substituted or combined with other means of instrument although Csound was originally written as an offline pro- instantiation. A number of frontends to the system exist, cessing program and indeed can still be used that way. allowing it to be used in different contexts and applications. Processing is generally done in a block-by-block basis, and For signal processing research, it allows prototyping of new UGs are provided in an object-oriented fashion. Csound algorithms (e.g., filters and synthesis equations) in a simple and SC3 provide two basic rates of processing for control and direct way, as well as sample-by-sample processing. and audio signals, and PD provides a single rate (for audio) Its integration with the Python language is particularly with control signals processed asynchronously by a message- useful as it allows for the graphing and numerical libraries passing mechanism. to be combined in scripts. In addition, for frequency- SC3 is actually based on two components, a program- domain applications, it provides a special signal type that ming language/interpreter, SCLang, and a sound synthesis is used for stream processing. This is a very useful feature engine, SCSynth. They are combined in a client-server for the testing and implementation of real-time spectral structure, with the former issuing Open Sound Control applications. UGs can be added to the system via dynamic (OSC) commands over an IP connection to the server. This loadable libraries, allowing for the implementation of newly- also allows the synthesis engine to be used independently developed algorithms, such as [8]. of the language interpreter, by any software capable of In addition to the MUSIC N-derived systems described providing OSC output. SCLang is an object-oriented lan- above, there is one further language of note. This is FAUST, guage that provides a symbolic representation of the UG a purely functional language for prototyping and implemen- entities residing in the server and allowing the user to create tation of audio processing algorithms [110]. It aims to be an connections between these. The synthesis engine will, on efficient alternative to implementation languages such as C receiving certain OSC messages, construct UG graphs and or C++. FAUST is better described as a signal processing, schedule processing accordingly. New UGs can be added to rather than a music, programming language. It is based the system as dynamic modules, which are loaded by the on the abstraction of a signal processor as a mathematical server on startup. For these to be legal SCLang constructs, function taking inputs and producing outputs. While not they also have to be provided an interface for the language. designed in the same vein, and with the same principles, as SC3 has been used by various research and artistic projects, the ones discussed above, it nevertheless shares their modular such as the ones described in [107]. approach, with structural elements that are analogous to Unlike SC3, PD is a flowchart programming language. UGs. FAUST shares the flowgraph approach that directly It provides a graphical interface that is used to create underpins flowchart languages such as PD (and indirectly, all programs (also known as patches), although these can also be other MUSIC N-derived languages), but provides an explicit created as a plain text script (or indeed programmatically). formal semantic. Also, unlike other systems, it produces Central to its operation is an object-oriented message- C++ code (as opposed to running DSP graphs) for various passing mechanism. UGs are built to respond to particular targets: UGs for SC3, PD (/MaxMSP), Csound, and so 12 EURASIP Journal on Advances in Signal Processing forth; standalone programs with various audio I/O backends; http://www.acoustics.hut.fi/publications/papers/MobileSyn- and plugins of various formats. FAUST was designed with thAndFXReport/MobileSynthAndFXReport.pdf. the aims of allowing rapid translation of algorithms and [7] C. Poepel and R. B. Dannenberg, “Audio signal driven sound flowcharts into functional code and generation of efficient synthesis,” in Proceedings of the International Computer Music C++ code, which is a very useful feature for real-time musical Conference, pp. 391–394, Barcelona, Spain, September 2005. signal processing applications. [8] V. Lazzarini, J. Timoney, and T. Lysaght, “The generation Finally, with the increased availability of multiple pro- of natural-synthetic spectra by means of adaptive frequency cessor systems in general-purpose computers, systems have modulation,” Computer Music Journal, vol. 32, no. 2, pp. 9– been developed to take advantage of these platforms. Two 22, 2008. ff [9] J. Pekonen, “Coefficient modulated first-order allpass filter opposing approaches have been taken, representing di erent ff ideas of how parallelization should be achieved. These are as distortion e ect,” in Proceedings of the International Con- ference on Digital Audio Effects, pp. 83–87, Espoo, Finland, represented typically by, on one side, a new version of SC3 September 2008. (SuperNova) [111] and, on the other, by an experimental [10] J. Kleimola, J. Pekonen, H. Penttinen, V. Valim¨ aki,¨ and version of Csound, ParCS [112] and the OpenMP-based code J. S. Abel, “Sound synthesis using an allpass filter chain output of FAUST [110]. The first approach follows the exist- with audio-rate coefficient modulation,” in Proceedings of the ing implementation of concurency in some programming International Conference on Digital Audio Effects,Como,Italy, languages, such as Occam [113], where the system provides September 2009. constructs for users to parallelize portions of their code. The [11] C. M. Cooper and J. S. Abel, “Digital simulation of brassi- other approach is to let the parser decide automatically how ness and amplitude-dependent propagation speed in wind the parallelization should be done, with the user having little instruments,” in Proceedings of the International Conference or no role in determining it. This theory allows complete on Digital Audio Effects, Graz, Austria, September 2010. backwards compatibility with existing code and a seamless [12] V. Verfaille, U. Zolzer,¨ and D. Arfib, “Adaptive digital audio transition to newer hardware. effects (A-DAFx): a new class of sound transformations,” IEEE Transactions on Audio, Speech and Language Processing, vol. 14, no. 5, pp. 1817–1831, 2006. 7. Conclusion [13] T. Stilson, “General weirdness with the Karplus-Strong string,” in Proceedings of the International Computer Music A selection of recent advances in musical effects processing Conference,Banff, Canada, 1995. and synthesis have been discussed in this paper. In particular, [14] T. I. Laakso, V. Valim¨ aki,¨ M. Karjalainen, and U. K. Laine, the advances in adaptive effects processing algorithms, “Splitting the unit delay: tools for fractional delay filter synthesis, and processing languages, and digital emulation of design,” IEEE Signal Processing Magazine, vol. 13, no. 1, pp. tube amplifiers, voltage-controlled filters, and vintage delay 30–60, 1996. and reverberation effects have been reviewed. [15]V.Lazzarini,J.Timoney,andT.Lysaght,“Asymmetric- spectra methods for adaptive FM synthesis,” in Proceedings of the International Conference on Digital Audio Effects Acknowledgment (DAFx ’08), pp. 233–240, Espoo, Finland, September 2008. [16] V. Lazzarini, J. Timoney, and T. Lysaght, “Nonlinear dis- This work has been funded by the Aalto University and the tortion synthesis using the split-sideband method, with Academy of Finland (Project no. 122815). applications to adaptive signal processing,” Journal of the Audio Engineering Society, vol. 56, no. 9, pp. 684–695, 2008. [17]V.LazzariniandJ.Timoney,“Theoryandpracticeof References modified frequency modulation synthesis,” Journal of the Audio Engineering Society, vol. 58, no. 6, pp. 459–471, 2010. [1] U. Zolzer,¨ Ed, DAFX—Digital Audio Effects,JohnWiley& [18] V.Valim¨ aki,¨ J. S. Abel, and J. O. Smith, “Spectral delay filters,” Sons, New York, NY, USA, 2002. Journal of the Audio Engineering Society,vol.57,no.7-8,pp. [2] J. O. Smith, “Physical Audio Signal Processing,” 2010, https:// 521–531, 2009. ccrma.stanford.edu/∼jos/pasp/. [3] V. Valim¨ aki,¨ F. Fontana, J. O. Smith, and U. Zolzer,¨ “Intro- [19]V.Lazzarini,J.Timoney,J.Pekonen,andV.Valim¨ aki,¨ ff “Adaptive phase distortion synthesis,” in Proceedings of the duction to the special issue on virtual analog audio e ects ff and musical instruments,” IEEE Transactions on Audio, International Conference on Digital Audio E ects,Como,Italy, Speech and Language Processing, vol. 18, no. 4, pp. 713–714, September 2009. 2010. [20] V. Valim¨ aki,¨ T. Tolonen, and M. Karjalainen, “Signal- [4] V. Valim¨ aki,¨ J. Pakarinen, C. Erkut, and M. Karjalainen, dependent nonlinearities for physical models using time- “Discrete-time modelling of musical instruments,” Reports varying fractional delay filters,” in Proceedings of the Interna- on Progress in Physics, vol. 69, no. 1, pp. 1–78, 2006. tional Computer Music Conference, pp. 264–267, Ann Arbor, [5] V. Valim¨ aki¨ and A. Huovilainen, “Antialiasing oscillators in Mich, USA, October 1998. subtractive synthesis,” IEEE Signal Processing Magazine,vol. [21] R. Msallam, S. Dequidt, S. Tassart, and R. Causse,´ “Physical 24, no. 2, pp. 116–125, 2007. model of the trombone including nonlinear propagation [6] J. Pakarinen, H. Penttinen, V. Valim¨ aki¨ et al., “Review of effects,” in Proceedings of the International Symposium on sound synthesis and effects processing for interactive mobile Musical Acoustics, vol. 2, pp. 419–424, Edinburgh, UK, 1997. applications,” Report 8, Department of Signal Processing [22] S. Tassart, R. Msallam, P. Depalle, and S. Dequidt, “A and Acoustics, Helsinki University of Technology, 2009, fractional delay application: time-varying propagation speed EURASIP Journal on Advances in Signal Processing 13

in waveguides,” in Proceedings of the International Com- [41] K.-S. Lee, J. S. Abel, V. Valim¨ aki,¨ and D. P. Berners, “The puter Music Conference, pp. 256–259, Thessaloniki, Greece, switched convolution reverberator,” in Proceedings of the September 1997. 127th AES Convention, New York, NY, USA, October 2009, [23] R. Msallam, S. Dequidt, R. Causse,´ and S. Tassart, “Physical preprint 7927. model of the trombone including nonlinear effects. Appli- [42]K.-S.Lee,N.J.Bryan,andJ.S.Abel,“Approximating cation to the sound synthesis of loud tones,” Acta Acustica measured reverberation using a hybrid fixed/switched con- united with Acustica, vol. 86, no. 4, pp. 725–736, 2000. volution structure,” in Proceedings of the 13th International [24] T. Tolonen, V. Valim¨ aki,¨ and M. Karjalainen, “Modeling of Conference on Digital Audio Effects (DAFx ’10), Graz, Austria, tension modulation nonlinearity in plucked strings,” IEEE September 2010. Transactions on Speech and Audio Processing,vol.8,no.3,pp. [43] L. Hammond, “Electrical musical instrument,” US patent no. 300–310, 2000. 2230836, February 1941. [25] J. Pakarinen, V. Valim¨ aki,¨ and M. Karjalainen, “Physics- [44] A. C. Young, “Artificial reverberation unit,” US patent no. based methods for modeling nonlinear vibrating strings,” 3106610, October 1963. Acta Acustica United with Acustica, vol. 91, no. 2, pp. 312– [45] S. Bilbao and J. Parker, “A virtual model of spring rever- 325, 2005. beration,” IEEE Transactions on Audio, Speech and Language [26] J. R. Pierce and S. A. van Duyne, “A passive nonlinear digital Processing, vol. 18, no. 4, pp. 799–808, 2010. filter design which facilitates physics-based sound synthesis of highly nonlinear musical instruments,” Journal of the [46] W. H. Wittrick, “On elastic wave propagation in helical Acoustical Society of America, vol. 101, no. 2, pp. 1120–1126, springs,” International Journal of Mechanical Sciences,vol.8, 1997. no. 1, pp. 25–47, 1966. [27] “Bill Putnam,” 2011, http://en.wikipedia.org/wiki/Bill Put- [47]J.S.AbelandJ.O.Smith,“Robustdesignofveryhigh-order nam. dispersive allpass filters,” in Proceedings of the International ff [28] A. V. Oppenheim and R. W. Schafer, Discrete-Time Signal Conference on Digital Audio E ects (DAFx ’06), pp. 13–18, Processing, Pearson Prentice Hall, Upper Saddle River, NJ, Montreal, Canada, September 2006. USA, 3rd edition, 2010. [48] J. S. Abel, D. P. Berners, S. Costello, and J. O. Smith, [29] W. G. Gardner, “Efficient convolution without input-output “Spring reverb emulation using dispersive allpass filters in delay,” Journal of the Audio Engineering Society, vol. 43, no. 3, a waveguide structure,” Journal of the Audio Engineering pp. 127–136, 1995. Society, vol. 54, p. 1277, 2006, presented at the 121th [30] D. S. McGrath, “Method and apparatus for filtering an elec- Convention of the Audio Engineering Society. tronic environment with improved accuracy and efficiency [49] J. Parker, H. Penttinen, S. Bilbao, and J. S. Abel, “Modeling and short flow-through delay,” US patent no. 5502747, Lake methods for the highly dispersive slinky spring: a novel musi- Technology Ltd., March 1996. cal toy,” in Proceedings of the 13th International Conference [31] G. Garcia, “Optimal filter partition for efficient convolution on Digital Audio Effects (DAFx ’10), Graz, Austria, September with short input/output delay,” in Proceedings of the 113th 2010. AES Convention, October 2002, paper no. 5660. [50] V. Valim¨ aki,¨ J. Parker, and J. S. Abel, “Parametric spring [32] J. Hurchalla, “Low latency convolution in one dimension reverberation effect,” Journal of the Audio Engineering Society, via two dimensional convolutions: an intuitive approach,” in vol. 58, no. 7-8, pp. 547–562, 2010. Proceedings of the 125th AES Convention, October 2008, paper [51] D. Leslie, “Rotatable tremulant sound,” US patent 2489653, no. 7634. 1949. [33] M. R. Schroeder, “Natural sounding reverberation,” Journal [52] J. Smith, S. Serafin, J. Abel, and D. Berners, “Doppler simu- of the Audio Engineering Society, vol. 10, pp. 219–223, 1962. lation and the Leslie,” in Proceedings of the 5th International [34]J.-M.JotandA.Chaigne,“Digitaldelaynetworksfor Conference on Digital Audio Effects (DAFx ’02),Hamburg, designing artificial reverberators,” in Proceedings of the 90th Germany, September 2002. AES Convention, Paris, France, 1991, preprint 3030. [53] R. Kronland-Martinet and T. Voinier, “Real-time perceptual [35] R. Stewart and D. Murphy, “A hybrid artificial reverberation simulation of moving sources: application to the Leslie algorithm,” in Proceedings of the 122th AES Convention,May Cabinet and 3D sound immersion,” EURASIP Journal on 2007, paper no. 7021. Audio, Speech, and Music Processing, vol. 2008, Article ID [36] J. S. Abel, D. P. Berners, and A. Greenblatt, “An emulation of 849696, 10 pages, 2008. the EMT 140 plate reverberator using a hybrid reverberator structure,” in Proceedings of the 127th AES Convention,New [54] J. Herrera, C. Hanson, and J. S. Abel, “Discrete time York, NY, USA, October 2009, paper no. 7928. emulation of the Leslie speaker,” in Proceedings of the 127th [37] A. Greenblatt, J. Abel, and D. Berners, “A hybrid reverber- AES Convention, New York, NY, USA, October 2009, paper ation crossfading technique,” in Proceedings of the Interna- no. 7925. tional Conference on Acoustics, Speech, and Signal Processing [55] S. Arnardottir, J. S. Abel, and J. O. Smith, “A digital model (ICASSP ’10), pp. 429–432, Dallas, Tex, USA, 2010. of the Echoplex tape delay,” in Proceedings of the 125th AES [38] S. Bilbao, “A digital plate reverberation algorithm,” Journal Convention, San Francisco, Calif, USA, October 2008. of the Audio Engineering Society, vol. 55, no. 3, pp. 135–144, [56] V.Valim¨ aki,S.Bilbao,J.O.Smith,J.S.Abel,J.Pakarinen,and¨ 2007. D. Berners, “Virtual analog effects,” in DAFX: Digital Audio [39] K. Arcas and A. Chaigne, “On the quality of plate reverbera- Effects,U.Zolzer,¨ Ed., chapter 12, John Wiley & Sons, New tion,” Applied Acoustics, vol. 71, no. 2, pp. 147–156, 2010. York, NY, USA, 2nd edition, 2011. [40] M. Karjalainen and H. Jarvel¨ ainen,¨ “Reverberation modeling [57] C. Raffel and J. O. Smith, “Practical modeling of bucket- using velvet noise,” in Proceedings of the 30th International brigade device circuits,” in Proceedings of the 13th Interna- Conference on Intelligent Audio Environments (AES ’07), tional Conference on Digital Audio Effects (DAFx ’10),Graz, Saariselka,¨ Finland, March 2007. Austria, September 2010. 14 EURASIP Journal on Advances in Signal Processing

[58] J. Pakarinen and D. T. Yeh, “A review of digital techniques for [75] J. Pakarinen, M. Tikander, and M. Karjalainen, “Wave digital modeling vacuum-tube guitar amplifiers,” Computer Music modeling of the output chain of a vacuum-tube amplifier,” Journal, vol. 33, no. 2, pp. 85–100, 2009. in Proceedings of the 12th International Conference on Digital [59] J. Macak and J. Schimmel, “Nonlinear circuit simulation Audio Effects (DAFx ’09), Como, Italy, September 2009. using time-variant filter,” in Proceedings of the International [76] M. Karjalainen, “BlockCompiler: efficient simulation of Conference on Digital Audio Effects,Como,Italy,September acoustic and audio systems,” in Proceedings of the 114th AES 2009. Convention, Amsterdam, The Netherlands, March 2003. [60] J. Macak and J. Schimmel, “Real-time guitar tube amplifier [77]K.Dempwolf,M.Holters,S.Moller,¨ U. Zolzer,¨ and G. Ham- simulation using an approximation of differential equations,” burg, “The influence of small variations in a simplified guitar in Proceedings of the 13th International Conference on Digital amplifier model,” in Proceedings of the 12th International Audio Effects (DAFx ’10), Graz, Austria, September 2010. Conference on Digital Audio Effects (DAFx ’09),Como,Italy, [61] N. Koren, “Improved vacuum-tube models for SPICE simu- September 2009. lations,” Glass Audio, vol. 8, no. 5, pp. 18–27, 1996. [78] J. Pakarinen, “Distortion analysis toolkit—a software tool for [62] D. T. Yeh and J. O. Smith, “Discretization of the ’59 Fender easy analysis of nonlinear audio systems,” EURASIP Journal Bassman tone stack,” in Proceedings of the International on Advances in Signal Processing, vol. 2010, Article ID 617325, Conference on Digital Audio Effects, pp. 1–6, Montreal, 13 pages, 2010. Canada, September 2006. [79] A. Novak,´ L. Simon, F. Kadlec, and P. Lotton, “Nonlinear [63] T. Serafini, personal correspondence, 2009. system identification using exponential swept-sine signal,” [64] D. T. Yeh, Digital implementation of musical distortion circuits IEEE Transactions on Instrumentation and Measurement,vol. by analysis and simulation, Ph.D. thesis, Stanford University, 59, no. 8, pp. 2220–2229, 2009. Palo Alto, Calif, USA, 2001, https://ccrma.stanford.edu/∼ [80] A. Novak,L.Simon,andP.Lotton,“Analysis,synthesis,and´ dtyeh/papers/DavidYehThesissinglesided.pdf. classification of nonlinear systems using synchronized swept- [65] D. T. Yeh, J. S. Abel, and J. O. Smith III, “Automated physical sine method for audio effects,” EURASIP Journal on Advances modeling of nonlinear audio circuits for real-time audio in Signal Processing, vol. 2010, Article ID 793816, 8 pages, effects—part I: theoretical development,” IEEE Transactions 2010. on Audio, Speech and Language Processing,vol.18,no.4,pp. [81] A. Farina, “Simultaneous measurement of impulse response 728–737, 2010. and distortion with a swept-sine technique,” in Proceedings [66] G. Borin, G. De Poli, and D. Rocchesso, “Elimination of the 108th AES Convention, Paris, France, February 2000, of delay-free loops in discrete-time models of nonlinear preprint 5093. acoustic systems,” IEEE Transactions on Speech and Audio [82] J. Abel and D. P. Berners, “A technique for nonlinear system Processing, vol. 8, no. 5, pp. 597–604, 2000. measurement,” in Proceedings of the 121st AES Convention, [67]K.Dempwolf,M.Holters,andU.Zolzer,¨ “Discretization San Francisco, Calif, USA, October 2006. of parametric analog circuits for real-time simulations,” in [83] A. Novak,L.Simon,P.Lotton,andJ.Gilbert,“Chebyshev´ Proceedings of the 13th International Conference on Digital model and synchronized swept sine method in nonlinear Audio Effects (DAFx ’10), Graz, Austria, September 2010. audio effect modeling,” in Proceedings of the 13th Interna- [68] I. Cohen and T. Helie,´ “Simulation of a guitar amplifier tional Conference on Digital Audio Effects (DAFx ’10),Graz, stage for several triode models: examination of some relevant Austria, September 2010. phenomena and choice of adapted numerical schemes,” in [84]E.Leinonen,M.Otala,andJ.Curl,“Amethodformeasuring Proceedings of the 127th AES Convention,NewYork,NY,USA, transient intermodulation distortion (TIM),” Journal of the 2009. Audio Engineering Society, vol. 25, no. 4, pp. 170–177, 1977. [69] I. Cohen and T. Helie,´ “Real-time simulation of a guitar [85] J. Timoney, V. Lazzarini, A. Gibney, and J. Pekonen, “Digital power amplifier,” in Proceedings of the 13th International emulation of distortion effects by wave and phase shaping Conference on Digital Audio Effects (DAFx ’10), Graz, Austria, methods,” in Proceedings of the 13th International Conference September 2010. on Digital Audio Effects (DAFx ’10), pp. 419–422, Graz, [70] G. De Sanctis and A. Sarti, “Virtual analog modeling in the Austria, September 2010. wave-digital domain,” IEEE Transactions on Audio, Speech [86] R. Garcia and K. Short, “Signal analysis using the complex and Language Processing, vol. 18, no. 4, pp. 715–727, 2010. spectral phase evolution (CSPE) algorithm,”in Proceedings of [71] M. Karjalainen, “Efficient realization of wave digital com- the 120th AES Convention, Paris, France, May 2006. ponents for physical modeling and sound synthesis,” IEEE [87] R. Moog, “A voltage-controlled low-pass high-pass filter for Transactions on Audio, Speech and Language Processing,vol. audio signal processing,” in Proceedings of the 17th AES 16, no. 5, pp. 947–956, 2008. Convention, New York, NY, USA, October 1965, preprint 413. [72] J. Pakarinen and M. Karjalainen, “Enhanced wave digital [88] F. Fontana, “Preserving the structure of the Moog VCF in the triode model for real-time tube amplifier emulation,” IEEE digital domain,” in Proceedings of the International Computer Transactions on Audio, Speech and Language Processing,vol. Music Conference, pp. 291–294, Copenhagen, Denmark, 18, no. 4, pp. 738–746, 2010. August 2007. [73] M. Karjalainen and J. Pakarinen, “Wave digital simulation [89] T. E. Stinchcombe, “Analysis of the Moog transistor lad- of a vacuum-tube amplifier,” in Proceedings of the IEEE der and derivative filters,” Tech. Rep., Freeserve, 2008, International Conference on Acoustics, Speech and Signal http://www.timstinchcombe.co.uk/. Processing (ICASSP ’06), vol. 5, pp. 153–156, Toulouse, [90] T. Stilson and J. O. Smith, “Analyzing the Moog VCF with France, May 2006. considerations for digital implementation,” in Proceedings of [74] R. Aiken, “What is blocking distortion?” internet article, the International Computer Music Conference, Hong Kong, 2006, http://www.aikenamps.com/BlockingDistortion.html. August 1996. EURASIP Journal on Advances in Signal Processing 15

[91] F. Fontana and F. Avanzini, “Computation of delay-free non- Paradigms for Computer Music, G. Assayag and A. Gerzso, linear digital filter networks: application to chaotic circuits Eds., Editions Delatour, Paris, France, 2009. and intracellular signal transduction,” IEEE Transactions on [111] T. Blechmann, “Supernova—a multiprocessor-aware synthe- Signal Processing, vol. 56, no. 10, pp. 4703–4715, 2008. sis server for SuperCollider,” in Proceedings of the Linux Audio [92] V. Valim¨ aki¨ and A. Huovilainen, “Oscillator and filter Conference, pp. 141–146, Utrecht, The Netherlands, 2010. algorithms for virtual analog synthesis,” Computer Music [112] J. ffitch, “Parallel execution of Csound,” in Proceedings of the Journal, vol. 30, no. 2, pp. 19–31, 2006. International Computer Music Conference,Montreal,Canada, [93] A. Huovilainen, “Nonlinear digital implementation of the 2009. Moog ladder filter,” in Proceedings of the International [113] M. Elizabeth and C. Hull, “Occam—a programming lan- Conference on Digital Audio Effects, Naples, Italy, October guage for multiprocessor systems,” Computer Languages,vol. 2004. 12, no. 1, pp. 27–37, 1987. [94] F. Fontana and M. Civolani, “Modeling of the EMS VCS3 voltage-controlled filter as a nonlinear filter network,” IEEE Transactions on Audio, Speech and Language Processing,vol. 18, no. 4, pp. 760–772, 2010. [95] T. Helie,´ “On the use of Volterra series for real-time simulations of weakly nonlinear analog audio device: application to the Moog ladder filter,” in Proceedings of the International Conference on Digital Audio Effects, pp. 7–12, Montreal, Canada, September 2006. [96] T. Helie,´ “Volterra series and state transformation for real- time simulations of audio circuits including saturations: application to the Moog ladder filter,” IEEE Transactions on Audio, Speech and Language Processing, vol. 18, no. 4, pp. 747– 759, 2010. [97] M. Civolani and F. Fontana, “A nonlinear digital model of the EMS VCS3 voltage-controlled filter,” in Proceedings of the International Conference on Digital Audio Effects (DAFX ’08), pp. 35–42, Espoo, Finland, September 2008. [98] M. Puckette, “Pure data,” in Proceedings of the International Computer Music Conference, pp. 224–227, Thessaloniki, Greece, 1997. [99] P. Manning, Electronic and Computer Music, Oxford Univer- sity Press, Oxford, UK, 1989. [100] M. V. Mathews, “The digital computer as a musical instrument,” Science, vol. 142, no. 3592, pp. 553–557, 1963. [101] M. V. Mathews, J. E. Miller, F. R. Moore, J. R. Pierce, and J. C. Risset, The Technology of Computer Music, MIT Press, Cambridge, Mass, USA, 1969. [102] J. ffitch, “On the design of Csound 5,” in Proceedings of the 3rd Linux Audio Conference, pp. 37–42, 2005. [103] M. Puckette, The Theory and Technique of Electronic Music, World Scientific Press, 2007. [104] J. McCartney, “Rethinking the computer music language: SuperCollider,” Computer Music Journal,vol.26,no.4,pp. 61–68, 2002. [105] R. B. Dannenberg, “Machine tongues XIX: Nyquist, a language for composition and sound synthesis,” Computer Music Journal, vol. 21, no. 3, pp. 50–60, 1997. [106] M. Laurson, M. Kuuskankare, and V. Norilo, “An overview of PWGL, a visual programming environment for music,” Computer Music Journal, vol. 33, no. 1, pp. 19–31, 2009. [107] M. Rumori, “Sound production and audio programming of the sound installation GROMA,” in Proceedings of the 27th AES Convention, New York, NY, USA, 2009, paper no. 7864. [108] V. Lazzarini, J. Timoney, J. Kleimola, and V. Valim¨ aki,¨ “Five variations on a feedback theme,” in Proceedings of the International Conference on Digital Audio Effects (DAFX ’09), pp. 139–145, Como, Italy, September 2009. [109] M. Puckette, “Max at seventeen,” Computer Music Journal, vol. 26, no. 4, pp. 31–43, 2002. [110] Y. Orlarey, D. Fober, and S. Letz, “Faust: an efficient functional approach to DSP programming,” in New Computational