Synthetic Spectra by Means of Adaptive Frequency Modulation

Victor Lazzarini, Joseph Timoney, and The Generation of Natural- Thomas Lysaght An Grúpa Theicneolaíocht Fuaime agus Ceoil Synthetic Spectra by Dhigitigh (Sound and Digital Music Technology Group) Means of Adaptive National University of Ireland, Maynooth Maynooth, Co. Kildare, Ireland [email protected] Frequency Modulation {JTimoney, TLysaght}@cs.nuim.ie

Frequency- modulation (FM) synthesis is widely the sound and in others by the synthetic- sounding known as a computationally effi cient method for quality of the generated spectra. These shortcom- synthesizing musically interesting timbres. How- ings spurred software and hardware designers to ever, it has suffered from neglect owing to the come up with new solutions for instrument control diffi culty in creating natural- sounding spectra and and improvements to the basic FM method (Pala- mapping gestural input to synthesis parameters. min, Palamin, and Ronveaux 1988; Tan and Gan Recently, a revival has occurred with the advent of 1993; Horner 1996). Nevertheless, these develop- adaptive audio- processing methods, and this work ments failed to stem the decline in the technique’s proposes a technique called adaptive FM synthesis. use as increasingly more powerful hardware became This article derives two novel ways by which an available. arbitrary input signal can be used to modulate a Some of the limitations of gestural controllers carrier. We show how phase modulation (PM) can and of synthetic sound in FM can be addressed be achieved fi rst by using delay lines and then by together by the use of adaptive techniques, which heterodyning. By applying these techniques to form an important subset of musical signal- real- world signals, it is possible to generate transi- processing techniques (Verfaille and Arfi b 2002; tions between natural- sounding and synthesizer- Verfaille, Zölzer, and Arfi b 2006). A key aspect of like sounds. Examples are provided of the spectral their usefulness in music composition and perfor- consequences of adaptive FM synthesis using inputs mance is that they provide a means to retain signifi - of various acoustic instruments and a voice. An cant gestural information contained in the original assessment of the timbral quality of synthesized signal. Therefore, these techniques seem to be well sounds demonstrates its effectiveness. suited to help develop more natural- sounding forms of FM synthesis. With them, it might be possible to obtain results that share much of the liveliness Background perceived in musical signals of instrumental origin. The traditional approach has been to treat synthe- Frequency modulation (FM), introduced by John sis and control parameters separately, using some Chowning in his seminal article on the technique means of mapping to control the process (Miranda (Chowning 1973), is one of the most important and Wanderley 2006; Wanderley and Depalle 2004). classic methods of synthesis. It has proved very This ultimately can lead to a split between gesture useful as an economical means of generating time- and sonic result, especially in the case of FM, where varying complex spectra. For this reason, it was the mapping is often not clear or too coarsely de- widely adopted at a time when computational speed fi ned. Alternatively, one can approach the problem was a determining factor in the choice of signal- from an adaptive point of view, whereby a signal is processing algorithms. However, the method always both the source of control information (extracted made it diffi cult for composers to produce natural- from it through different analysis processes) and the sounding spectral evolutions. This in some cases input to the synthesis algorithm. Some pioneering was caused by the lack of fi ne gestural control over works in the area have proposed interesting applications of this principle in what has been called Computer Music Journal, 32:2, pp. 9–22, Summer 2008 audio- signal driven sound synthesis (Poepel 2004; © 2008 Massachusetts Institute of Technology. Poepel and Dannenberg 2005).

Lazzarini et al. 9

Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/comj.2008.32.2.9 by guest on 27 September 2021 In the specifi c case of FM synthesis, it is possible The Technique to use an arbitrary input signal in two ways, either as a modulator or as a carrier. In the former case, The synthesis technique discussed here is based on this signal is used to modulate the frequency of one two elements: some means of phase modulation of or more oscillators. When the input is anything but an input signal; and the use of an arbitrary, mono- a sinusoidal wave, this arrangement produces what phonic, pitched or quasi- pitched input to which we normally describe as complex FM (Schottstaedt parameter estimation will be applied. The phase 1977). Although this setup, proposed by Poepel and modulation effect can be achieved by two basic Dannenberg (2005), provides a richer means of methods: through the use of a variable delay line or gestural control over the process, it does not seem by heterodyning. to capture well the original spectral characteristics of interesting input sounds (such as the ones originating from instrumental sources). The spectral Delay- Line Based Phase Modulation evolutions allowed by the method still resemble the more synthetic results typical of standard FM A well-known side-effect of variable delays is the synthesis, because the carrier is still a sine wave phase modulation of the delay- line input (Dilsch oscillator. If we want to allow as much of the tim- and Zölzer 1999). This is the basis for all classic bral qualities of the input sound to affect the gener- variable- delay effects such as fl anging, chorusing, ated sound, we will get better results using the pitch shifting, and vibrato. The principle has also input as a carrier signal. been used in audio- rate modulation of waveguide Considering non-sinusoidal inputs, this case is models (Van Duyne and Smith 1992). It is thus similar to multiple- carrier FM (Dodge and Jerse possible to model simple (sinusoidal) audio- rate 1985). The techniques described in this article phase modulation using a delay- line with a suitable implement this arrangement. Standard multiple- modulating function (see Figure 1). carrier FM is defi ned by a single modulator being We now consider the case where the input to the

used to vary the frequency of several sinusoidal delay line is a sinusoidal signal of frequency fc: carriers. It has proved useful in a variety of applica- x(t) = sin(2f t) (1) tions, including vocal synthesis (Chowning 1989) c and instrumental emulation via spectral matching When the modulating source is s(t) = d D(t), (Horner, Beauchamp, and Hakken 1993). By applying max where D(t) ∈{0 . . . 1} is an arbitrary function, and d the technique to real-world signals, it is possible to max generate transitions between natural- sounding and is the maximum delay, the delay- line phase modulation of Equation 1 can be deﬁ ned (with ω = 2πf ) as synthesizer-like sounds. Depending on the levels of c modulation, we are able to reveal more or less of the y(t) = sin([t − d D(t)]) (2) original timbral qualities of the input. This is the max basis for our technique of adaptive FM synthesis, or ω The instantaneous radian frequency i(t) of such a AdFM (Lazzarini, Timoney, and Lysaght 2007). phase- modulated signal can be estimated from the To use an arbitrary input as a carrier, we must derivative of the phase angle θ(t): develop some means of modulating the frequency (or, to be more precise, the phase) of that signal. ∂(t) ∂[t − d D(t)] ∂D(t) (t) = = max = − d (3) This is required because we no longer use an oscilla- i max ∂t ∂t ∂t tor to produce the sound, so we have no implicit frequency control of the arbitrary signal. The and the instantaneous frequency IF(t) in Hz can be following section addresses two different methods deﬁ ned as of achieving this. We then discuss the implications ∂D(t) of using complex signals as carriers and details of IF(t) = f − d (4) parameter extraction. c ∂t max

10 Computer Music Journal

Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/comj.2008.32.2.9 by guest on 27 September 2021 Figure 1. Delay-line based phase modulation.

= + y(t) J0(I)sin( ct)

I +1 (11) ∑ + + − J k(I)sin( ct k mt) J −k(I)sin( ct k mt) k=1 ω π ω π where c = 2 fc, m = 2 fm, Jk(I) are Bessel functions of the ﬁ rst kind of order k, and

J (I)=(−1) k J (I) (12) −k k Considering the case where the modulating signal Note that to match the phases as closely as pos- is a scaled raised cosine (i.e., a periodically repeating sible to Equation 11, we require an offset of π / 2 + 2I Hanning window), we have in the input sinusoid and π / 2 in the modulator (both in relation to cosine phase). Because the carrier = + D(t) 0.5cos(2 fmt) 0.5 (5) phase depends on the index of modulation in general, we only rarely achieve an exact match. Thus, D(t) IF(t) and, by substituting in Equation 4, is now in delay- line phase modulation, we need not be too = + concerned with phase offsets. IF(t) fm sin(2 fmt)dmaxfc fc (6) Interestingly enough, in the delay- line formula- which characterizes the instantaneous frequency in tion of FM / PM, the index of modulation for a given sinusoidal phase modulation. In such an arrange- variable delay- width is proportional to the carrier- ment, the sinusoidal term in Equation 6 is known signal frequency (as seen in Equation 9). This as the frequency deviation, whose maximum situation does not arise in classic FM. Also, when

absolute value DEVmax is considering the width of variable delay for a given value of I, we see that it gets smaller as the frequency DEV = d × f f (7) max m c rises. In a digital system, for I = 1, the width will be Δ less than one sample at the Nyquist frequency. with d = dmax – dmin. Now, turning to FM theory, we characterize the index of modulation I as the ratio of the maximum deviation and the modulation frequency: Phase Modulation Through Heterodyning DEV df f The second method proposed here is based on a I = max = m c = df (8) f f c simple re-working of the PM formula. We begin by m m proposing the following synthetic signal, where I is Δ ω The d that should apply as the amplitude of our the index of modulation and m is the radian modu- ω π sinusoidal modulating signal can now be put in lation frequency ( m = 2 fm): terms of the index of modulation y(t) = x(t)cos(I sin( t)) (13) m I d = (9) Using a sinusoid described in Equation 1 as our input f c signal x(t), we obtain, by manipulating the expres- and the modulating signal is now sion, the following combination of PM signals: y(t) = sin( t)cos(I sin( t)) = I ⎡ + ⎤ c m d(t) ⎣0.5cos(2 fmt) 0.5⎦ (10) f = 0.5[sin( t + I sin( t)) + sin( t − I sin( t))] (14) c c m c m = + − The resulting spectrum according to FM theory is 0.5[PM( c, m ,I,t) PM( c, m ,I,t)] dependent on the values of both I and the carrier- to- modulator (c:m) frequency ratio: where the PM signal is deﬁ ned as

Lazzarini et al. 11

Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/comj.2008.32.2.9 by guest on 27 September 2021 I f PM(c,m,I,t)= sin(ct+ Isin(mt)) (15) I = df = f = I n (19) n n f n f c c By inspecting Equation 11 , it is clear that this formulation, based on the mixing of two PM sig- Again, we see here that the effect of the relation- nals, will lead to the cancellation of certain compo- ship between the index of modulation and the carrier nents in the output signal, namely the ones where k frequency is that higher- frequency partials will be is odd (called in FM theory the odd sidebands). modulated more intensely than lower ones. Depend- The signiﬁ cance of this and the previous imple- ing on the bandwidth and richness of the input sig- mentations of PM can be fully appreciated only nal, it is quite easy to generate very complex spectra, once we move from using sinusoidal inputs to which might be objectionable in some cases. This arbitrary signals. This will allow us to develop the increase in brightness has also been observed in synthesis designs we propose in this work. other applications of audio- rate modulation of delay lines (Välimäki, Tolonen, and Karjalainen 1998; Tolonen, Välimäki, and Karjalainen 2000). Using Arbitrary Input Signals Turning now to the second technique introduced herein, we will have a signiﬁ cantly different output, We will now examine the results of applying described by arbitrary input signals to both formulations just ⎡N −1 ⎤ ∑ + + described, beginning with the delay- line based PM. ⎢ an sin( nt I sin( mt) n) ⎥ n=0 In Equation 11, we see the ordinary spectrum of y(t) = 0.5⎢ ⎥ (20) ⎢ N −1 ⎥ simple FM. However, for our present purposes, we +∑ + − + ⎢ an sin( nt I sin( mt) n)⎥ ⎣ = ⎦ will assume the input x(t) to be a complex arbitrary n 0 signal made up of N sinusoidal partials of ampli- The most important differences between the spec- tudes a , radian frequencies ω , and phase offsets φ , n n n trum of this signal and that described by Equation originating, for instance, from instrumental sources: 18 are that odd sidebands are now canceled, and the N −1 = ∑ + index of modulation I is now constant across the x(t) an sin( nt n) (16) n=0 modulated carrier components. Whereas the former is responsible for an overall timbral difference The resulting phase- modulated output is equiva- between the two spectra, the latter is responsible for lent to what is normally called multiple- carrier FM a more controlled and subtle handling of high synthesis, because the carrier signal is now com- frequencies. plex. This output y(t) can be described as Another key aspect of the proposed methods is N −1 = ∑ + + that the c:m ratio parameter can also be taken y(t) an sin( nt In sin( mt) n) (17) n=0 advantage of by estimating the fundamental frequency of the input signal (assumed to be mono- where ω is the modulation frequency and I is the m n phonic). In this case, a variety of different spectral index of modulation for each partial. According to combinations can be produced, from inharmonic to Equation 11, this would be equivalent to the follow- harmonic and quasi- harmonic. ing signal: ⎡ J (I )sin( t + )+ ⎤ ⎢ 0 n n n ⎥ N −1 Fundamental Frequency Estimation = ∑ ⎢ I +1 + + + ⎥ y(t) an J k(In)sin( nt k mt n) (18) n=0 ⎢∑ ⎥ ⎢ k=1 J (I )sin( t − k t + ) ⎥ To allow for a full control of c:m ratio and modula- ⎣ −k n n m n ⎦ tion index, it is necessary to estimate the funda- The different indices of modulation for each compo- mental frequency of the carrier signal. That will nent of the carrier signal can be estimated by the allow the modulator signal frequency and ampli- following relationship, derived from Equation 9: tude to be set according to Equation 10. This can be

12 Computer Music Journal

Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/comj.2008.32.2.9 by guest on 27 September 2021 Figure 2. Delay-line based AdFM design: (a) original; (b) with the optional low- pass ﬁ lter.

achieved with the use of a pitch tracker, which is a (a) standard component of many modern music signal- processing systems. For the current implementation, a spectral- analysis pitch- tracking method was devised, based on an algorithm by Puckette, Apel, and Ziccarelli (1998) and Puckette and Brown (1998), that provides ﬁ ne accuracy of fundamental-frequency estimation. In addition to tracking the pitch, it is also useful (but not essen- tial) to obtain the amplitude of the input signal, which can be used in certain applications to scale the index of modulation. This is also provided by our parameter- estimation method.

Signal Bandwidth

Although the spectrum of FM is, in practical terms, (b) band- limited, it is capable of producing very high frequencies, as seen in Equations 11 and 18. With digital signals, this can lead to aliasing problems if the bandwidth of the signal exceeds the Nyquist frequency. The fact that in the delay- based formulation the index of modulation increases with frequency for a given Δd (Equation 19) is obviously problematic. However, in practice, the kind of input signals we will be employing generally exhibit a spectral envelope that decays with frequency. In this case, objectionable aliasing problems might be

greatly minimized, given that an in Equation 18 for higher values of n will be close to zero. Of course, if our input contains much energy in the higher end of the spectrum, such as for instance an impulse train, then aliasing will surely occur. The simplest solution for such problematic signals described herein. These two instrument designs can is to impose a decaying spectral envelope using a serve as the basis for further software- or hardware- fi lter. This will have the obvious side-effect of modi- based implementations. The basic fl ow chart of the fying the timbre of the input signal. Another, more delay- based PM instrument is shown in Figure 2a. computationally costly, solution is to oversample There are three basic components: a pitch tracker, a the input signal. This would either remove the modulating source (a table- lookup oscillator), and a aliased signals or place them at an inaudible range. variable delay line with interpolated readout. Each of these components is found in modern music signal- processing systems, so the technique is Implementation highly portable. The implementation discussed here uses Csound 5 (ffi tch 2005) as the synthesis engine, We now present a reference implementation of but similar instruments can be developed under AdFM using both methods of phase modulation other musical signal- processing environments,

Lazzarini et al. 13

Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/comj.2008.32.2.9 by guest on 27 September 2021 Figure 3. Delay-based Figure 4. The heterodyning Figure 5. Heterodyning AdFM code. AdFM design. AdFM Csound code.

Figure 4

such as the SndObj library and PySndObj (Lazza- rini 2000, 2007). It is important to note that this design can be used either for real-time or off-line applications. In addition, plug-ins can be easily developed from it using csLadspa (Lazzarini and Walsh 2007). The equivalent Csound 5 code for the ﬂ owchart design in Figure 2, which implements the delay- based version, is shown in Figure 3. The heterodyning PM design is simpler, based on a more or less straight translation of the formula in Equation 13. Its ﬂ owchart is shown in Figure 4 and the corre- Figure 5 sponding Csound code in Figure 5. Both implementations use a spectral- analysis samples to avoid errors in the circular- buffer pitch- tracking opcode (ptrack) written by the readout. authors and linear interpolation oscillators to A number of variations can be made to the basic generate the modulation signal. The DFM opcode design. For instance, the amplitude of the signal, uses a cubic- interpolation variable delay line which is produced together with the pitch- tracking (Laakso et al. 1996). Owing to the use of cubic information, can be used to scale the index of interpolation, the minimum delay is set to two modulation. This can be used to generate typical

14 Computer Music Journal

Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/comj.2008.32.2.9 by guest on 27 September 2021 Figure 6. Steady- state spectrum of a ﬂ ute playing C4.

brass- like synthesizer tones (Risset 1969), where the The sound examples discussed here will be found brightness of the synthetic output is linked to the on the annual Computer Music Journal DVD (to be amplitude evolution of the input sound. Alterna- released with the Winter 2008 issue). tively, it can be used to determine the c:m ratio. Depending on the characteristics of the input signal, it might be useful to include a low-pass fi lter Flute Input before the signal is sent to the AdFM processors, especially in the delay- based-version, as shown in The original steady- state fl ute spectrum, effectively Figure 2b. The cutoff frequency of the low-pass with I = 0, is shown in Figure 6. As clearly seen in fi lter can also be controlled by the estimated input that fi gure, it features quite prominent lower har- amplitude. As discussed earlier, this will reduce monics. Using delay- line AdFM and applying an aliasing as well as overall brightness, both of which index of modulation of 0.3 on a 1:1 c:m confi gura- are sometimes a downside of FM synthesis. tion, we can start enriching the spectrum with higher harmonics (see Figure 7). At these low values of I, there is already a considerable addition of com- Examples and Discussion ponents between 5 and 10 kHz. The overall spectral envelope still preserves its original decaying shape. Four different types of carrier signals were chosen as Using the delay line method with higher values of a way of examining the qualities of the AdFM I, we can see a dramatic change in the timbral synthetic signal using both methods described in characteristics of the original fl ute sound. Figure 8 this article. A fl ute input with its spectral energy shows the resulting spectrum, now with I = 1.5. concentrated in the lower harmonics is a prime Here, we can see that components are now spread to candidate for experimentation. The clarinet was the entire frequency range. The original decaying chosen for its basic quality of having more promi- spectral envelope is distorted into a much more nent odd harmonics. Finally, the piano and voice gradual shape, and the difference between the were used as a means of exploring the possibilities loudest and the softest harmonic is only about 30 of synthesizing different types of harmonic and dB. The resulting sound can been described as inharmonic spectra by the use of various c:m ratios. “string-like,” and the transition between the fl ute

Lazzarini et al. 15

Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/comj.2008.32.2.9 by guest on 27 September 2021 Figure 7. AdFM spectrum Figure 8. AdFM spectrum using a ﬂ ute C4 signal as using same input as Figure carrier with c:m = 1 and 3, but now with I = 1.5. I = 0.3.

Figure 7

Figure 8

and AdFM spectra is capable of providing interest- As I gets higher, the spectrum gets even brighter, ing possibilities for musical expression. Also, it is but the problems with aliasing start to become important to note that important gestural charac- signifi cant. To prevent this and also to allow for a teristics of the original sound, such as pitch fl uctua- different spectral envelope, an optional low-pass tions, vibrato, and articulation, are preserved in the fi ltering of the input signal is suggested. In that synthetic output. case, the fi lter is inserted in the signal path at the

16 Computer Music Journal

Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/comj.2008.32.2.9 by guest on 27 September 2021 Figure 9. Heterodyning AdFM synthesis using a ﬂ ute input with I = 5 and c:m = 1.

delay- line input. A Butterworth low-pass fi lter with tially. In delay- line AdFM with I = 1.5, it is possible a cutoff frequency between 1,000 and 5,000 Hz has to see that there is very little difference between the proven useful. It is possible to couple the cutoff strengths of odd and even components (see Figure 11). frequency with I, so that for higher values of that In addition, higher- order harmonics become more parameter, more fi ltering is applied. present, and the spectral envelope levels out, owing The addition of higher harmonics is signifi cantly to the well-known spread of energy that is charac- reduced in the heterodyning AdFM method. We can teristic of FM synthesis. see in Figure 9 how much more attenuated the top The heterodyning method also provides similar end of the spectrum is in comparison to the pre- transformations, although again with more subtle vious technique. This in some cases might be high- frequency results, and still retaining some of advantageous; however, the effect of the technique the odd / even balance of the input. Figure 12 demon- is subtler, resulting in a transition between natural- strates that the resulting spectrum features a decay- sounding and synthesizer- like spectra that is less ing envelope, in contrast to the previous example dramatic. (see Figure 11), which is much fl atter.

Clarinet Input Piano Input

Our second experiment used a clarinet signal as a In the previous examples, we have kept the ratio carrier wave for AdFM. The clarinet exhibits a between the modulating frequency and carrier fun- steady- state spectrum in which the lower- order damental at unity. However, as we know from FM even harmonics are signiﬁ cantly less energetic than theory, a range of different spectra is possible if we their odd neighbors (see Figure 10). As a result, the use different ratios. It is possible to create a range of multiple-carrier- like characteristic of AdFM helps effects that range from changing the fundamental of generate quite a change in the spectra of that the sound to transforming a harmonic spectrum instrument. into an inharmonic one. We took a piano C2 signal As the index of modulation increases, the balance as our carrier and then tuned our modulator to 1.41 between odd an even harmonics changes substan- times that frequency. The original piano spectrum

Lazzarini et al. 17

Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/comj.2008.32.2.9 by guest on 27 September 2021 Figure 10. Detail of Figure 11. Detail of AdFM steady- state spectrum of spectrum using a clarinet clarinet C3. Note the C3 signal as carrier with higher relative strength of c:m = 1 and I = 1.5. Odd lower- order odd harmonics and even harmonics now versus even ones. have comparable strengths.

Figure 10

Figure 11

is shown in Figure 13, where we can clearly see its generating what is perceived as an inharmonic harmonics. spectrum. With the 1:1 ratio, the sums and differ-

The resulting delay- line AdFM spectrum with ences between fc and fm created components whose I = 0.15 is shown in Figure 14. This particular ratio frequencies were mostly coincident. Here, a variety creates a great number of components whose rela- of discrete components will be generated, creating tionship implies a very low fundamental, thus the denser spectrum seen in Figure 14. The AdFM

18 Computer Music Journal

Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/comj.2008.32.2.9 by guest on 27 September 2021 Figure 12. Steady- state Figure 13. Spectrogram of a Figure 14. Spectrogram of inharmonic spectrum, spectrum of clarinet- input piano C2 tone, showing its an AdFM sound using a with a large number of heterodyning AdFM, with ﬁ rst harmonics in the piano C2 signal as carrier, components, is clearly I = 5 and c:m = 1. 0–1.2 kHz range. with c:m = 1:1.41 and I = seen in comparison with 0.15, showing the 0–1.2 Figure 13. kHz range. The resulting

Figure 12

Figure 13 Figure 14

Lazzarini et al. 19

Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/comj.2008.32.2.9 by guest on 27 September 2021 Figure 15. Comparison of spectral snapshots of a vocal and an AdFM vocal sounds, with I = 0.1 and c:m = 2.

sound resulting from this arrangement has been components and a ﬂ attening of spectral peaks, the described as “bell-like.” Transitions between piano AdFM voice is still perfectly intelligible. and bell sounds can be effected by changing I from 0 Figure 15 shows a comparison between a vowel to the desired value. The application of a low-pass steady- state spectrum and its AdFM-processed ﬁ lter at the delay- line input will also allow for some counterpart. The sub-harmonic peak can be seen variety and control over the brightness of the result. at the left of the picture below the original funda- Again, if we apply the heterodyning technique mental. (A peak at 0 Hz is also present, owing to the

instead to this input using a similar ratio, we will fc – 2fm component.) The recording of the phrase, obtain a bell-like output that is better behaved in “This is AdFM Synthesis,” is shown as a spectro- the higher end of the spectrum. Here the second gram in Figure 16, both as the original signal (left) method might in fact be more useful, as it can and the AdFM output (right), using the same param- control the quality of the output more effectively. eters as in the previous example. Again, the octave change is clearly seen, as well as the increase in the number of signiﬁ cant components in the signal. Voice Input In general, we achieved better results using the delay- line method with vocal inputs. The hetero- A vocal input was used as the fourth different dyning process seems to be too prone to artifacts source examined in this work, demonstrating a generated by unvoiced phonemes, resulting in

pitch- shift effect. Setting the fc:fm ratio to 2, we are chirps and glitches. Although these are originally able to obtain a sound that is now half the pitch of caused by the pitch- tracking mechanism, they the original. This is due to the introduction of a are emphasized by certain characteristics of the component at half the fundamental frequency method’s implementation.

corresponding to fc – fm in Equation 18. With the index of modulation at low values (around 0.15), it is possible to preserve some of the Conclusion spectral shape of the original sound, a crucial step in keeping the intelligibility of the vocal phonemes. We presented an alternative approach to the classic Although there is some addition of high-frequency technique of FM synthesis, based on an adaptive

20 Computer Music Journal

Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/comj.2008.32.2.9 by guest on 27 September 2021 Figure 16. Detail of spectrogram of a recording of the phrase, “This is AdFM synthesis,” with the original vocal sound on the left and the AdFM vocal on the right

design, which we call AdFM. Two different methods parameters. It is our plan to develop a spectral were proposed as a means of modulating an arbi- version of AdFM in Csound, as SPV analysis / syn- trary carrier signal. As the FM synthesis theory is thesis and audio- rate frequency scaling have been well known, it was possible to adapt it to determine added to the language in version 5.07. the precise characteristics of the output signal. With this technique, it is possible to achieve fi ne control over the synthetic result, which also preserves a References substantial amount of the gestural information in the original signal. Four different types of carrier Bradford, R., R. Dobson, and J. ffi tch. 2007. “The signals were used in this work to demonstrate the Sliding Phase Vocoder.” Proceedings of the 2007 wide range of spectra that the technique can gener- International Computer Music Conference. San ate. We are confi dent this is a simple yet effective Francisco: International Computer Music Association, way of creating hybrid natural- synthetic sounds for pp. 449–452. musical applications. Chowning, J. 1973. “The Synthesis of Complex Audio Future prospects for research into AdFM involve Spectra by Means of Frequency Modulation.” Journal of the development of alternative implementations of the Audio Engineering Society 21:526–534. Chowning, J. 1989. “Frequency Modulation Synthesis the technique, both in terms of time-domain of the Singing Voice.” In M. Mathews and J. R. Pierce, variations of the methods discussed here and new eds., Current Directions in Computer Music Research. frequency- domain processes. The latter have been Cambridge, Massachusetts: MIT Press, pp. 57–63. facilitated by the development of the Sliding Phase Dilsch, S., and U. Zölzer. 1999. “Modulation And Delay Vocoder (SPV; Bradford, Dobson, and ffi tch 2007), Line Based Digital Audio Effects.” Proceedings of the which allows for audio- rate modulation of its 2nd Conference on Digital Audio Effects. Trondheim:

Lazzarini et al. 21

Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/comj.2008.32.2.9 by guest on 27 September 2021 Norwegian University of Science and Technology, Puckette, M., T. Apel, and D. Ziccarelli. 1998. “Real- pp. 5–8. Time Audio Analysis Tools for PD and MSP.” Pro- Dodge, C., and T. Jerse. 1985. Computer Music. New ceedings of the 1998 International Computer Music York: Schirmer Books. Conference. San Francisco: International Computer ffi tch, J. 2005. “On the Design of Csound5.” Proceed- Music Association, pp. 109–112. ings of the 3rd International Linux Audio Conference. Puckette, M., and J. Brown. 1998. “Accuracy of Frequency Karlsruhe: Zentrum für Künst und Medientechnologie, Estimates from the Phase Vocoder.” IEEE Transactions pp. 37–42. on Speech and Audio Processing 6(2):116–172. Horner, A. 1996. “Double-Modulator FM Matching Risset, J. C. 1969. An Introductory Catalogue of Com- of Instrument Tones.” Computer Music Journal puter Synthesized Sounds. Murray Hill, New Jersey: 20(2):57–71. AT&T Bell Laboratories. Horner, A., J. Beauchamp, and L. Hakken. 1993. “Ma- Schottstaedt, W. 1977. “The Simulation of Natural Instru- chine Tongues XVI: Genetic Algorithm and Their Ap- ment Tones Using a Complex Modulating Wave.” plication to FM Synthesis.” Computer Music Journal Computer Music Journal 1(4):46–50. 17(4):17–29. Tan, B. J. , and S. L. Gan. 1993. “Real-Time Implementa- Laakso, T. I., et al. 1996. “Splitting the Unit Delay: Tools tion of Asymmetrical Frequency- Modulation Syn- for Fractional Delay Filter Design.” IEEE Signal Pro- thesis.” Journal of the Audio Engineering Society cessing Magazine 13(1):30–60. 41(5):357–363. Lazzarini, V. 2000. “The Sound Object Library.” Or- Tolonen, T., V. Välimäki, and M. Karjalainen. 2000. ganised Sound 5(1):35–49. “Modeling of Tension Modulation Nonlinearity in Lazzarini, V. 2007. “Musical Signal Scripting with Plucked Strings.” IEEE Transactions on Speech and PySndObj.” Proceedings of the 5th International Linux Audio Processing 8(3):300–310. Audio Conference. Berlin: Technische Universität Välimäki, V., T. Tolonen, and M. Karjalainen. 1998. Berlin, pp. 18–23. “Signal- Dependent Nonlinearities for Physical Models Lazzarini, V., J. Timoney, and T. Lysaght. 2007. “Adaptive Using Time-Varying Fractional Delay Filters.” Proceed- FM Synthesis.” Proceedings of the 10th International ings of the 1998 International Computer Music Confer- Conference on Digital Audio Effects. Bordeaux: Univer- ence. San Francisco: International Computer Music sity of Bordeaux, pp. 21–26. Association, pp. 264–267. Lazzarini, V., and R. Walsh. 2007. “Developing LADSPA Van Duyne, S. A., and J. O. Smith. 1992. “Implementa- Plugins with Csound.” Proceedings of the 5th Inter- tion of a Variable Pick-Up Point on a Waveguide String national Linux Audio Conference. Berlin: Technische Model with FM / AM applications.” Proceedings of the Universität Berlin, pp. 30–36. 1992 International Computer Music Conference. San Miranda, E., and M. Wanderley. 2006. New Digital Musi- Francisco: International Computer Music Association, cal Instruments. Middleton, Wisconsin: A-R Editions. pp. 154–157. Palamin, J.P., P. Palamin, and A. Ronveaux. 1988. “A Verfaille, V., and D. Arfi b. 2002. “Implementation Strate- Method of Generating and Controlling Musical Asym- gies for Adaptive Digital Effects.” Proceedings of the metric Spectra.” Journal of the Audio Engineering 5th Conference on Digital Audio Effects. Hamburg: Society 36(9):671–685. University of the Federal Armed Forces, pp. 21–26. Poepel, C. 2004. “Synthesized Strings for String Players.” Verfaille, V., U. Zölzer, and D. Arfi b. 2006. “Adaptive Proceedings of 2004 Conference on New Instruments Digital Audio Effects (a-DAFx): A New Class of Sound for Musical Expression. New York: Association for Transformations.” IEEE Transactions on Audio, Speech Computing Machinery, pp. 150–153. and Language Processing 14(5):1817–1831. Poepel, C., and R. Dannenberg. 2005. “Audio Signal Wanderley, M. M., and P. Depalle. 2004. “Gestural Driven Sound Synthesis.” Proceedings of the 2005 Control of Sound Synthesis.” Proceedings of the IEEE International Computer Music Conference. Barce- 92(4):632–644. lona: International Computer Music Association, pp. 391–394.

22 Computer Music Journal

Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/comj.2008.32.2.9 by guest on 27 September 2021