The Synthesis of Complex Audio Spectra by Means of Frequency Modulation JOHN M. CHOWNING Stanford Artificial Intelligence Laboratory, Stanford, California A new application of the well-known process of frequency modulation is shown to result in a surprising control of audio spectra. The technique provides a means of great simplicity to control the spectral components and their evolution in time. Such. dynamic spectra are diverse in their subjective impressions and include sounds both known and unknown. INTRODUCTION: Of interest in both acoustical re- judge to be their lively quality. In contrast, it is large- search and electronic music is the synthesis of natural ly the fixed proportion spectrum of most synthesized sound. For the researcher, it is the ultimate test of acous- sounds that so readily imparts to the listener the elec- tical theory, while for the composer of electronic music tronic cue and lifeless quality. it is an extraordinarily rich point of departure in the The special application of the equation for frequency domain of timbre, or tone quality. The synthesis of natur- modulation, clescribed below, allows the production of al sounds 'has been elusive; however, recent _research in complex spectra with very great simplicity. The fact computer analysis and synthesis of some tones of mu- that the temporal evolution of the frequency compo- sical instruments [1] has yielded an insight which may nents of the spectrum can be easily controlled is perhaps prove to have general relevance in all natural sounds: the the most striking attribute of the technique, for dy- character of the temporal evolution of the spectral corn- namic spectra are achieved only with considerable dif- ponents is of critical importance in the determination of ficulty using current techniques of synthesis. At the end timbre, of this paper some simulations of brass, woodwind, and In natural sounds the amplitudes of the frequency eom- percussive sounds are given. The importance of these ponents of the spectrum are time-variant, or dynamic, simulations is as much in their elegance and simplicity The energy of the components often evolve in compli- as it is in their accuracy. This frequency modulation cated ways; in particular, during the attack and decay technique, although not a physical model for natural portions of the sound. The temporal evolution of the sound, is shown to be a very powerful perceptual model spectrum is in some cases easily followed as with bells, for at least some. whereas in other cases not, because the evolution oc- FREQUENCY MODULATION curs in a very short time period, but it is nevertheless perceived and is an important cue in the recognition of Frequency modulation (FM) is well-understood as ap- timbre. Many natural sounds seem to have characteris- plied in radio transmission, but the relevant equations tic spectral evolutions which, in addition to providing have not been applied in any significant way to the their "signature," are largely responsible for what we generation of audio spectra where both the carrier and 526 JOURNALOFTHEAUDIOENGINEERINSOCG IETY AMPLITUDE frequencies. This increasing bandwidth as I increases, is % _ shown in Fig. 1, with a constant-modulating frequency. x:o'[ / FREQ. areThedeterminedamplitudesby ofBesselthe carrierfunctionsand ofsidebandthe firstcomponentskind and c a. nth order, Jrt(I), the argument to which is the modula- tion index. The first six Bessel functions, Jo through Is, 1=1 . _t · . , T i I , . indexare shownI, Jo(Iin), Fig.yields2. Thean amplitude0th orderscalingBesselcoefficientfunction andfor c-Sm c-2m c-m C c+m c+2rnc+3rn ' the carrier frequency; the ist order, Il(I), yields a scaling b. coefficient for the first upper- and lower-side frequen- '_ i cies; the 2nd order, J2(I), for the second upper- and z:z .51 , , t /t f: J* T , . , derlower-sideof the frequencies;side frequencyand theso largerforth. theThe indexhigher mustthe orbe- c-zm c c+2m C. for that side frequency to have significant amplitude. The total bandwidth is approximately equal to twice the i=3 .s_ · .t Ttt itt.. sumquency,of theor frequency deviation and the modulating fre- c-4m C c+4m d. BW _. 2(d+m). _=". ' _' 0,_]-_! J I i !_I I I I ! , t I t, I, t I t , ,,_ oeL__J_J J___l_j-l--rT-:Yt __-- c-5m c c+Sm _ ' j _ / J-C_rr,_r J'_o_-_ -'_i l t t I _- e. '_ 071__:y=T22 ' _-'_ I , t I ___.t I !---1-- ._ _,1 ill i [_i t j i , I : J I I I I l Fig. 1. Example to show increasing bandwidth with in- ._ u._ l--i-i_X_----.x--_-_---T- F''. J'-o.''*''-, --5%_......../:c,,d co,,_;o.....r._ i_-;1_,--'_'- -_o_ i I - : quencies are at intervals of the modulating frequency, m, and '2 are symmetrical around the carrier, c. mo,., t,on,nde ,,3 a._I_LL_Ii\I1 'i__1 t I-a_-_-.[T-T-i- the modulating frequencies are in the audio band and _ _.1u. 1__i t_ t /FA_\q_-T4-'-_.----, --7--- th eInsideFM,'frethequeueinstantaneousirs form thefrequencyspec trumof directlya carrier. wave _< -0 'OJJ______!__J--_J/--J-- _'-'jj-_ --'71 1 _-J is varied according to a modulating wave, such that the g_-0z_\J,____'-/J/I I ,x, _¥ I/I J I'q J I rate at which the carrier varies is the frequency of the _ o'3J__j__[_[ i_f i i [ J'"¢/l_' k_W_l -,-_ T] modulating wave, or modulating frequency. The amount cc_04! F ;_)__--_ _ I I I _ I { I ! I the carrier varies around its average, or peak frequency - n I _. , 3. 4 5 0 N17odu8_a_,io}9 ni0Indext1.12 13 14 15 15 17 18 deviation, is proportional to the amplitude of the modu- _ o7 lating wave. The parameters of a 'frequency-modulated -_ 0_- i I [ i I I I I ! I I t [ I [ I _ °_I--F-Y-,_I t-I t I I t ]IFT-FD'_I md== peakmodulatingdeviation.frequency _ __J I/+,,)5-J /-I t I I The equationC_Ca_ierfor frequencya frequency-modulatedoravera_efrequencywave of peak _ 0I____[ _ / i J_l _ J / iJ J amplitude ,4 where both the carrier and modulating _ n[ I I I I :\_X_} f {,/_ j lx_ Xx i//I j, I waves are sinnsoids is -c_-_-_l}_'[V_-_ 't *_ ---,I--_q-,7/-q-__-5,/7---17_: '..... /_2_5Z e = Z sin(at + I sin_t) (1) (-o]_.-_.__-p?-i-Wj--j i i i i i t _ l00 I Z 3 4 5 6 7 8 g 10 II 12 13 14 15 16 17 18 where / lvlodula_, ion Index 07,' e = the instantaneous amplitude of the modu- _ 0.6_l._] j 'J If...... jl J ! i I I i _EL lated carrier ._' II I Fo.r?,'_-or_r _,_-b_' co,_,*e,*_z _+____ a: the carrier frequency in rad/s i /gI = thed/m----themodulatingmodulationfrequency index,in radthe/s ratio of _ _l _--_i--_'____'J, M_ - ' , ...... ' j I t [_ -- the peak deviation to the modulating fre- '_ 0'2 I [ lj_._/ _ i 1,__ 05;I'[--'',N,.j--Il! --/'_'J J Itmustis oalsobviousbe zerothat andwhenthere, =is 0nothemodfrequencyolation. Wdeviatihen oInis _-01 -_ 1_ ,] '.._R ! _-_-_7 t ' / I i X! _ i greater than zero, however, frequencies occur above and _ n'_ *_ _--I_ I .'_-_-_ I___1_.-_--_, ¢-_'_I-- II //I' I-i\--' ' [AilN'_" -- below the carrier frequency at intervals of the modulat- = l , _ I I I I I I L-FY ingfrequency. The number of side frequencies which oc- _-°x0 I z 3 4 5 _- 7 8 _ _0 _ _z I_ 14 15 ts _7 _s cur is related to the modulation index in such a way that t.:o4_]___o_,I i [)__[ as I increases from zero, energy is "stolen" from the car- Fig. 2. Bessel functions which determine the amplitudes rier and distributed among an increasing number of side of --the0I_sidebandiicomponents. SEPTEMBER 1973, VOLUME 21, NUMBER 7 527 JOHN M. CHOWNING AMPLITUDE will add to the first upper-sidefrequency with unlike I signs, decreasingthe energy at 200 Hz. The spectrum, _, adjusted for the reflected frequency components and _=4 I withthe bars all up to revealthe spectralenvelope,is I It : III _ _ FREQ. .own c+m HARMONIC AND INHARMONIC SPECTRA ,_ [ ¢-2mc-m [ I' c+zmc+3m c-3m ¢ The significance of the case above, where the ratio of the carrier to the modulating frequencies is 1/1, is Fig. 3. Plot of Fig. le with phase information included (modulation index = 4). The bars extending downward that it is a member of the class of ratios of integers represent spedtral components whose phases differ by 180°. (rational numbers), thus c/m = Nx/N 2 All of the above relationships are expressed in the and N_ and N2 are integers. These ratios result in har- trigonometric expansion of Eq. [2] monic spectra. If in addition, all common factors have e = A { J0(l) sinat been divided out of N_ .and N2, then the fundamental +Jl(1) [sin(a+ /_)t--sin(a-- fi)] frequency of the modulated wave will be + Jo(I) [sin(a + 2fi)t + sin(a -- 2fi) ] +Ja(I) [sin(a+ 3/g)t-- sin(a-- 3fi)] f0 = c/N1 = m/N2. + .................................................... ). (2) The position of the side frequencies in the harmonic It can be seen in Eq.