Evaluation of Modern Sound Synthesis Methods
Helsinki University of Technology
Department of Electrical and Communications Engineering
Lab oratory of Acoustics and Audio Signal Pro cessing
Evaluation of Mo dern Sound
Synthesis Metho ds
Tero Tolonen Vesa V lim ki and Matti Karjalainen
Rep ort
March
ISBN
ISSN
Esp o o
Table of Contents
Intro duction
Abstract Algorithms Pro cessed Recordings and Sampling
FM Synthesis
FM Synthesis Metho d
Feedback FM
Other Developments of the Simple FM
Waveshaping Synthesis
Karplus Strong Algorithm
Sampling Synthesis
Lo oping
Pitch Shifting
Data Reduction
Multiple Wavetable Synthesis Metho ds
Asynhcronous Granular Synthesis
Pitch Synchronous Granular Synthesis
Other Granular Synthesis Metho ds
Sp ectral Mo dels
Reduction of Control Data in Additive Synthesis by Line
Segment Approximation
The Phase Vocoder
Source Filter Synthesis
McAulay Quatieri Algorithm
Time Domain Windowing
Computation of the STFT
Detection of the Peaks in the STFT
Removal of Comp onents below Noise Threshold Level
Peak Continuation
Peak Value Interp olation and Normalization
AdditiveSynthesis of Sinusoidal Comp onents
Sp ectral Mo deling Synthesis
SMS Analysis
SMS Synthesis
Mo deling Synthesis Transient
Transient Mo deling with Unitary Transforms
TMS System
Inverse FFT FFT Synthesis
FormantSynthesis
FormantWave Function Synthesis and CHANT
VOSIM
Physical Mo dels
Equation Numerical Solving of the Wave
Damp ed Sti String
Di erence Equation for the Damp ed Sti String
The Initial Conditions for the Plucked and Struck String
Boundary Conditions for Strings in Musical Instruments
Vibrating Bars
Results Comparison with Real Instrument Sounds
Mo dal Synthesis
Mo dal Data of a Substructure
Synthesis using Mo dal Data
Application to an Acoustic System
Mass Spring Networks the CORDIS System
Elements of the CORDIS System
Comparison of the Metho ds Using Numerical Acoustics
Digital Waveguides and Extended Karplus Strong Mo dels
Digital Waveguides
Waveguide for Lossless Medium
Waveguide with Disp ersion and Frequency Dep endent Damping
Applications of Waveguides
Waveguide Meshes
Scattering Junction Connecting N Waveguides
Two Dimensional Waveguide Mesh
Analysis of Disp ersion Error
Single Delay Lo op Mo dels
Waveguide Formulation of a Vibrating String
Single Delay Lo op Formulation of the Acoustic Guitar
Single Delay Lo op Mo del with Commuted Bo dy Resp onse
Commuted Mo del of Excitation and Bo dy
General Plucked String Instrumen t Mo del
Analysis of the Mo del Parameters
Evaluation Scheme
Usabilityofthe Parameters
Qualityand Diversity of Pro duced Sounds
Implemention Issues
Evaluation of Several Sound Synthesis Metho ds
Evaluation of Abstract Algorithms
FM synthesis
Waveshaping Synthesis
Karplus Strong Synthesis
Evaluation of Sampling and Pro cessed Recordings
Sampling
Multiple Wavetable Synthesis
Granular synthesis
Evaluation of Sp ectral Mo dels
Basic AdditiveSynthesis
FFT based Phase Vocoder
McAulay Quatieri Algorithm
Source Filter Synthesis
Sp ectral Mo deling Synthesis
Transient Mo deling synthesis
FFT
FormantWave Function Synthesis
VOSIM
Evaluation of Physical Mo dels
Finite Di erence Metho ds
Mo dal Synthesis
CORDIS
Digital Waveguide Synthesis
Waveguide Meshes
veguide Synthesis Commuted Wa
Results of Evaluation
Summary and Conclusions
Bibliography
List of Figures
Three FM systems
Frequency domain presentation of FM synthesis
A comparison of three di erent FM techniques
Waveshaping with four di erent shaping functions
The Karplus Strong algorithm
Frequency resp onse of the Karplus Strong mo del
Time varying additivesynthesis after Roads
The additive analysis technique
The line segment approximation in additivesynthesis
The phase v ocoder
Source lter synthesis
An example of zero phase windowing
An example of the peak continuation algorithm
An example of p eak picking in magnitude sp ectrum
A detail of phase sp ectra in a STFT frame
Additive synthesis of the sinusoidal signal comp onents
The analysis part of the SMS technique after Serra and Smith
The synthesis part of the SMS technique after Serra and Smith
An example of TMS An impulsive signal top is analyzed
An example of TMS A slowly varying signal top is analyzed A
an DFT magnitude in b ottom is DCT middle is computed and
p erformed in the DCT representation
A blo ck diagram of the transient mo deling part of the TMS system
after Verma et al
Atypical FOF
The VOSIM time function N b A M and
T ms
Illustration of the recurrence equation of nite di erence metho d
Mo dels for b oundary conditions of string instruments
A mo dal scheme for the guitar
A mo del of a string according to the CORDIS system
d Alemb ert s solution of the wave equation
The one dimensional digital waveguide after Smith
aveguides Lossy and disp ersive digital w
A scattering junction of N waveguides
Blo ck diagram of a D waveguide mesh after Van Duyne and Smith
a
Disp ersion in digital waveguides
Dual delay line waveguide mo del for a plucked string with a force
output at the bridge
A blo ck diagram of transfer function comp onents as a mo del of the
plucked string with force output at the bridge
The principle of commuted waveguide synthesis
An extended string mo del with dual p olarization vibration and sym
pathetic coupling
An example of the e ect of mistuning the p olarization mo dels
An example of sympathetic coupling
List of Tables
Criteria for the parameters of synthesis metho ds with ratings used in
the evaluation scheme
Criteria for the quality and diversity of synthesis metho ds with ratings
used in the evaluation scheme
Criteria for the implementation issues of synthesis metho ds with rat
ings used in the evaluation scheme
Tabulated evaluation of the sound synthesis metho ds presented in
this do cument
Abstract
In this rep ort several digital sound synthesis metho ds are describ ed and evaluated
The metho ds are divided into four groups according to a taxonomy prop osed by
Smith Representative examples of sound synthesis techniques in each group are
chosen The evaluation criteria are based on those prop osed by Ja e The selected
synthesis metho ds are rated with a discussion concerning each criterion
Keywords sound synthesis digital signal pro cessing musical acoustics computer
music
Preface
The main part of this work has b een carried out as part of phase I of the TEMA
Testb ed for Music and Acoustics pro ject that has b een funded within Europ ean
Union s Op en Long Term Research ESPRIT program The duration of phase I was
months during the year This rep ort discusses digital sound synthesis metho ds
As Deliverable a of the TEMA pro ject phase I it aims at giving guidelines for the
second phase of the pro ject for the development of a sound synthesis and pro cessing
environment
The partners of the TEMA consortium in the phase I of the pro ject were Helsinki
UniversityofTechnology Staatliches Institut f r Musikforschung Berlin Germany
the University of York United Kingdom and SRF PACT United Kingdom In
the Helsinki University of Technology HUT two lab oratories were involved in
TEMA pro ject the Lab oratory of Acoustics and Audio Signal Pro cessing and the
Telecommunications and Multimedia Lab oratory
The authors would like to thank Professor Tapio Tassu Takala for his sup
port and guidance as the TEMA pro ject leader at HUT We are also grateful to
Dr Dr Ioannis Zannos who acted as the co ordinator of the TEMA pro ject and
representatives of other partners for smo oth collab oration and fruitful discussions
This rep ort summarizes and extends the contribution in the TEMA pro ject by
the HUT Lab oratory of Acoustics and Audio Signal Pro cessing We w ould like to
acknowledge the insightful comments on our manuscript given by Professor Julius
O Smith CCRMA Stanford University California USA Dr Davide Ro cchesso
and Professor Giovanni de Poli b oth at CSC DEI University of Padova Padova
Italy
Esp o o March
Tero Tolonen Vesa V lim ki and Matti Karjalainen
Intro duction
Digital sound synthesis metho ds are numerical algorithms that aim at pro ducing
musically interesting and preferably realistic sounds in real time In musical ap
plications the input for sound synthesis consists of control events only Numerous
di erent approaches are available
The purp ose of this do cument is not to try to reach the details of every metho d
Rather we attempt to give an overview of several sound synthesis metho ds The
second aim is to establish the tasks or synthesis problems that are b est suited for a
given metho d This is done byevaluation of the synthesis algorithms Wewould like
to emphasize that no attempt has been made to put the algorithms in any precise
order as this in our opinion would b e imp ossible
The synthesis algorithms were chosen to b e representative examples in each class
With each algorithm an attempt was made to givean overview of the metho d and
to refer the interested reader to the literature
The approach followed for evaluation is based on a taxonomy by Smith
Smith divides digital sound synthesis metho ds into four groups abstract algorithms
pro cessed recordings sp ectral mo dels and physical mo dels This do cument follows
Smith s taxonomy in a slightly mo di ed form and discusses representative metho ds
from each category
Each metho d was categorized into one of the following groups abstract algo
rithms sampling and pro cessed recordings sp ectral mo dels and physical mo dels
More emphasis is given to sp ectral and physical mo deling since these seem to pro
vide more attractive future prosp ects in high quality sound synthesis In these last
categories there is more activity in research and in general their future p otential
lo oks esp ecially promising
This do cument is organized as follows Selected synthesis metho ds are presented
in Chapters After that evaluation criteria are develop ed based on those
prop osed by Ja e An additional criterion is included concerning the suit
evaluation results ability of a metho d for distributed and parallel pro cessing The
are collected in a table in which the rating of each metho d can be compared The
do cument is concluded with a discussion of the features desirable in an environment
in which the metho ds discussed can b e implemented
Chapter Intro duction
Abstract Algorithms Pro cessed
Recordings and Sampling
The rst exp eriments that can be interpreted as ancestors of computer music were
done at s by comp osers like Milhaud Hindemith and Toch who exp erimented
with variable sp eed phonographs in concert Roads In Pierre Schae er
founded the Studio de Musique Concr te in Paris Roads In musique concr te
the comp oser works with sound elements obtained from recordings or real sounds
The metho ds presented in this chapter are based either on abstract algorithms or
on recordings of real sounds According to Smith these metho ds may b ecome
less common in commercial synthesizers as more p owerful and expressive techniques
arise However they still serve as a useful background for the more elab orate sound
synthesis metho ds Particularly they may still prove to b e sup erior in some sp eci c
sound synthesis problems e g when simplicity is of highest imp ortance and we are
likely to see them in use for decades
The chapter starts with three metho ds based on abstract algorithms FM synthe
sis waveshaping synthesis and the Karplus Strong algorithm Then three metho ds
utilizing recordings are discussed These are sampling m ultiple wavetable synthesis
and granular synthesis
FM Synthesis
FM frequency modulation synthesis is a fundamental digital sound synthesis tech
nique employing a nonlinear oscillating function FM synthesis in a wide sense con
sists of a family of metho ds eachofwhich utilizes the principle originally intro duced
by Chowning
The theory of FM was well established by the mid twentieth century for radio
frequencies The use of FM in audio frequencies for the purp oses of sound synthesis
was not studied until late s John Chowning at Stanford Universitywas the rst
to study systematically FM synthesis The time variant structure of natural sounds
is relatively hard to achieve using linear techniques such as additive synthesis see
that complex audio sp ectra can be achieved with section Chowning observed
just two sinusoidal oscillators Furthermore the synthesized complex sp ectra can
Chapter Abstract Algorithms Pro cessed Recordings and Sampling
be varied in time
FM Synthesis Metho d
In the most basic form of FM two sinusoidal oscillators namely the carrier and
the mo dulator are connected in such a way that the frequency of the carrier is
mo dulated with the mo dulating waveform A simple FM instrument is pictured in
Figure a The output signal y n of the instrument can b e expressed as
y n A n sin f n I sin f n
c m
f is the carrier frequency I is the mo dulation index where A n is the amplitude
c
and f is the mo dulating frequency The mo dulation index I represents the ratio
m
of the peak deviation of mo dulation to the mo dulating frequency It is clearly seen
that when I the output is the sinusoidal y n A n sin f n corresp onding
c
to zero mo dulation Note that there is a slight discrepancy between Figure
a and Equation since in the equation the phase and not the frequency is
b eing mo dulated However since these presentations are frequently encountered
Poli Roads they are also adopted here in literature e g in De
Holm and Bate discuss the e ect of phase and di erences between
implementations of the simple FM algorithm
I f m f f c FD1
M f FD2
A ( n ) A ( n ) FD
A n FDC( ) β
y ( n ) y ( n ) y (n ) FD1 FD2
(a) (b)
Figure a A simple FM synthesis instrument b one oscillator feedback
system with output y n and two oscillator feedback system with output y n
FD FD
after Roads
The expression of the output signal in Equation can be develop ed further
FM Synthesis
Chowning De Poli
X
J I sinj f kf n j y n
k c m
k
where J is the Bessel function of order k Insp ection of Equation reveals that
k
the frequency domain representation of the signal y n consists of a p eak at f and
c
additional p eaks at frequencies
f f nf n
n c m
as pictured in Figure Part of the energy of the carrier waveform is distributed
to the side frequencies f Note that Equation allows the partials b e determined
n
analytically
A
f
f - 2 f f - f f f + f f + 2 f
c m c m c c m c m
Figure Frequency domain presentation of FM synthesis
A harmonic sp ectrum is created when the ratio of carrier and mo dulator fre
quency is a member of the class of ratios of integers i e
N f
c
N N Z
f N
m
Otherwise the sp ectrum of the output signal is inharmonic Truax discusses
the mapping of frequency ratios into sp ectral families
Feedback FM
In simple FM the amplitude ratios of harmonics vary unevenly when the mo dulation
index I is varied Feedback FM can b e used to solve this problem Tomisawa
Two feedback FM systems are pictured in Figure b The one oscillator feedback
FM system is obtained from the simple FM by replacing the frequency mo dulation
Chapter Abstract Algorithms Pro cessed Recordings and Sampling
I = 12, b = 1.5 I = 10, b = 0.8 0 0
−50 −50
0 0.05 0.1 0.15 0.2 0 0.05 0.1 0.15 0.2 0 0
−50 −50
0 0.05 0.1 0.15 0.2 0 0.05 0.1 0.15 0.2 0 0
−50 −50
0 0.05 0.1 0.15 0.2 0 0.05 0.1 0.15 0.2 normalized frequency normalized frequency
(a) (b)
Figure A comparison of three di erent FM techniques Sp ectra of one oscillator
feedback FM are presented on top those of two oscillator feedback FM in the middle
and sp ectra of simple FM on the b ottom The frequency values f and f of
FD FD
the oscillators in the feedback system are equal The mo dulation index M is set to
Parameter b is the feedback co e cient
oscillator by a feedback connection from the output of the system The two oscillator
system uses a feedback connection to drive the frequency mo dulation oscillator
Figure shows the e ect of the feedback connections The sp ectra of signals
pro duced by the two feedback systems as well as the sp ectra of the signal pro duced
by the simple FM are computed for two sets of parameters in gures a and
b The more regular b ehavior of the harmonics in the feedback systems is clearly
visible Furthermore it can be observed that the two oscillator system pro duces
more harmonics for the same parameters
Other Developments of the Simple FM
Roads gives on overview of di erent metho ds based on simple FM The rst
commercial FM synthesizer the GS digital synthesizer was intro duced byYamaha
after developing the FM synthesis metho d patented byChowning further The rst
synthesizer was very exp ensive and it was only after intro duction of the famous
DX synthesizer that FM b ecame the dominating sound synthesis metho d for years
and in SoundBlaster compatible computer It is still used in many synthesizers
sound cards chips and software Yamaha has patented the feedback FM metho d
Tomisawa
Waveshaping Synthesis
Waveshaping synthesis also called nonlinear distortion is a simple sound synthesis
metho d using a nonlinear shaping function to mo dify the input signal First exp er
Karplus Strong Algorithm
iments on waveshaping were made by Risset in Roads Ar b
and Le Brun develop ed indep endently the mathematical formulation of the
waveshaping Both also p erformed some empirical exp eriments with the metho d
In the most fundamental form waveshaping is implemented as a mapping of a
sinusoidal input signal with a nonlinear distortion function w Examples of these
mappings are illustrated in Figure The function w maps the input value x n
in the range to an output value y n in the same range Waveshaping can b e
very easily implemented by a simple table lo okup i e the function w is stored in a
table which is then indexed with x n to pro duce the output signal y n
Both Ar b and Le Brun observed that the ratios of the harmon
ics could be accurately controlled by using Chebyshev p olynomials as distortion
functions The Chebyshev p olynomials have the interesting feature that when a
p olynomial of order n is used as a distortion function to a sinusoidal signal with
frequency the output signal will be a pure sinusoid with frequency n Thus
by using a linear combination of Chebyshev p olynomials as the distortion function
the ratio of the amplitudes of the harmonics can be controlled Furthermore the
signal can b e maintained bandlimited and the aliasing of harmonics can b e avoided
See Le Brun for discussion on the normalization of the amplitudes of the
harmonics
The signal obtained by the waveshaping metho d can be p ostpro cessed e g
by amplitude mo dulation This way the sp ectrum of the waveshap ed signal has
comp onents distributed around the mo dulating frequency f spaced at intervals f
m
If the sp ectrum is aliased an the frequency of the undistorted sinusoidal signal
inharmonic signal may b e pro duced See Ar b for more details and Roads
for references on other developments of the waveshaping synthesis
Karplus Strong Algorithm
Karplus and Strong develop ed a very simple metho d for surprisingly high
quality synthesis of plucked string and drum sounds The Karplus Strong KS
algorithm is an extension to the simple wavetable synthesis technique where the
sound signal is p erio dically read from a computer memory The mo di cation is
to change the wavetable each time a sample is b eing read A blo ck diagram of the
simple wavetable synthesis and a generic design of the Karplus Strong algorithm are
shown in Figure a and b resp ectively In the KS algorithm the wavetable is
initialized with a sequence of random numb ers as opp osed to wavetable synthesis
where usually a p erio d of a recorded instrument tone is used
The simplest mo di cation that pro duces useful results is to average two con
secutive samples of the wavetable as shown in Figure c This can be written
as
y n P y n P y n
Chapter Abstract Algorithms Pro cessed Recordings and Sampling
Input 1
0.5
0
−0.5
−1 −1 −0.5 0 0.5 1
Shaping function Output
1 1
0.5 0.5
0 0
−0.5 −0.5
−1 −1 −1 −0.5 0 0.5 1 −1 −0.5 0 0.5 1
1 1
0.5 0.5
0 0
−0.5 −0.5
−1 −1 −1 −0.5 0 0.5 1 −1 −0.5 0 0.5 1
1 1
0.5 0.5
0 0
−0.5 −0.5
−1 −1 −1 −0.5 0 0.5 1 −1 −0.5 0 0.5 1
1 4
0.5 2
0 0
−0.5 −2
−1 −4
1 05 0 05 1 1 05 0 05 1
Figure Waveshaping with four di erent shaping functions The input function
is presented on the top
Karplus Strong Algorithm
Output Wavetable -P (delay of P samples) z signal
(a)
Output Wavetable -P (delay of P samples) z signal
Modifier
(b)
Output Wavetable -P (delay of P samples) z signal
1 2
z-1 (c)
Output Wavetable -P (delay of P samples) z signal
1 2
+ or - with probability b z-1
(d)
Figure The Karplus Strong algorithm The simple wavetable synthesis is
shown in a a generic design with an arbitrary mo di cation function in b a
Karplus Strong mo del for plucked string tones in c and a Karplus Strong mo del
for p ercussion instrument tones in d after Karplus and Strong
Chapter Abstract Algorithms Pro cessed Recordings and Sampling
where P is the delay line length The transfer function of the simple mo di er lter
is
z H z
This is a lowpass lter and it accounts for the decay of the tone A multiply free
structure can b e implemented with only a sum and a shift for every output sample
This structure can be used to simulate plucked string instrument tones
The mo del for p ercussion timbres is shown in Figure d Now the output
sample y n dep ends on the wavetable entries by
n P y n P if r b
y n
n P y n P if r b
where r is a uniformly distributed random variable between and and b is a
parameter called the blend factor When b the algorithm reduces to that of
Equation When b drum liketimbres are obtained With b the entire
samples and a octave lower tone with odd harmonics signal is negated every p
only is pro duced
The KS algorithm is basically a comb lter This can b e seen by examining the
frequency resp onse of the algorithm In order to compute the frequency resp onse we
assume that we can feed a single impulse into the delay line that has b een initialized
with zero values We then compute the output signal and obtain a frequency domain
representation from the resp onse i e weinterpret the output signal as the impulse
resp onse of the system The corresp onding frequency resp onse is depicted in Figure
Notice the harmonic structure and that the magnitude of the peaks decreases
with frequency as exp ected
Karplus and Strong prop ose mo di cations to the algorithm including
excitation with a nonrandom signal A physical mo deling interpretation of the
Karplus Strong algorithm is taken by Smith and Ja e and Smith
Extensions to the Karplus Strong algorithm are presen ted in Section
Sampling Synthesis
Sampling synthesis is a metho d in which recordings of relatively short sounds are
played back Roads Digital sampling instruments also called samplers are
typically used to p erform pitch shifting lo oping or other mo di cation of the original
sound signal Borin et al b
Manipulation of recorded sounds for comp ositional purp oses dates back to the
s Roads Later magnetic tap e recording p ermitted cutting and splicing
of recorded sound sequences Thus editing and rearrangement of sound segments
was available In Pierre Schae er founded the Studio de Musique Concr te
at Paris and began to use tap e recorders to record and manipulate sounds Roads
Analog samplers were based on either optical discs or magnetic tap e devices
Sampling Synthesis
50
40
30
20 Magnitude (dB) 10
0
−10 0 0.2 0.4 0.6 0.8 1
Normalized frequency
Figure Frequency resp onse of the Karplus Strong mo del
Sampling synthesis typically uses signals of several seconds The synthesis itself
is very e cient to implement In its simplest form it consists only of one table
lo okup and p ointer up date for every output sample However the required amount
of memory storage is huge Three most widely used metho ds to reduce the memory
requirements are presented in the following They are lo oping pitch shifting and
data reduction Roads The interested reader should also consult a work by
Bristow Johnson on wavetable synthesis
Lo oping
One obvious way of reducing the memory usage in sampling synthesis is to apply
lo oping to the steady state part of a tone Roads With a number of instru
ment families the tone stays relatively constant in amplitude and pitch after the
attack until the tone is released by the player The steady state part can thus be
repro duced by lo oping over a short segment between so called lo op points After
the tone is released the lo oping ends and the sampler will play the decay part of the
tone
The samples provided with commercial samplers are typically pre lo op ed i e the
lo op p oints are already determined for the user For new samples the determination
metho d is to estimate the of the lo oping points has to be done by the user One
pitch of the tone and then select a segment of length of a multiple of the wavelength
of the fundamental frequency This kind of lo oping technique tends to create tones
with smo oth lo oping part and constant pitch Roads If the lo oping part
is to o short an arti cial sounding tone can be pro duced b ecause the time varying
Chapter Abstract Algorithms Pro cessed Recordings and Sampling
qualities of the tone are discarded The lo oping p oints can also b e spliced or cross
faded together A splice is simply a cut from one sound to the next and it is b ound
to pro duce a p erceivable click unless done very carefully In cross fading the end of
a lo oping part is faded out as the b eginning of the next part is faded in Even more
sophisticated techniques for the determination of go o d lo oping p oints are available
see Roads for more information
Pitch Shifting
In less exp ensive samplers there might not be enough memory capacity to store
every tone of an instrument or not all the notes have b een recorded Typically only
every third or fourth semitone is stored and the intermediate tones are pro duced
reduces by applying pitch shifting to the closest sampled tone Roads This
the memory requirements by a factor of three or four thus the data reduction is
signi cant
Pitch shifting in inexp ensive samplers is typically carried out using simple time
domain metho ds that a ect the length of the signal The two metho ds usually em
ployed are varying the clo ck frequency of the output digital to analog converter and
sample rate conversion in the digital domain Roads More elab orate metho d
for pitch shifting exist see Roads for references One wayofachieving sam
pling rate conversion is to use interp olated table reads with adjustable increments
This can b e done using fractional delay lters describ ed in V lim ki Laakso
et al
Data Reduction
In many samplers the memory requirements are tackled by data reduction tech
niques These can be divided into plain data reduction where part of the informa
tion is merely discarded and data compression where the information is packed into
a more economical form without any loss in the signal quality
Data reduction usually degrades the p erceived audio quality It consists of simple
but crude techniques that either lower the dynamic range of the signal by using less
bits to represent each sample or reduce the sampling frequency of the signal These
metho ds decrease the signal to noise ratio or the audio bandwidth resp ectively
More elab orate metho ds exit and these usually take into account the prop erties of
the human auditory system Roads
Data compression eliminates the redundancies present in the original signal to
represent the information more memory e ciently Compression should not degrade
the qualityofthe repro duced signal
Metho ds Multiple Wavetable Synthesis
Multiple wavetable synthesis is a set of metho ds that have in common the use of
Granular Synthesis
multiple wavetables i e sound signals stored in a computer memory The most
widely used metho ds are wavetable cross fading and wavetable stacking Roads
Horner et al intro duce metho ds obtaining optimal wavetables to
match signals of existing real instruments
In wavetable cross fading the tone is pro duced from several sections each con
sisting of a wavetable that is multiplied with an amplitude envelop e These p ortions
are summed together so that a sound event b egins with one wavetable that is then
cross faded to the next This pro cedure is rep eated over the course of the event
Roads A common way to use wavetable cross fading is to start a tone with
a rich attack such as a stroke or a pluck on a string and then cross fade this into
a sustain part of a synthetic waveform Roads
Wavetable stacking is a variation of the additive synthesis discussed in Section
Several arbitrary sound signals are rst multiplied with an amplitude envelop e
and then summed together to pro duce the synthetic sound signal Using wavetable
stacking hybrid timbres can be pro duced combining elements of several recorded
or synthesized sound signals In commercial synthesizers usually from four to eight
wavetables are used in wavetable stacking
In Horner et al metho ds for matching the time varying sp ectra of a
harmonic wavetable stacked tone to an original are presented The ob jective is to
nd wavetable sp ectra and the corresp onding amplitude envelop es that pro duce a
close t to the original signal First the original signal is analyzed using an extension
of the McAulay Quatieri McAulay and Quatieri see Section analysis
omponent analysis PCA are metho d A genetic algorithm GA and principal c
applied to obtain the basis sp ectra The amplitude envelop es are created by nding
a solution that minimizes a least squares error The metho d pro duced go o d results
with four wavetables when the GA was applied Horner et al
Granular Synthesis
Granular synthesis is a set of techniques that share a common paradigm of rep
resenting sound signals by sound atoms or grains Granular synthesis originated
from the studies by Gab or in the late s Cavaliere and Piccialli Roads
The synthetic sound signal is comp osed by adding these elementary units in
the time domain
In granular synthesis one sound grain can have duration ranging from one mil
lisecond to more than a hundred milliseconds and the waveform of the grain can b e
a windowed sinusoid a sampled signal or obtained from a physics based mo del of
a sound pro duction mechanism Cavaliere and Piccialli The granular tech
niques can be classi ed according to how the grains are obtained In the following
a classi cation derived from one given byCavaliere and Piccialli is presented
and the techniques of each category are shortly describ ed
Chapter Abstract Algorithms Pro cessed Recordings and Sampling
Asynhcronous Granular Synthesis
Asynchronous granular synthesis AGS has b een develop ed by Roads
It is a metho d that scatters sound grains in a statistical manner over a region in
the time frequency plane The regions are called sound clouds and they form the
elementary unit the comp oser works with Roads A cloud is sp eci ed by the
following parameters start time and duration of a cloud grain duration density
of grains amplitude envelop e and bandwidth of the cloud waveform of each grain
and spatial distribution of the cloud
of The grains of a cloud can all have similar waveforms or a random mixture
di erent waveforms A cloud can also mutate from grains with one waveform to
grains with another over the duration of the cloud The duration of a grain e ects
also its bandwidth The shorter a grain is the more it is spread in the frequency
domain Pitched sounds can be created with low bandwidth clouds Roads
gives a more detailed discussion on the parameters of the AGS
AGS is e ective in creating new sound events that are not easily pro duced by
musical instruments On the other hand simulations of existing sounds are very
hard to achieve with AGS In the following discussion granular synthesis metho ds
b etter suited for that are presented
Pitch Synchronous Granular Synthesis
Pitch synchronous granular synthesis PSGS is a metho d develop ed byDePoli and
Piccialli The metho d is also discussed in Cavaliere and Piccialli and
brie y in Roads In PSGS grains are derived from the short time Fourier
transform STFT The signal is assumed to b e nearly p erio dic and rst the fun
damental frequency of the signal is detected The p erio d of the signal is used as
the length of the rectangular window used in the STFT analysis When used syn
chronously to the fundamental frequency the rectangular window has the attractive
prop erty of minimizing the side e ects that o ccur with windowing i e the sp ectral
energy spread
After windowing a set of analysis grains are obtained in such a way that each
grain corresp onds to one p erio d of the signal From these analysis grains impulse
resp onses corresp onding to prominent content in the frequency domain representa
tion are derived Metho ds for the system impulse resp onse estimation include linear
predictive coding LPC and interp olation of the frequency domain representation
of a single p erio d of the signal Cavaliere and Piccialli
In the resynthesis stage a train of impulses is used to drive a set of parallel FIR
of the pulse train is lters obtained from the system impulse resp onse The period
obtained from the detected fundamental frequency See De Poli and Piccialli
for transformations that can create variations to the pro duced signal
Granular Synthesis
Other Granular Synthesis Metho ds
Some sound synthesis metho ds presented elsewhere in this do cument can also be
interpreted as sp ecial cases of granular synthesis These include the wave function
synthesis Section and VOSIM Section In fact all metho ds that use
the overlap add technique for synthesizing sound signals can b e thought ofasbeing
instances of granular synthesis
Another p ossibility is to apply the wavelet transform to obtain a multiresolution
representation of the signal See Evangelista for a discussion on wavelet
representations of musical signals
Chapter Abstract Algorithms Pro cessed Recordings and Sampling
Sp ectral Mo dels
Sp ectral sound synthesis metho ds are based on mo deling the prop erties of sound
waves as they are p erceived by the listener Many of them also take the knowledge
of psychoacoustics into account Sp ectral sound synthesis metho ds are general in
that they can be applied to mo del a wide variety of sounds
In this chapter three traditional linear synthesis metho ds namely additive syn
thesis the phase vocoder and source lter synthesis are rst discussed Second
McAulay Quatieri algorithm Spectral Modeling Synthesis SMS Transient Model
ing Synthesis TMS and the inverse FFT based additive synthesis metho d FFT
synthesis are describ ed Finally two metho ds for mo deling the human voice are
shortly addressed These metho ds are the CHANT and the VOSIM
Additive Synthesis
Additive synthesis is a metho d in which a comp osite waveform is formed by sum
ming sinusoidal comp onents for example harmonics of a tone to pro duce a sound
Mo orer It can b e interpreted as a metho d to mo del the time varying sp ectra
of a tone by a set of discrete lines in the frequency domain Smith
The concept of additive synthesis is very old and it has b een used extensively
in electronic music see Roads pp for references and historical
treatment In Risset applied the metho d for the rst time to repro duce
sounds based on the analysis of recorded tones With this application to trump et
tones the control data was reduced by applying piecewise linear approximation
of the amplitude envelop es Many of the mo dern sp ectral mo deling metho ds use
additivesynthesis in some form These metho ds are discussed later in this chapter A
blo ck diagram of additive synthesis with slowly varying control functions is depicted
in Figure
In additive synthesis three control functions are needed for every sinusoidal
phase of each comp onent In many cases oscillator the amplitude frequency and
the phase is left out and only the amplitude and frequency functions are used The
Chapter Sp ectral Mo dels
k = 0 1 2 M -1
F ( n ) k A ( n ) k
sin
Figure Time varying additive synthesis after Roads
output signal y n is the sum of the comp onents and can be represented as
M
X
y n A n sin F n
k k
k
where T is the sampling interval n is the time index M is the number of the
sinusoidal oscillator is the radian frequency of the oscillator A n is the time
k k
th th
varying amplitude of the k partial and F n is the frequency deviation of the k
k
partial If the tone is harmonic is a multiple of the fundamental radian frequency
k
i e k A n and F n are assumed to b e slowly time varying
k k k
The control functions can be obtained with several pro cedures Roads
One is to use arbitrary shap es for instance some comp osers have tracked the shap es
of mountains or urban sky lines The functions can also b e generated using comp o
sition programs An analysis metho d can be applied to map a natural sound into
a series of control functions Such a system is pictured in Figure The Short
Time Fourier Transform STFT is calculated from the input signal Harmonics
Additive Synthesis
x ( n )
STFT
F ( n ) k A ( n ) k
Frequency and amplitude trajectories in the time domain
Figure The additive analysis technique after Roads A STFT is
calculated from the input signal Frequency and amplitude tra jectories in the time
domain are formed
are mapp ed to p eaks in the frequency domain and their frequency and amplitude
functions can be detected from the series of STFT frames These control functions
can be used directly to synthesize tones in a system of Figure
The main drawbacks of the additive synthesis are the enormous amount of data
involved and the demand for a large number of oscillators The metho d gives best
little noise is results when applied to harmonic or almost harmonic signals where
present Synthesis of noisy signals requires a vast number of oscillators A metho d
for the reduction of control data is discussed in the following subsection
Reduction of Control Data in Additive Synthesis by
Line Segment Approximation
There are several ways to reduce the amount of control data needed Roads
p Mo orer The main criteria for the data reduction metho d are to
retain the intuitively app ealing form of the control data i e the comp oser has to
be able to easily mo dify the control data to obtain musically in teresting e ects on
sound and to preserve the original sound in the absence of transformation
Line segment approximation can be utilized to obtain a set of piecewise lin
ear curves approximating the frequency and the amplitude control functions The
Chapter Sp ectral Mo dels
metho d has been used by Risset and it is also discussed by Mo orer
and Strawn The idea of the line segment approximation is to t a set of
straight lines to each control function in such away that the curve obtained resem
bles the original curve An example of the line segment approximation is illustrated
th
in Figure where the amplitude tra jectory of the partial of a ute tone has
been approximated using line segments
Figure The line segment approximation in additivesynthesis
When partials of a tone are approximated by line segment approximation
using ten segments with bit numb ers for each partial the result is approximately
bits of data for a half second tone The same tone when using sampling rate
of Hz and bit samples amounts to bits Thus the data reduction
ratio is ab out to
The Phase Vocoder
The phase vocoder was develop ed at Bell lab oratories and was rst describ ed by
Flanagan and Golden All vo co ders present the input signal in multiple
parallel channels each of which describ es the signal in a particular frequency band
V o co ders simplify the complex sp ectral information and reduce the amount of data
needed to present the signal
In the original channel vo co der Dudley the signal is describ ed as an
excitation signal and values of short time amplitude sp ectra measured at discrete
frequencies The phase vocoder however uses complex short time spectra and thus
preserves the phase information of the signal
The analysis part of the metho d can b e considered to b e either a bank of bandpass
lters or a short time sp ectrum analyzer These viewp oints are mathematically
equivalent for in theory the original signal can b e repro duced undistorted Gordon
and Strawn Dolson Portno gives a mathematical treatment
on the sub ject and he also shows that the phase vo co der can be formulated as an
The Phase Vocoder
identity system in the absence of parameter mo di cations An intro ductory text of
the phase vo co der can b e found in Serra a
The implementation of the phase vocoder using the STFT is computationally
more e cient than using a lter bank since the complex sp ectrum can b e evaluated
with the fast Fourier transform FFT algorithm Detailed discussions on the phase
vo co der and practical implementations are given byPortno Mo orer
and Dolson Co de for implementing the phase vocoder can be found in
Gordon and Strawn and Mo ore
The phase vocoder is pictured in Figure The input signal is divided into
equally spaced frequency bands This can be done by applying the STFT to the
windowed signal Each bin of the STFT frame corresp onds to the magnitude and
phase values of the signal in that frequency band at the time of the frame
frequency analysis synthesis real imaginary
x n ( ) inverse FFT y ( n ) STFT with overlap- add
time
Figure The phase vocoder after Roads First the STFT is calculated
from the input signal The signal is now presented as a multiple series of complex
number pairs corresp onding to the signal comp onents in each frequency band The
output signal is comp osed by calculating the inverse FFT for each frame and by
using the overlap add metho d to reconstruct the signal in the time domain
Time scaling and pitch transp osition are e ects that can be easily p erformed
using the phase vocoder Dolson Serra a Time varying ltering can
also b e utilized Serra a Time scaling is done by mo difying the hop size in the
synthesis stage If the hop size is increased each STFT frame will e ectively sound
longer and the pro duced signal is a stretched version of the original If the hop size
is reduced the opp osite o ccurs The mo di cation of the hop size has to be taken
into account in the analysis stage by cho osing a window that minimizes the side
e ects Otherwise some output samples are given more weight and the synthetic
signal is amplitude mo dulated The phase values need also to be comp ensated for
in the mo di cation stage The phase values have to be multiplied by a scaling
factor in order to retain the correct frequency Pitch shifting without changing
the temp oral evolution can be accomplished by rst mo difying the time scale by
the desired pitch scaling factor and then changing the sampling rate of the signal
Chapter Sp ectral Mo dels
corresp ondingly See Serra a for examples and more details on time scale
mo di cations and problems that arise when the frequency resolution of the STFT
analysis is not su cient The problem of phasiness in time scale mo di cations
is discussed by Laro che and Dolson where also a phase synchronization is
prop osed
The phase vocoder works best when used with harmonic and static or slowly
changing tones It has di culties with noisy and rapidly changing sound signals
These signals can be mo deled b etter with a tracking phase vocoder or the sp ectral
mo deling synthesis describ ed in Section
Source Filter Synthesis
In source lter synthesis the sound waveform is obtained by ltering an excitation
signal with a time varying lter The metho d is sometimes called subtractive synthe
sis The technique has been used esp ecially to pro duce synthetic sp eech see e g
Mo orer for more details and references but also for musical applications
Roads
A blo ck diagram of the metho d is depicted in Figure The idea is to have a
broadband or harmonically rich excitation signal which is ltered to get the desired
output as opp osed to additive synthesis where the waveform is comp osed as a
of simple sinusoidal comp onents In theory an arbitrary p erio dic bandlimited sum
waveform can b e generated from a train of impulses by ltering Complex waveforms
are simple to generate by using a complex excitation signal A new metho d to
generate bandlimited pulse trains is intro duced by Stilson and Smith
a ( n ) b ( n )
White noise generator H ( z )
Time-varying
Impulse train filter
generator
Figure Source lter synthesis The transfer function of the time varying lter
H z is describ ed by lter co e cients a n and b n
The human voice pro duction mechanism can be approximated as an excitation
sound source feeding a resonating system When source lter synthesis is used to
synthesize sp eech the sound source generates either a p erio dic pulse train or white
noise dep ending on whether the sp eechisvoiced or unvoiced resp ectively The lter
P
K
k
b z
k
k
H z
P
L
l
a z
l
l
McAulay Quatieri Algorithm
mo dels the resonances of the vo cal tract The co e cients a n and b n of the lter
vary with time thus simulating the movements of lips the tongue and other parts
of the vo cal tract The periodic pulse train simulates the glottal waveform Many
traditional musical instruments have a stationary or slowly time varying resonating
system and source lter synthesis can be used to mo del such instruments The
metho d has also b een used in analog synthesizers When applied to sp eech or
singing the metho d can be interpreted as physical mo deling of the human sound
pro duction mechanism
The excitation signal and the lter co e cients fully describ e the output wave
form If only a wideband pulse train and noise is used it is enough to decide
between unvoiced and voiced sounds If the pulse form is xed only the p erio d i e
the fundamental frequency of the pulse train remains to b e determined
Detection of the pitch is not a trivial problem and it has b een studied extensively
mainly byspeech researchers Pitch detection metho ds can b e divided into ve cat
egories time domain metho ds auto correlation based metho ds adaptive ltering
frequency domain metho ds and mo dels of the human ear These are discussed
e g in Roads with references
The lter co e cients can be e ciently computed by applying linear predictive
LP analysis The basic idea of LP is that it is p ossible to design an all p ole lter
whose magnitude frequency resp onse closely matches that of an arbitrary sound
The di erence between STFT and LP is that LP measures the envelop e of the
magnitude sp ectrum whereas the STFT measures the magnitude and phase at a
large number of equally spaced p oints LP is a parametric metho d whereas STFT
is non parametric and LP is optimal in that it is the b est match of the sp ectrum in
the minimum squared error sense The metho d is discussed in detail by Makhoul
The fundamental frequency of the w aveform dep ends only on the fundamental
frequency of the pulse train So the timing and the fundamental frequency can
be varied indep endently Also the synthesis system can be excited with a complex
waveform thus creating new sounds that havecharacteristics of the excitation sound
as well as the resonating system
Although in theory arbitrary signals can be pro duced source lter synthesis
is not a very robust representation for generic wideband audio or musical signals
Ways to improvethe sound qualityare discussed by Mo orer
McAulay Quatieri Algorithm
An analysis based representation of sound signals suitable for additive synthesis has
been presented by McAulay and Quatieri The McAulay Quatieri MQ
algorithm originated from research of sp eech signals but it was already rep orted in
the rst study that the algorithm is capable of synthesizing a broader class of sound
signals McAulay and Quatieri Other algorithms of parameter estimation
Chapter Sp ectral Mo dels
for additive synthesis exist Risset Smith and Serra and the MQ and
related algorithms have b een utilized in many sp ectral mo deling systems Serra and
Smith Fitz and Haken
In the MQ algorithm the original signal is decomp osed into signal comp onents
th
that are resynthesized as a set of sinusoids The k signal comp onent at time
l l l
lo cation l is represented as a set of triplets fA g constituting three typ es of
k k k
tra jectories namely amplitude frequency and phase tra jectories that are used in
the synthesis stage The time lo cations l are determined by the hop size parameter
N of the STFT as
hop
l nN n
hop
The MQ algorithm can be programmed to adapt to the analysis signal e g the
numb er of detected signal comp onents and the hop size parameter can vary in time
The metho d is e cient in presenting harmonic or voiced signals with little noise
or transitions If noisy or unvoiced signals are to be repro duced a large number
of sinusoids is needed In the example describ ed in the study by McAulay and
Quatieri the maximum numb er of the detected p eaks was set to with the
waveforms sampling frequency of kHz and hop size of ms It was shown that
of harmonic signals are repro duced accurately whereas the repro duced waveforms
of noisy signals do not resemble the original well
The analysis part of the MQ algorithm uses the STFT to obtain a representation
for each signal comp onent The input signal x n is windowed in the time domain
to a length ranging typically between and ms A discrete Fourier transform
DFT is computed from the windowed signal x n Peaks in the complex sp ectrum
w
X n corresp onding to sinusoidal signal comp onents are detected and they are
w
used to obtain the amplitude frequency and phase tra jectories that comp ose the
signal representation The analysis steps are elab orated further in the following
subsections
Time Domain Windowing
Cho osing a prop er window function is a compromise b etween the width of the main
lob e frequency resolution of each signal comp onent and attenuation of the side
lob es spreading in the frequency domain A detailed discussion of window functions
is given by Harris and Nuttall In the original study McAulay and
Quatieri utilize a Hamming window In the frequency domain it has a dB
attenuation of the largest side lob e and an asymptotic decay of dB o ctave Nuttall
The same window function is also used in the examples presented in this
section
The length of the window function determines the time resolution of the analysis
The length of the Hamming window function should be at least times the
period of the fundamental frequency McAulay and Quatieri The window
length can be time varying to adapt to the analyzed signal
It is b ene cial to increase the frequency resolution of the DFT by increasing
McAulay Quatieri Algorithm
1 1 1
0.5 0.5 0.5
0 0 0 Amplitude −0.5 −0.5 −0.5
−1 −1 −1 −200 −100 0 100 200 0 200 400 0 500 1000
Time (samples) Time (samples) Time (samples)
Figure An example of zero phase windowing On the left a signal is windowed
ab out the time origin An equivalent signal for the DFT is displayed in the middle
In zero padding the zeros are inserted in the middle of the signal as shown on the
right
the length of the windowed signal by concatenating the windowed signal with zeros
Smith and Serra This is called zero padding and typically the length of
the windowed signal is increased to a power of two to allow for the use of the fast
Fourier transform FFT an e cient algorithm for the computation of the DFT
Note howev er that the frequency resolution in the analysis is further improved by
applying an interp olation scheme prop osed by Smith and Serra
For the detection of the phase values it is imp ortant to use zero phase windowing
to avoid a linear trend in the phase sp ectra Serra An example of zero phase
windowing is shown in Figure On the left a p ortion of a guitar signal is windowed
using a Hamming window with a length of samples and the windowed signal
is centered ab out the time origin at indices In practice the
circular prop erties of the DFT are used and the left half indices
indices as shown in the middle of of the signal is p ositioned at time
Figure On the right the signal is zero padded to a length of Notice that
the zeros are inserted in the middle of the wrapp ed signal
Computation of the STFT
The STFT is comp osed as a series of DFTs computed on windowed signal p ortions
separated in time by the hop size parameter N A value varying b etween N
hop win
and N is typically used for the hop size parameter where N is the length
win win
of the time domain analysis window
A DFT is p erformed on the zero phase windowed and zero padded signal x n
w
The DFT returns a complex sequence X k of length of the original signal Se
w
quence X k is a frequency domain representation of the signal and it is centered
w
around the frequency origin As a result of the analyzed signal b eing real the values
at p ositive and negative frequencies are complex conjugates i e X k X k
w
w
In the following we will only consider the values of X k at p ositive frequencies
w
Sequence X k is interpreted as magnitude and phase sp ectra of the windowed sig
w
nal by changing to polar coordinates An example of a single STFT frame is shown
in Figure where the magnitude top and phase b ottom sp ectra of a window ed
Chapter Sp ectral Mo dels
guitar signal are plotted
Detection of the Peaks in the STFT
The peaks in the magnitude sp ectrum corresp ond to prominent signal comp onents
that are mo deled as sinusoidal signals In general the determination of whether
a peak is a prominent one is rarely trivial In the case of an harmonic tone the
harmonic structure of the magnitude sp ectrum can be exploited The fundamen
tal frequency of the recorded signal is estimated and it su ces to search for the
lo cal maxima of each magnitude sp ectrum in the vicinities of the multiples of the
fundamental frequency
A peak detection is b est p erformed in the dB scale Serra A lo cal max
imum in the vicinity of a harmonic frequency can be detected by rst determining
th
the range of the search Typically a the p eak corresp onding to the k partial is
searched for in the range k f k f The maximum value in this range
is detected and if it is a lo cal maximum it is marked as a p eak This pro cedure is
carried out for every partial in every frame
Removal of Comp onents b elow Noise Threshold Level
The p eaks detected in the previous section will contain values that do not corresp ond
to the sinusoidal comp onents It is therefore essential to remove the detected p eaks
with values below a chosen noise threshold value Typically this value should be
frequency dep endent The noise level can b e estimated in the recorded signals e g
in the pauses of sp eech
If a single tone is analyzed the sinusoidal comp onents can be detected starting
from the end of the signal Smith and Serra Then the amplitude values can
be set to a zero value b efore a distinctive signal comp onent is found
Peak Continuation
After the p eaks b elow the noise threshold level are removed a p eak continuation
algorithm is utilized to pro duce the amplitude and frequency tra jectories corre
sp onding to the sinusoidal comp onents of the original signal It is assumed that the
sinusoids are fairly stationary b etween frames and thus the algorithm assigns a p eak
for an existing tra jectory if their frequency values are close enough A parameter for
the maximum frequency deviation f between consecutive frames is used as a lim
D
iting criterion If there is no existing tra jectory for that comp onent in the previous
frame a new tra jectory is started i e it is b orn McAulay and Quatieri
This is done by creating a triplet in the previous frame with zero amplitude the
same frequency and a phase value that is computed by subtracting a phase shift in
one frame from the detected phase value Similarly if no p eak matching an existing
tra jectory is found that tra jectory is killed McAulay and Quatieri In this
case a triplet with zero amplitude the same frequency and a shifted phase v alue is
inserted in the next frame
McAulay Quatieri Algorithm
f f f D D D
Frequency tracked Frequency track killed Frequency track born
Figure An example of the peak continuation algorithm after McAulay and
Quatieri On the left a match is found and the p eak is assigned to the track
In the middle no peak within the maximum deviation f is found and the track is
D
killed On the left a p eak is detected that do es not matchany p eaks in the previous
frame and a trackisborn
An example of the nearest neighbor p eak continuation algorithm is illustrated
in Figure On the left a frequency value is detected that is within the frequency
deviation threshold f and the peak is assigned to the corresp onding track In the
D
middle no peak with a frequency value within the limit is found and the track
is detected that do es not is killed On the right a new track is b orn i e a p eak
corresp ond to any of the p eaks in the previous frame
Peak Value Interp olation and Normalization
A b etter frequency resolution of the p eak detection can be obtained by applying
a parab olic interp olation scheme prop osed by Smith and Serra and detailed
by Serra In parab olic interp olation a parab ola is tted to the three p oints
consisting of the maximum and the adjacent values A p oint corresp onding to the
maximum value of the parab ola is detected The p oint yields the amplitude and the
frequency values of the corresp onding p eak The phase value is detected in the phase
sp ectrum by interp olating linearly b etween the adjacen t frequency p oints enclosing
the lo cation of the peak
An example of detection of the p eaks is shown in Figure The p eaks ab ove
dB in magnitude are detected and denoted with a cross in the magnitude
and phase sp ectra A zo om to the phase sp ectrum in Figure shows the e ciency
of the zero phase windowing The phase values are almost constant in the vicinityof
an harmonic comp onent This greatly reduces the estimation error of the detected
phase value
The e ect of the time domain windowing has to be comp ensated for in the am
plitude values The normalization factor c of the window function can b e computed
w
by solving Serra
N
X X
w m c w n c
w w
m
Chapter Sp ectral Mo dels
0
−20
−40
Magnitude(Db) −60
−80 0 2000 4000 6000 8000 10000
0
−2
−4
−6
−8
−10
−12 0 2000 4000 6000 8000 10000
Frequency
Figure Magnitude top and phase b ottom sp ectra corresp onding to a frame
in STFT The lo cations of p eaks corresp onding to harmonic comp onents are denoted
with A zo om to the dashed b ox in the phase sp ectrum is shown in Figure
4
2
Phase 0
−2
0 500 1000 1500
Frequency (Hz)
Figure A detail of phase sp ectra in a STFT frame Zero phase windowing
yields at p ortions of the phase sp ectrum in the vicinity of a harmonic comp onent
McAulay Quatieri Algorithm
l φ k Instantaneous l instantanious ω phase interpolation k phases Additive y ( n ) synthesis Amplitude Amplitude l A
k interpolation envelopes
Figure Additive synthesis of the sinusoidal signal comp onents after McAulay
and Quatieri Linear interp olation is used for the amplitude envelop e and
cubic interp olation for the instantaneous phase of each partial
which yields
c
w
N
X
w m
m
Furthermore in the DFT half of the energy of each sinusoid is on the negative
frequencies and thus the amplitude value of the detected peak has to b e multiplied
by a factor of The overall normalization factor is thus
c
N
X
w m
m
where w m the window function of length N
Additive Synthesis of Sinusoidal Comp onents
The additive synthesis of the sinusoidal signal comp onents is pictured in Figure
In this case a phase included additive synthesis is used i e the signal is
approximated as
N n
sig
X
x n x n A n cos n
k k
k
where A n is the amplitude envelop e and n is the instantaneous phase of the
k k
th
k signal comp onent Notice that the number of signal comp onents N n may
sig
dep end on time n This implies that the number of signal comp onents adapts to
the analyzed signal
The analysis stage provides the amplitude the frequency and the phase tra jec
l l l
tories of the signal comp onents The values of each triplet fA g corresp ond
k k k
th
to the detected values of amplitude frequency and phase of the k signal comp o
nent at frame l They are separated in time by an amount determined by the hop
size parameter N of the STFT The tra jectories have to be interp olated from
hop
frame to frame in order to obtain the amplitude envelop es and the instantaneous
Chapter Sp ectral Mo dels
l th
phases for additivesynthesis Amplitude tra jectory A of the k signal comp onent
k
is interp olated linearly from frame l to frame l to obtain instantaneous amplitude
l
l
A A
k l
k
m m N A m A
hop k
k
N
hop
This pro cedure is applied to all frame b oundaries to obtain the amplitude envelop es
A n for the additive synthesis
k
Both the detected frequency and phase a ect the instantaneous phase m
k
l l
l l
and that have to be Thus there are four variables namely
k k
k k
involved in the interp olation As prop osed by McAulay and Quatieri cubic
interp olation can b e used with
m m m m
k
This equation is solved as
l l
m m m m
k
k k
where
l l
l
T M M
N
N
hop k
k hop k
l
l
M
k
k
N N
hop hop
The value of M is chosen so that the instantaneous phase function is maximally
smo oth This is done by taking M to be the integer value closest to x when
McAulay and Quatieri
N
hop
l l l l l
x
k k
k k k
The instantaneous phase m is obtained by applying Equation to all frame
k
b oundaries The synthetic signal is now computed as
N n
sin
X
x n A n cos n
sin k k
k
The residual signal corresp onding to the sto chastic comp onent Serra is
obtained as
x n x n x n
res sin
The sto chastic signal contains information on b oth the steady state noise and rapid
transients in the signal
Sp ectral Mo deling Synthesis
The Sp ectral Mo deling Synthesis SMS technique was develop ed in the late s
at CCRMA Stanford University Serra develop ed a metho d for decomp osing
Sp ectral Mo deling Synthesis
a sound signal into deterministic and sto chastic comp onents The deterministic com
ponent can b e obtained by using the MQ algorithm McAulay and Quatieri
Section or by using a magnitude only analysis The deterministic part is sub
tracted from the original signal either in the time or the frequency domain to pro duce
a residual signal which corresp onds to the sto chastic comp onent The residual sig
nal can b e represented e ciently using metho ds discussed in this section In Serra
and Smith a detailed discussion of the magnitude only analysis synthesis is
given and a description of that system will be given here The metho d is also dis
cussed in Serra b The analysis scheme with phase included can be used to
obtain the residual signal by a time domain subtraction as discussed in Section
This metho d is used in various musical analysis applications including analysis of
recorded plucked string tones to derive prop er excitation signals for physical mo del
ing of plucked string tones Tolonen and V lim ki The interested reader is
also referred to work by Evangelista where awavelet representation is
intro duced that is suitable for representing separately pseudo p erio dic and ap erio dic
comp onents of a signal
The SMS technique is based on the assumption that the input sound can be
represented as a sum of two signal comp onents namely the deterministic and the
sto chastic comp onent By de nition a deterministic signal is any signal that is fully
predictable The SMS mo del however restricts the deterministic part to sinusoidal
comp onents with piecewise linear amplitude and frequency variations This a ects
the generality of the mo del and some sounds cannot be accurately mo deled by
the technique In the metho d the sto chastic comp onent is describ ed by its power
sp ectral density Therefore it is not necessary to preserve phase information of the
sto chastic comp onent The sto chastic comp onent can be e ciently represented by
the magnitude sp ectrum envelop e of the residual of each DFT frame
The SMS mo del consists of an analysis part and a synthesis part describ ed in
the following two subsections
SMS Analysis
from the time domain into the The analysis part is used to map the input signal
representation domain as is depicted in Figure The sto chastic representation is
given by the sp ectral envelop es of the sto chastic comp onent of the input signal The
envelop es are calculated from each DFT frame and they can b e e ciently describ ed
using a piece wise linear approximation Serra and Smith The deterministic
representation is comp osed of two tra jectories the frequency and the magnitude
tra jectory
The analysis part is fairly similar to that of the MQ algorithm The rst step is
to calculate the STFT of each windowed p ortion of the signal The STFT pro duces a
series of complex sp ectra from which the magnitude sp ectra is calculated From each
sp ectrum the prominent p eaks are detected and the p eak tra jectories are obtained
utilizing a peak continuation algorithm
The sto chastic comp onent is obtained by subtracting the deterministic comp o
Chapter Sp ectral Mo dels
input deterministic representation signal STFT Peak detection Additive synthesis frequency representation time time domain domain domain domain STFT stochastic representation envelope
approximation
Figure The analysis part of the SMS technique after Serra and Smith
nent from the signal in the frequency domain First the deterministic waveform is
computed from the p eak tra jectories Then the STFT of the deterministic waveform
is calculated similarly to the one obtained from the original signal By calculating
the di erence of the magnitude sp ectra of the input and the deterministic signal
the corresp onding magnitude sp ectrum of the sto chastic comp onent is obtained for
each windowed waveform p ortion The envelop es of these sp ectra are then approx
imated using a line segment approximation These envelop es form the sto chastic
representation
SMS Synthesis
The synthesis part of the tec hnique maps a signal from the representation domain
into the time domain This pro cess is illustrated in Figure The deterministic
comp onent of the signal is obtained by a magnitude only additive synthesis An
optional transformation can be used to alter the synthesized signal This allows
the pro duction of new sounds using the information of the analyzed signal for
example the duration of the signal temp o can be varied without changing the
peak frequencies key of the signal Similarly the frequencies can be transp osed
without in uencing the duration
computed from the sp ectral envelop es or their mo di The sto chastic signal is
cations by calculating an inverse STFT The phase sp ectra are generated using a
random number generator
The SMS metho d is very e cient in reducing the control data and computational
demands The metho d is general and can be applied to many sounds There are
some problems with the use of STFT is not su ciently well time lo calized and short
transient signals will b e spread in the time domain Go o dwin and Vetterli
In the next section a metho d for improving the accuracy on transient signals is
presented
Transient Mo deling Synthesis
Peak trajectories time domain Transformations Additive synthesis Synthesized representation domain waveform Spectral envelopes magnitude Transformations Polar to inverse- STFT Random number rectang. generator phase
frequency domain
Figure The synthesis part of the SMS technique after Serra and Smith
Transient Mo deling Synthesis
An extension to Sp ectral Mo deling Synthesis discussed in the previous section is
presented by Verma et al In this approach the residual signal obtained
by subtracting the sinusoidal mo del from the original signal is represented in two
parts transients and steady noisy comp onents Transient Mo deling Synthesis TMS
provides a parametric representation of the transient comp onents
TMS is based on the dualitybetween the time and the frequency domains Verma
et al Transient signals are impulsive in the time domain and th us they are
not in a form that is easily parameterizable However with a suitable transfor
mation impulsive signals are presented as frequency domain signals that have a
sinusoidal character This implies that sinusoidal mo deling can be applied in the
frequency domain to obtain a parametric representation of the impulsive signal
In the next subsection the principles utilized in TMS are presented Second the
structure of the TMS system is describ ed
Transient Mo deling with Unitary Transforms
The idea is to apply sinusoidal mo deling on a frequency domain signal that corre
sp onds to rapid changes in the time domain signal For sinusoidal mo deling we wish
to have a real valued signal Thus in this case the DFT is not an appropriate choice
discrete cosine transform DCT since it pro duces a complex valued sp ectrum The
provides a mapping in which an impulse in the time domain maps into a real valued
sinusoidinthe frequency domain The DCT is de ned as
N
X
n k
C k k x n cos n k N
N
n
where N is the length of the transformed signal x n Co e cients k are
q
fork
N
q
k
fork N
N
Chapter Sp ectral Mo dels
It is obvious from Equation that if x n n l the frequency domain
representation is
l
C k cos k
N
i e it is a sinusoid with a p erio d dep ending on the lo cation l of the time domain
impulse n l Thus Equation implies that impulsive time domain signals
e g corresp onding to attacks of tones pro duce a DCT that has strong sinusoidal
comp onents whereas steady state signals pro duce a DCT with little or no sinusoidal
comp onents
Equation is exempli ed in Figures and On the top and in the
middle of Figure an impulse like time domain signal and its DCT are illus
trated resp ectively The DCT is clearly a sinusoid with an amplitude envelop e that
aveform of the DCT can be represented by applying varies with frequency The w
sinusoidal mo deling Notice that in this case the sinusoidal analysis is p erformed
on a frequency domain signal On the b ottom of Figure the magnitude of the
complex valued DFT computed on the sinusoidal DCT is presented Notice that
only values corresp onding to the p ositive indexes of the DFT are shown This plot
shows that the p erio d of the DCT corresp onds to the lo cation of the impulse
To demonstrate the duality principle applied in TMS similar plots corresp onding
to a slowly varying signal are presented in Figure In this case an exp onentially
decaying sinusoid top pro duces an impulsive DCT middle Again the magnitude
of the DFT b ottom computed on the DCT closely follows the amplitude envelop e
of the original signal Observe that in b oth Figures and the DFT do es
not provide a parametric representation of the transients in the residual signal The
magnitude plots are only shown to clarify the unitary transforms applied in the
TMS
TMS System
As mentioned ab ove TMS is an extension to the SMS system discussed in Section
in that the residual signal is further decomp osed into two comp onents corresp onding
to transient and noisy parts of the original signal In this context only the extension
part of the TMS is presented A blo ck diagram of the system is illustrated in Figure
Verma et al First a blo ck DCT is computed on the residual signal
The length of the DCT blo ckischosen to b e su ciently large so that the transients
are compact entities within the blo ck A blo ck size of one second has found to be
agoodchoice Verma et al The transient detection blo ck is optional and it
can b e used to determine the regions of interest in the sinusoidal analysis The SMS
applied to the frequency domain DCT signal and the obtained representation is is
used to synthesize the transients and subtract them from the residual signal in the
time domain The residual signal is now expressed as comp onents corresp onding to
slowly varying noise and transients The analysis steps are elab orated further in the
following discussion
The transient detection blo ck is optional and the system can op erate without it
However it is useful since if the approximate lo cations of the transients in the time
Transient Mo deling Synthesis
1
0.8
0.6
0.4 Amplitude 0.2
0 0 50 100 150 200 250 Time (samples) 0.4
0.2
0 Amplitude −0.2
−0.4 0 50 100 150 200 250 Frequency (DCT bins) 10
8
6
4 Magnitude 2
0 0 20 40 60 80 100 120
Pseudo time (DFT bins)
Figure An example of TMS An impulsive signal top is analyzed A DCT
middle is computed and an DFT magnitude in b ottom is p erformed in the DCT
representation
Chapter Sp ectral Mo dels
1
0.5
0 Amplitude −0.5
−1 0 50 100 150 200 250 Time (samples) 3
2
1
0 Amplitude −1
−2 0 50 100 150 200 250 Frequency (DCT bins) 15
10
Magnitude 5
0 0 20 40 60 80 100 120
Pseudo time (DFT bins)
Figure An example of TMS A slowly varying signal top is analyzed A
DCT middle is computed and an DFT magnitude in b ottom is p erformed in the
DCT representation
Transient Mo deling Synthesis
Residual Sinusoids from SMS from SMS
Transient detection
Block Sinusoidal Representation DCT analysis/ of transients synthesis
Block IDCT
Representation Noise of noise
analysis
Figure A blo ck diagram of the transient mo deling part of the TMS system
after Verma et al
domain are known the sinusoidal mo deling op erating on the DCT can b e restricted
to only select those comp onents that corresp ond to the transient p ositions The
transients are detected in the residual signal by computing a ratio of the energies
of the residual and sinusoidal signals as a function of time Verma et al
In practice this is done within a DCT blo ck by rst computing the energies of the
sinusoidal and residual signals as
N N
X X
jx n E jx n j j and E
sin sin res res
n n
where N is the length of the DCT The instantaneous energies of the sinusoidal
and the residual signal are approximated by computing the energy within a short
window that is slid in time within the DCT blo ck This is expressed as
L
k
X
jx n j e k
sin sin
L
n k
and
L
k
X
e k jx n j
res res
L
n k
Chapter Sp ectral Mo dels
for k N N N where L is the length of the sliding window N
hop hop hop
is the hop size parameter and x n is the signal within the DCT blo ck zero padded
in a manner that it is de ned in the region of computation
The lo cations of the transients are determined to be in the vicinityof p ositions
k where the ratio of normalized instantaneous energies of the residual and the sinu
soidal signal is ab ove agiven threshold value This is expressed explicitly as
e k E
res res
R
thr
e k E
sin sin
After the lo cations of the transients have been detected the sinusoidal mo del
is restricted to estimating periodic sp ectral comp onents corresp onding to the esti
mated lo cations If the transient detection is not used SMS is applied on the whole
period range of the sp ectral representation The sp ectral mo deling parameters are
used to resynthesize the transient signal comp onents and subtract them from the
residual signal The obtained signal lacks the rapid variations and can therefore be
approximated as slowly varying noise
Inverse FFT FFT Synthesis
Inverse FFT FFT synthesis is presented in Ro det and Depalle a and
Ro det and Depalle b In this metho d additive synthesis is used in the
frequency domain i e all the signal comp onents are added together as sp ectral
envelop es comp osing a series of STFT frames The waveform can be constructed
by calculating the inverse FFT of each frame The overlap add metho d is used to
attach the consecutive frames to each other
Sinusoidal signals are simple to represent in the frequency domain A windowed
sinusoid in the frequency domain is a scaled and shifted version of the DFT of the
window function For the synthesis metho d to be computationally e cient the
DFT of the windowing function should have low sidelob es i e it should have few
signi cant values Ro det and Depalle a On the other hand the frequency
and the amplitude of the sinusoid in consecutive frames are linearly interp olated
This requirement yields a triangular window The DFT of a triangular window
A solution to this however has quite signi cant sidelob es and is not appropriate
problem is to use two windows one in the frequency domain and one in the time
domain Ro det and Depalle a
Using the FFT synthesis quasip erio dic signals can be easily comp osed The
parameters namely the frequency and the amplitude are intuitive although it is
useful to apply higher level controls in order to e ciently create complex sounds
with many partials It is straightforward to add additional noise of arbitrary shap e
in the frequency domain representation This is done by adding STFTs of desired
noise in frequency domain representation of the signal under construction Ro det
and Depalle a
Formant Synthesis
There are several metho ds to improve the problems which arise mainly from the
interp olation between consecutive frames These are discussed in Go o dwin and
Ro det and Go o dwin and Gogol
Formant Synthesis
In many cases it is useful to insp ect sp ectral envelop es i e a more general view of
the sp ectra instead of the ne details provided by the Fourier transform A central
concept of sp ectral envelop es is a formant which corresp onds to a p eak in the
envelop e of the magnitude sp ectrum A formant is thus a concentration of energy
in the sp ectrum It is de ned by its center frequency bandwidth amplitude and
envelop e Formants are useful for describing many musical instrument sounds but
they have been used extensively for synthesis of sp eech and singing See Roads
for more details and references on the use of formants in sound synthesis
In this section two sound synthesis metho ds based on formants are discussed
FormantWave Function synthesis is used in the CHANT program to pro duce high
quality synthetic singing VOSIM is a metho d for creating synthetic sound by trains
of pulses of a simple waveform These metho ds can also be interpreted as granular
synthesis metho ds as b oth of them use short grains of sounds to pro duce the output
signal See Section for a discussion and references of granular synthesis metho ds
Formant Wave Function Synthesis and CHANT
The formantwave function synthesis has b een develop ed at IRCAM Paris France
Ro det The metho d starts from the premise that the pro duction mechanism
of many of the real world sound signals can be presented as an excitation function
and a lter Ro det The metho d assumes that the lter is linear and the
excitation signal is comp osed of pulses of impulses or arches The fundamental fre
quency of the tone is then readily determined as the p erio d of the train of excitation
pulses In general the resp onse of the lter can b e interpreted as a sum of resp onses
h of which corresp onds to a formant in the synthe of a set of parallel lters eac
sized waveform The impulse resp onses of the parallel lters can be determined by
analyzing one p erio d of a recorded signal by linear prediction Ro det
The main elements of the formantwave function synthesis are the formant wave
functions French fonction d onde formantique FOF describ ed by Ro det
EachFOF corresp onds to a formant or a main mo de of the synthesized signal and it
is obtained by analyzing a recorded signal as explained ab ove FOFs are computed
in the time domain A typical FOF s k is pictured in Figure and it can be
written as
s k for k
k
s k cos k e sin k for k
k
s k e sin k for k
Chapter Sp ectral Mo dels
1
0.8
0.6
0.4
0.2
0 Amplitude −0.2
−0.4
−0.6
−0.8
−1 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
Time
Figure Atypical FOF
where is the center frequency is the dB bandwidth parameter governs
the skirt width and is the initial phase Naturally the amplitude of the FOF can
also b e mo di ed
AFOF synthesizer is constructed by connecting FOF generators in parallel The
synthesizer can b e controlled via instructions from the CHANT program The user
can utilize high level commands and achieve comprehensive control without having
to adjust the low level parameters directly
The CHANT program was originally written to pro duce high quality singing
voices but it can also be employed to synthesize musical instruments Ro det et
al It employs semiautomatic analysis of the sp ectrum of recorded sounds
extraction of gross formant characteristics and fundamental frequency estimation
Ro det The program is discussed in detail in Ro det et al Sound
examples of synthesized singing can be found in Bennett and Ro det
VOSIM
VOSIM VOice SIMulation is develop ed by Kaegi and Temp elaars It starts
from the idea of presenting a sound signal as a set of tone bursts that haveavariable
duration and delay
The pulses used in VOSIM are of xed waveform The VOSIM time function
consists of N pulses that are shap ed like squared sinusoids The pulses are of
from value A and followed equal duration T with decreasing amplitude starting
by a delay M Each pulse is obtained from the previous pulse by multiplying with
a constant factor b Such a time function is pictured in Figure The ve
Formant Synthesis
parameters presented ab ove are the primary parameters of VOSIM For vibrato
frequency mo dulation and noise sounds the delay M is mo dulated Three more
parameters are required S is the choice of random or sine wave D is the maximum
deviation of M and NM is the mo dulation rate Four additional variables allow
for transitional sounds NP is the numb er of transition p erio ds and DT DM and
DA the p ositiveor negative increments of T M and A resp ectively
1
0.9
0.8
0.7
0.6
0.5
0.4 Normalized amplitude
0.3
0.2
0.1
0 0 20 40 60 80 100 120
Time (ms)
Figure The VOSIM time function N b A M and
T ms
Chapter Sp ectral Mo dels
Physical Mo dels
Physical mo deling of musical instruments has evolved to one of the most active
elds in sound synthesis musical acoustics and computer music research Physical
mo deling applications gain p opularity by giving users b etter to ols for controlling
and pro ducing both traditional and new synthesized sounds The user is provided
with a sense of a real instrument
The aim of a mo del is to simulate the fundamental physical b ehavior of an actual
instrument This is done by employing the knowledge of the physical laws that
govern the motions and interactions within the system under study and expressing
them as mathematical formulae and equations These mathematical relationships
provide the to ols for physical mo deling
There are two main motivations for developing physics based mo dels The rst
is that of science i e mo dels are used to gain understanding of physical phenomena
The other is pro duction of synthesized sound From the days of rst physics based
mo dels researchers and engineers have utilized them for sound synthesis purp oses
Hiller and Ruiz a
Physical mo deling metho ds can be divided into ve categories V lim ki and
Takala
Numerical solving of partial di erential equations
Source lter mo deling
Vibrating mass spring networks
Mo dal synthesis
Wav eguide synthesis
Waveguide synthesis is one of the most widely used physics based sound syn
thesis metho ds in use to day It is very e cient in simulating wave propagation in
one dimensional homogeneous vibratory systems The metho d is very much digital
signal pro cessing oriented andanumb er of real time implementations using waveg
uide synthesis exists Waveguide synthesis and single delay lo op SDL mo dels are
discussed further in Chapter
Chapter Physical Mo dels
Source lter mo dels have been used esp ecially for mo deling the human sound
pro duction mechanism The interaction of the vo cal chords and the vo cal tract is
mo deled as a feedforward system E ective digital ltering techniques for source
lter mo deling have b een develop ed esp ecially for sp eech transmission purp oses
The technique is basically a physical mo deling interpretation of the source lter
synthesis presented in Section
The mo deling metho ds simulate the system either in the time or the frequency
domain The frequency domain metho ds are very e ective for mo dels of linear
systems Musical instruments cannot in general be approximated accurately as
b eing linear Nonlinear systems make the frequency domain approach infeasible
All the metho ds presented here mo del the system under study in the time domain
This chapter starts with describing three physical mo deling metho ds that use
numerical acoustics First mo dels using nite di erence metho ds are represented
Applications to string instruments as well as to mallet p ercussion instruments are
presented Second mo dal synthesis is discussed Third CORDIS is a system of
mo deling vibrating ob jects by mass spring networks
The interested reader is also referred to an interesting web site by De Poli and
Ro cchesso http www dei unipd it eng lish cs c pa pers dpr oc d proc htm l
Numerical Solving of the Wave Equation
In this section mo deling metho ds based on nite di erence equations will be dis
cussed These metho ds have been used esp ecially for string instruments
The metho d is in general applicable to any vibrating ob ject i e a string a
bar a membrane a sphere etc Hiller and Ruiz a The basic principle is
to obtain mathematical equations that describ e the vibratory motion in the ob ject
under study These wa ve equations are then solved in a nite set of points in
the ob ject thus obtaining a di erence equation The use of di erence equations
leads to a recurrence equation that can be interpreted as a simulation of the wave
propagation in the vibrating ob ject The nite di erence metho d is computationally
most e cient with one dimensional vibrators as the computational demands rapidly
increase with intro duction of more dimensions The number of p oints in space
increases prop ortional to the p ower of the number of dimensions Furthermore the
number of computational op erations for each p oint is increased and the e ective
meshes presented in Section sampling frequency is increased Digital waveguide
are DSP formulations of di erence equations in two and three dimensions
Hiller and Ruiz a were the rst to take the approach of solving the di er
ential equations of a vibrating string for sound synthesis purp oses They develop ed
mo dels of plucked struck and bowed strings The sti ness of the string was mo d
eled as well as the frequency dep endent losses Hiller and Ruiz b were able
to pro duce synthesized sound and plots of the resulting waveforms by a computer
program Since that pioneer work developments have been made in mo deling the
Numerical Solving of the Wave Equation
excitation e g the interaction of the hammer and the piano strings see Chaigne
and Askenfelt a for references
More recently Chaigne has been studying nite di erence metho ds with appli
cations to mo deling of the guitar the piano and the violin Chaigne et al
Chaigne and Askenfelt a Chaigne and Askenfelt b He has taken
similar approaches to mo deling a vibrating bar with application to the xylophone
Chaigne and Doutaut
In this section a di erence equation with initial and b oundary conditions for a
damp ed sti string is rst intro duced Then similar treatmentisgiven for a vibrating
bar Finally the synthesized waveforms are compared with original recordings of
real instrument sounds
Damp ed Sti String
The mo del of a vibrating string presented here includes mo deling the sti ness of
the string as well as the frequency dep endent losses due to the friction with air
viscosity and nite mass of the string It describ es the transversal wave motion of
the string in a plane The wave equation for the mo del is Chaigne and Askenfelt
a
y y y y y
c c L b b f x x t
t x x t t
where y is the displacement of the string x the axis along the string c the transverse
wave velo cityofthe string the sti ness parameter L the string length b and b
the loss parameters and f x x t the excitation acceleration applied at p oint x
The excitation term is actually a force density term normalized by the mass density
of the string so that the term will give acceleration of the string at point x
The sti ness parameter is given by
ES
TL
where is the radius of gyration of the string E the Young s mo dulus S the area
of the string cross section and T the string tension
In Equation the two partial time derivative terms of odd order mo del the
frequency dep endent losses i e the decay of the vibration The decay is an e ect of
several physical phenomena The e ect of each phenomenon can b e hard to separate
from the total decay and it will not be attempted here However some qualitative
interpretations can b e made In the low frequency range the main causes for losses
are the air resistance and the resistive imp edances at the ends of the string Chaigne
In the high frequency range the damping is mainly created byinternal losses
in the string such as the visco elastic losses in nylon strings discussed by Chaigne
The parameters for the losses b and b are obtained via the analysis of
real instrument tones The mo del do es not try to mo del the individual physical
pro cesses that cause the dissipation of energy separately The frequency dep endent
Chapter Physical Mo dels
decay rate is given by
b b
The string is excited by the force densitytermf x x t It is assumed that the
force density do es not propagate along the string thus time and space dep endence
can be separated in order to get
f x x t f t g x x
H
The term g x x can be understo o d as a spatial window which distributes the
excitation energy to the string This window smo othes the applied excitation e g
hammer strike on a piano string so that artifacts that o ccur in the solution b ecause
of discontinuities will b e eliminated
The force density term f t is related to the force F t exerted in the excitation
H H
by
F t
H
f t
R
H
x x
g x x dx
x x
where the e ective length of the string section interacting with the exciter is x
and is the linear mass density of the string
Di erence Equation for the Damp ed Sti String
The di erence equation for the sti damp ed string is obtained by discretizing the
time and space by taking Chaigne and Askenfelt a
L
x k x k
k
x
and
t n t n
n
The t and x are related by
t
r c
x
The condition r gives the exact solution with no numerical disp ersion Chaigne
However r equals unity only in the case of an ideal string For values
r numerical disp ersion will be present in the mo del This will not be dis
cussed further in this context see Chaigne for more details The main
variable of interest will be the discretized transversal string displacement denoted
y k n y k x n t for convenience The derivation of the di erence equation
approximating Equation is given by Hiller and Ruiz a and will not be
Numerical Solving of the Wave Equation
rep eated here However it should b e noted that for the sake of e ciency in compu
tation one further simpli cation is made The third order time derivative term in
Eq would yield the following approximation with time t n as central p oint
n
y
y k n y k n y k n y k n
t
i e the need for implicit metho ds This can be overcome by noticing that the
y
magnitude of the term b is relatively small and by reducing the number of
t
time steps by employing the recurrence equation for the ideal string
y k n y k n y k n y k n
Using this equation to simplify Eq will not increase the numb er of time or space
steps involved in the recurrence equation The general recurrence equation is now
given by
y k n a y k n a y k n
k n a y k n y
a y k n y k n
a y k n y k n y k n
t NF n g k i M
H S
where the co e cients a to a are given with Equations
a r b t N r D
a b t b t D
a r N D
a b t N r D
a b t D
where
D b t b t and r c t x
Figure shows the how displacement y k n dep ends on previous values of
This equation can b e directly utilized to compute displacement when using Eq
the displacements of the chosen discrete points on the string
The Initial Conditions for the Plucked and Struck String
The initial conditions are given for mo dels of the guitar and the piano The con
ditions are very dissimilar and corresp ond to the excitation by either plucking or
striking the string
Plucked String
Excitation byplucking is the simplest case and its initial conditions are given directly
by Equation The spatial window g x x and the string section a ected are
Chapter Physical Mo dels
n +1 n t n -1
n -2
kk-2-1k kk +1 +2
x
Figure Dep endence of displacement y k n on previous values of the dis
placement after Chaigne The next value marked with will dep end on
those p oints in time and space marked with
determined mainly bythetyp e of the pluck The velo cityofthepluck determines the
time distribution of the excitation Naturally these are not mutually indep endent
It may b e helpful to consider the plucking event as b eing mapp ed to a force density
distribution that can then be separated to parts dep ending on space and time
A more detailed mo del of the plucking event including mo deling the nger string
interaction is given by Chaigne
For the guitar the initial condition is intro duced by rewriting the last term on
the right hand side of Eq
N
F n g k i t
m
S
where N is the number of points on the string m is the mass of the string F n is
S
the force applied by nger or plectrum and g k i is the discretized spatial window
Chaigne et al
Struck String
For the development of an expression of the initial conditions for the piano an
assumption of zero initial velo city and displacement of the string is made by Chaigne
and Askenfelt a This assumption is made only for the sake of simplicity in
discussing the initial condition the mo del has no restrictions on the initial condition
With the string at rest at t we have
y k
assumption is needed for Equation to be applicable to the rst One further
three time steps The calculation involves the states of the string at three past time
Numerical Solving of the Wave Equation
steps Thus y k is estimated by using approximated Taylor series to obtain
y k y k
y k
Now for the displacement of the hammer at time n we calculate
V t
H
where V is the hammer velo cityatt and for the force exerted by the hammer
H
p
F K j y k j
H
Note that the force term at t is computed using the initial velo city i e a unit
delay is intro duced in order for the force to be computable Borin et al a
prop ose a more elab orate metho d for eliminating delay free lo ops in discrete time
mo dels Interestingly they also apply the metho d for mo deling the hammer string
interaction
Continuing with the treatment of Chaigne and Askenfelt a the displace
ment y k is computed using a simpli ed version of Eq
t NF
H
y k y k y k y k
M
H
Here the sti ness and damping terms are neglected in order to limit the space
and time dep endence i e no terms with n are included For the hammer
displacement and force F are computed by
H
t F
H
M
H
p
F K j y k j
H
The e ect of the simpli cations is discussed by Chaigne and Askenfelt a
After the displacements y k n are known to rst three time samples it is p os
sible to start using the recurrence formula of Eq directly The force F n is
H
assumed to be known and its e ect for the string is taken into account until time
n when
n y k n
After this the string is left free for vibrations unless recontact of the hammer is
mo deled The force density term f x x t can b e applied at the string at any time
o
Boundary Conditions for Strings in Musical Instruments
Terminations of strings in musical instruments are not completely rigid For exam
ple in the guitar the bridge has a nite imp edance and the nger terminating the
string against the ngerb oard is far from rigid The b oundary conditions are given
for the guitar the piano and the case of the violin is discussed brie y
The b oundary conditions for plucked b owed and struck string instruments like
the guitar the violin and the piano can be describ ed by one of the three mo dels
Chapter Physical Mo dels
N-1 N N+1 Bridge and clamping VBC N-1 N N+1
Bridge GBC N-1 N
Rigid end support PBC
x
Figure Mo dels for b oundary conditions of string instruments after Chaigne
VBC Violin like b oundary condition GBC Guitar like b oundary condition
PBC Piano like b oundary condition
presented in Figure Chaigne Here p oint N on the string corresp onds to
the p oint of the bridge For the violin like b oundary conditions it is assumed that
the displacement y N n of the string is non zero Furthermore the displacement
of the string at the point x N is taken to be much smaller than at x N
If the distance between the bridge and the clamping p osition is greater than the
the b oundary condition can be written as y N p n space step e g p x
instead of y N n Then the expressions for the intermediate p oints would
have to b e develop ed
In the guitar like b oundary condition the string is clamp ed just b ehind the
bridge so that the distance b etween the bridge and the clamping p osition is usually
ab ove the audible wavelength rangeofahuman Thus it can be assumed that
y N n y N n
i e y N n denotes the displacement of the string as well as the resonating box at
the bridge This allows for mo deling of the coupling of the bridge and the resonat
ing body by using measured values of the input admittance at the guitar bridge
Mo deling the resonances and the radiated sound pressure is discussed by Chaigne
The piano string is assumed to be hinged at b oth ends yielding the following
b oundary conditions Fletcher and Rossing
y t y L t
y y
t L t
x x
Numerical Solving of the Wave Equation
For the mo del of the piano the b oundary conditions are obtained by discretizing
Eqs and they can be expressed as
y n y N n
y n y n and y N n y N n
The string is coupled to the soundb oard at point N If the frequency dep endent
prop erties of the coupling e ect are desired the second condition in Eq can b e
replaced with a di erence equation approximating the di erential equation governing
the coupling Equation is imp ortantfor deriving expressions for string motion
at p oints y and y N for these p oints are not explicitly included in the
mo del These p oints are needed for the calculation of the displacement of the string
at p oints y and y N b ecause the di erential equation for the string is of
fourth order i e the recurrence equation for point k dep ends on p oints k and
k
Vibrating Bars
An approach similar to the case of string is taken by Chaigne and Doutaut
for the vibrating bar Theoretical treatment of vibrating bars is given by e g Morse
and Ingard and mallet p ercussion instruments are discussed byFletcher and
Rossing
It is assumed that the vertical comp onent w x t of the displacement of a xylo
phone bar is given by the two following equations
w x t
M x t EI x
t x
and
M x t w x t w x t
w x t f x x t
B
t S x x t M
B
M x t is the b ending momentand I x the moment ab out the x axis S x t is the
cross sectional area of the bar E is the Young s mo dulus and the density of the
vibrating bar The co e cients and account for losses They are obtained by
B
analyzing the decay times of partials on real instruments Estimation of the sti ness
co e cient is obtained by measuring the natural frequency of a spring mass system
comp osed of the bar with mass M and the supp orting cord
B
The mo del for the interaction between the bar and the mallet is similar to the
one used for the hammer string interaction in the piano mo del with force
F t
M
f t
R
H
x x
S x g x x dx
x x
where S x is the cross section of the bar at p oint x and is the density of the
bar The spatial smo othing of the impact is obtained by employing a spatial window
as in Eq
Chapter Physical Mo dels
The impact force is given by Eq with p The non integer exp onent
is now derived from the general theory of elasticity as opp osed to the case of the
piano where analysis of exp erimental data must b e used see Chaigne and Doutaut
App endix A for derivation The sti ness co e cient K is obtained by analysis
of exp erimental data
This interaction mo del is able to simulate three imp ortant physical asp ects of
the instrument
The intro duction of kinetic energy lo calized in time and space into the vi
brating system
The in uence of the initial velo city on both the contact duration and the
impact force due to the nonlinear force deformation law This determines the
tone sp ectrum of the
The in uence of the sti ness of the two materials in contact which strongly
determines the tone qualityofthe initial blow i e the attack
These principles apply to the mo del of the piano as well
For the numerical formulation of the xylophone mo del the same principles as
in the case of the string are employed However the explicit computation scheme
already used in the guitar and piano mo dels is applicable only to a simple case of a
uniform bar with constant cross sectional area This is the only mo del discussed in
this context Chaigne and Doutaut discuss also the more demanding mo del
of a bar with a variable section
is The di erential equation for the uniform bar
w x t w x t w x t
a
t x t x
w x t
w x t f x x t
B
t M
B
where a EI S
The recurrence equation approximating Eq is given by Chaigne and
Doutaut
y k n c w k n c w k n
c w k n w k n w k n w k n
w k n c w k n
w k n w k n
c F n g k i
M
Numerical Solving of the Wave Equation
where
r t r
B
c c
r r N
c c c
M f
B
s
where
N
B
and r a f
s
B
M f f L
B s s
It can be shown that the explicit scheme remains stable if the number of spatial
points N is Chaigne and Doutaut App endix B
s
f
s
N M
MAX
f f
s
where f is the frequency of the lowest partial For wo o den bars the order of
magnitude for the term f It can be seen that according to this stability
s
criterion the maximum numb er of spatial p oints is roughly prop ortional to the square
ro ot of the sampling frequency Thus to double the spatial resolution sampling
frequency of four times the original is required Furthermore it can be observed
p
that there is an asymptotic limit f for N as f increases The maximum
MAX s
spatial resolution obtained by using the sampling frequency of kHz is equal to
cm
A comparison of the original measured signals and those obtained with the mo del
of variable cross sectional area is discussed in the next section
Results Comparison with Real Instrument Sounds
The mo dels describ ed in the previous sections have been evaluated and compared
to real instruments by Chaigne and Askenfelt b Chaigne et al and
Chaigne and Doutaut This is imp ortant not only for the validation of the
mo dels but also for studying the contribution of each individual physical parameter
e ect of a single parameter on pro duced sound can be to the signal Typically the
hard to establish by observing the instrument or the pro duced sound
Only a short qualitative comparison between measured and simulated signals is
given in this section References to detailed presentations of each instrument are
given in the corresp onding subsection
The Piano
Chaigne and Askenfelt b give a detailed and systematic discussion on the
comparison of real signals to those obtained by simulation
Chapter Physical Mo dels
The string velo cities were computed for bass C midrange C and treble
C tones The overall result is that the mo del is capable of pro ducing the wave
forms quite well over the whole register of the piano including the attack transients
Some small discrepancies in the bass range can b e caused by the non rigid termina
tion of a real piano string The mo del do es not attempt to take this phenomenon
into account
The sp ectra of the string velo cities with notes played at di erent ranges with
di erent dynamics show a go o d b ehavior of the mo del The sp ectra show increased
sp ectral content with increased hammer velo city as exp ected Large and audible
di erences were observed ab ove the rst partials although these discrepancies
had little e ect to the waveforms
The Guitar
The guitar and the corresp onding nite di erence mo del are compared by Chaigne
et al The waveforms of a vibrating guitar string were obtained by a simple
electro dynamic metho d A concentrated magnetic eld was applied p erp endicular
to the vibrating string The generated voltage prop ortional to the string velo cityat
the pointofthe magnetic eld was measured between the string ends
It was observed that the measured and simulated waveforms were similar Fur
thermore the in uence of the b o dy resp onse was more clearly visible in the measured
signal
The Xylophone
The measurements and comparison were conducted by Chaigne and Doutaut
In this case the acceleration of the mallet s head was measured The corresp onding
force signal was derived bymultiplication by the equivalent mass of the mallet The
acceleration of the chosen p ointonthebarwas either measured with the help of an
accelometer or derived from the velo city signal obtained by a laser vibrometer
Two di erent typ es of mallets were simulated a soft mallet with rubb er head
and a hard mallet with boxwood head For b oth mallets signals of weak piano
and strong mezzo forte impact were measured and simulated Three comparisons
were made bar accelerations impact forces and bar acceleration sp ectra
soft mallet the general shap e and amplitude of the For a weak impact with a
waveform of the bar accelerations were similar However the upp er partials seemed
to b e damp ed more rapidly in the measured acceleration For a strong impact b oth
the magnitude and the shap e of the signals were very similar The mo del seems to
work b etter with hard mallets b ecause the bar acceleration waveforms showagood
match with b oth the weak and the strong impact
The magnitude of the impact forces on the bar showed that the order of magni
tude for both shap es and amplitudes are repro duced fairly well for the soft mallet
Mo dal Synthesis
The impact durations were systematically shorter by approximately With
hard mallets the impact durations were identical as well as the shap es and magni
tudes of the force signals
The frequency domain comparison of the bar acceleration signal showed again
b etter match with the hard mallet The rst three partials were almost identical
with a discrepancy of less than or equal to dB With the soft mallet the third
partial is approximately dB below the corresp onding partial of the measured
signal
cause of the discrepancies see For a detailed comparison and discussion on the
Chaigne and Doutaut
Mo dal Synthesis
The mo dal synthesis metho d has b een develop ed mainly at IRCAM in Paris France
Adrien Adrien They have pro duced a commercial software appli
cation Mo dalys Eckel et al formerly called Mosa c Morrison and Adrien
With this application the user can simulate vibrating structures The user
describ es the structure under study for the program and the program computes the
mo dal data and outputs the signal observed at a p oint de ned by the user
Mo dal synthesis is based on the premise that any sound pro ducing ob ject can
be represented as a set of vibrating substructures which are de ned by mo dal data
Adrien Substructures are coupled and they can resp ond to external ex
citations These coupling connections also provide for the energy ow between
substructures Typical substructures are
b o dies and bridges
bows
acoustic tub es
membranes and plates
b ells
The simulation algorithm uses the information of each substructure and their inter
actions
The metho d is general as it can b e applied to structures of arbitrary complexity
The computational e ort needed increases rapidly with complexitythus setting the
practical limits of the metho d Next the formulation of mo dal data of a substructure
is presented Then an application to real musical instrument is shortly discussed
Chapter Physical Mo dels
Mo dal Data of a Substructure
The mo dal data for a substructure consists of the frequencies and damping co ef
cients of the structure s resonant mo des and of the shap es of each of the mo des
Adrien A vibrating mo de is essentially a particular motion in which every
point of the structure vibrates with same frequency It should be noted that an
arbitrary motion of a structure can be expressed as a sum of the contributions by
the mo des as can b e done by Fourier series expansion
force applied at a given p oint on the The mo des are excited by an external
structure The excitation energy is distributed to the mo des dep ending on the form
of the excitation It is assumed that there exists no exchange of energy b etween the
mo des In practice the vibration pattern is never fully describ ed by a single mo de
but it is a sum of an in nite series of vibrating mo des This accounts for an in nite
number of degrees of freedom in a continuous structure For numeric computation
of vibration of the structure to b ecome realizable the continuous structure must b e
spatially divided into a nite set of p oints
Given a set of N p oints on a structure a number of N mo des can b e represented
Each mo de is describ ed by its resonant frequency and damping co e cients
m m
e displacements of the The N N modes shape matrix describ es the relativ
m k
N points in each mo de Column m of the mo des shap e matrix corresp onds to the
contribution of mo de m to the displacements of the N points Each mo de can then
be presented as a second order resonator connected in parallel with the others as
pictured in Fig
The mo dal data can b e obtained analytically for simple vibrating structures The
expressions for the mo dal data for each mo de can be obtained from the di erential
equation system governing the motion of the simple vibrating system For complex
structures direct computation of mo dal data is not p ossible and analysis based on
measuring exp eriments must be utilized Mo dal analysis is used extensively in air
craft and car industry andthus the to ols are e cient and available They t ypically
consist of excitation and pickup devices signal pro cessing hardware and software
for Fourier transforms and p olynomial extraction of mo dal data Similar metho ds
have been used for parameter calibration of other physical mo dels V lim ki et al
The metho d is similar for mechanical and acoustical systems In mechanical
systems the de ections in the mo des shap e matrix are the actual displacements of
the p oints on the surface of the vibrating structure In acoustical systems elements of
the mo des shap e matrix corresp ond to the de ections of sound pressure or particle
velo city
Synthesis using Mo dal Data
Mo dal synthesis is very powerful in that all vibrating structures can be describ ed
using the same equations These equations describ e the resp onse of the structure to
hanical structure partitioned to N an excitation applied at a given p oint Foramec
Mo dal Synthesis
F, v Φ
Point
mass damper spring
Figure A mo dal scheme for the guitar A complex vibrating structure is
represented as a set of parallel second order resonators resp onding to the external
force F and contributing to the resulting velo city v
th
points the equation for the instantaneous velo cityofthe k pointis Adrien
P
P
N
m t
i ext
X
F
y
m t
m
k t
l l t
l
t t
m
k
t
t
m m
m
t
m
m th
where is the contribution of the m mo de to the de ection of point k on the
k
ext
structure F is the instantaneous external force on p oint l of the structure t is
l t
the time step and and the angular frequency the damping co e cient
m m m
th
and the instantaneous de ection asso ciated with the m mo de
A similar equation can be applied to acoustic systems with the external forces
ext ext
F replaced by external ows U The density of air is denoted by The
l t l t
equation b ecomes
P
P
N
m t
i ext
X
U
y
m t
m
k t
l l t
l
t t
m
p
k t
k
t
t
m m
m
t
m
If all instantaneous external excitations are known the velo cities of the mo des
and thus the velo cities of all of the points can be calculated However typically
only the excitation corresp onding to control and driving data are known and other
forces of ows have to b e determined These forces or ows implement the coupling
i e the energy ow between substructures The couplings are often nonlinear
The reed air column interaction in wo o dwind instruments is an example of coupling
of linear systems governed by Equations and resp ectively The coupling
Chapter Physical Mo dels
ext
equation involves the owentering the b ore U and the pressure di erence b etween
the mouth and the bore P P the p osition of the reed the Backus constant
m
B and the additional ow S due to the displacement of the reed The interaction
shifts b etween two regions Adrien
p
ext
p p S Op en reed U B
m t t t
t
ext
U
t
Closed reed
t
Application to an Acoustic System
When the mo dal synthesis metho d is applied to a simple acoustical system consisting
of a conical tub e with a simple reed mouthpiece and ve holes six equations of the
form of Eq are utilized one for the cone and one for each hole The reed is
presented as a mechanical system with nonlinear coupling to the cone with Equations
and The interactions between substructures involve ow conservation
Using this principle it is p ossible to eliminate all pressure terms from the equations
and present the equations in a matrix form For a description of the matrix
equations see Adrien
The mo dal synthesis metho d provides many p ossible output signals It is inter
esting at least for research purp oses to try to recreate the acoustic eld of a real
instrument This can be done by utilizing a body of a real instrument to radiate
the created acoustic signal In the case of the violin the string the bridge and the
exciter is mo deled as usual but the b o dy is replaced by an in nite imp edance in the
mo del The sound outputs obtained at the fo ot of the bridge are used as force signals
to drive shakers at the fo ot of a real instrument bridge The implicit assumption
made ab ove is that the body do es not act as a load for the strings and therefore
it do es not a ect the attenuation and phase of the partials in bow string interac
tion Ro cchesso Adrien gives a detailed discussion of the simulated
signals but lacks comparison to measured real instrument signals
Mass Spring Networks the CORDIS System
Cadoz et al attempt to mo del the acoustical system under study using simple
ideal mechanical elements such as masses damp ers and springs They aim to
develop a paradigm which can be applied to an arbitrary acoustic system The
CORDIS system was the rst system capable of pro ducing sound based on a physical
mo del in real time Florens and Cadoz In this section rst the basic elements
of the system are describ ed Second application to mo deling a plucked string is
discussed
Elements of the CORDIS System
The most primitive and fundamental elements of the system are the following
Mass Spring Networks the CORDIS System
point masses
ideal springs
ideal damp ers
When these comp onents are combined and connected in su cient number repro
duction of spatial continuum and an acoustical signal should b e p ossible at a desired
sampling rate Florens and Cadoz The ob ject under study is then approx
imated as a set of these elements discretely distributed over its surface A ma jor
simpli cation is obtained by taking each element as b eing one dimensional i e each
element can only move or act in one dimension For mo deling interactions that vary
in time e g b owing striking by a hammer or plucking a conditional link is intro
duced It consists of a spring and a damp er with adjustable parameters connected
in parallel
are given The mathematical presentations for the elements are simple and they
by Florens and Cadoz with
x
Mass F m
t
Spring F F K x x
x x
Damp er F F Z
t t
x x
Conditional link F F K x x Z
t t
where F is the force driving the mass F and F are the forces at p oints x and x
of the spring damp er or the condition link Z is the friction co e cient and K the
spring co e cient
The same equations maybe obtained in discretized form by taking
x n
x n x n
t
and
x n x n x n
t t t
thus
Mass F n m x n x n x n
Spring F n F n K x n x n
Damp er F n F n
Z x n x n x n x n
n K x n x n Conditional link F n F
Z x n x n x n x n
Temp oral discretization intro duces an error to the frequency of each of the mo d
eled harmonic in the form
T
Chapter Physical Mo dels
Since the sampling frequency f is the recipro cal of the time step T the error can b e
s
reduced by increasing the sampling frequency Using a sampling frequency of three
times the frequency of the highest harmonic comp onent guarantees a maximum error
of percent on all partials
The vibrating string is mo deled with N masses connected with N identical
parallel springs and damp ers as illustrated in Figure This continuum of points
on the string is connected at b oth ends to rigid end supp orts with a damp er and a
spring in parallel In this case there will b e N harmonics present in the signal
Rigid N Rigid end end
Point
mass damper spring
Figure A mo del of a string according to the CORDIS system N point masses
are connected to each other and to rigid end supp orts with a damp er and a spring
in parallel at each connection
The creators of CORDIS have develop ed a system called ANIMA for two and
three dimensional elements Florens and Cadoz
Comparison of the Metho ds Using Numerical
Acoustics
In this section the three presented metho ds are compared by discussing the appli
cation of each metho d to an acoustic system the guitar
Using nite di erence equations for simulating the vibration of a string is pre
metho d is very accurate in repro ducing sented in Section The nite di erence
the original waveform if the mo del parameters are correct The approachisinterest
ing esp ecially in a scienti c sense b ecause the vibratory motion can b e observed at
any discrete pointon the string Furthermore the parameters of the mo del are the
actual parameters of the real instrument such as the sti ness and the loss param
eters of the string and the input admittance at the bridge These parameters can
b e obtained via measurements and analysis on b oth the instrument and the signals
pro duced by it
Comparison of the Metho ds Using Numerical Acoustics
For real time sound synthesis purp oses the nite di erence mo del is not very
attractive The mo del can only be applied to a simple structure such as a vi
brating string in real time including a guitar body in the mo del would imply
the need for hybrid systems An estimation on the complexity of the computa
tion can be obtained by insp ecting Equation For each of the N p oints on
a string ve multiplications and eight summations are needed with an additional
multiplication and summation if an excitation is applied at that p oint For go o d
spatial resolution N needs to be large So several hundreds of op erations are
needed for every output sample Also numerical disp ersion might be a problem
with the FD metho d Ro cchesso An example program for simulating string
vibration as well as the e ect of each individual parameter is written by Kurz and
Feiten The program for Silicon Graphics workstations can be downloaded
at ftp ftp kgw tu berlin de p ub v stri ng
With mo dal synthesis the guitar can be divided into three substructures one
instrument namely the excitation the vibrating for every functional part of the
strings and the body radiating the sound eld The excitation substructure only
interacts with other parts when the string is b eing plucked The excitation can be
applied at anypoints on other two substructures The vibrating string is simulated
with N parallel indep endent second order resonators each pro ducing one harmonic
comp onent The resonators can be computationally e ciently implemented but
a large number of them are needed for high quality synthesis The mo del for the
body of the instrument is obtained by mo dal analysis of the structure This is a
very time consuming pro cess esp ecially for a complex structure such as the violin
Adrien
If a body of a real instrument is used as a transducer the radiated sound eld
pro duced by the vibrating string coupled to the b o dy can b e simulated Naturally
this metho d can also be used with other metho ds capable of pro ducing a driving
force signal at the bridge
The CORDIS system divides each vibrating structure into idealized elements i e
point masses vibrating in one direction connected with ideal damp ers and springs
A vibrating string is thus simulated with N p oint masses connected together with
N links comp osed of a damp er and a string connected in parallel This structure
ber of computational op erations is capable of pro ducing N harmonics The num
for each cell is relatively low An estimation can be made by analyzing Equations
One output sample requires approximately N multiplications and N
summations Unfortunately an estimation on the number of points needed for the
simulation of a guitar b o dy was not available
Todraw a summary several observations are made The nite di erence metho d
can b e used for simulating vibrations on essentially one dimensional ob jects very ac
curately The other metho ds attempt to b e more general at the cost of the accuracy
and the detailed mathematic presentation of the vibratory phenomena The nite
di erence metho d and the mo dal synthesis metho d provide to ols for the study of
of the metho ds is very well applicable for real time sound real instruments None
synthesis purp oses The rst reason is the computational cost when high quality
Chapter Physical Mo dels
synthesis is desired Second the parameters of the mo del are non intuitive in a
musical sense and they are hard to control in the same way the actual instrument
is controlled esp ecially in real time p erformance situations Finally sound synthe
sis metho ds with more e cient computation and control exist esp ecially for string
instruments and wo o dwinds with conical b ores
Digital Waveguides and Extended
Karplus Strong Mo dels
Digital waveguides and single delay loop SDL mo dels are the most e cient metho ds
for physics based real time mo deling of musical instruments High quality mo dels
exist for a number of musical instruments and the research in this eld is active
In this chapter the digital waveguides are rst discussed Second waveguide
meshes which are D and D mo dels are presented The equivalence of the bi
directional digital waveguide mo del and the SDL mo del is detailed by Karjalainen
et al and it will b e describ ed shortly The last sections of the chapter present
a case study of mo deling the acoustic guitar using c ommuted waveguide synthesis
Digital Waveguides
The concept of digital waveguides has b een develop ed by Smith
Digital waveguides and metho ds based on nite di erences are closely related in
that they b oth start from the premise of solving the wave equation We recall from
Section that with the nite di erence metho d the wave equation is solved in a set
of discrete p oints on the vibrating ob ject Atevery time p erio d a physical variable
such as displacement is computed for every p oint This implies that the vibratory
motion of the whole discretized vibrating ob ject is readily observable While this
may be attractive for the study of the vibrating ob ject and the vibratory motion
more e cien t metho ds are needed for sound synthesis purp oses
Waveguide for Lossless Medium
The digital waveguide is based on a general solution of the wave equation in a one
dimensional homogeneous medium The lossless wave equation for a vibrating string
can be expressed as Morse and Ingard
y y
K
x t
where K is the string tension the linear mass density and y the displacement
of the string This equation is applicable to any lossless one dimensional vibratory
Chapter Digital Waveguides and Extended Karplus Strong Mo dels
y ( x-ct ) l y ( x+ct ) r
x
Figure d Alemb ert s solution of the wave equation
motion like that of the air column in the b ore of a cylindrical wo o dwind instru
ment Naturally in that case the parameters and the wave variables are interpreted
accordingly Itcanbeseen by direct computation that the equation is solved byan
arbitrary function of the form
y x t y x ct or
l
y x t y x ct
r
where
r
K
c
The functions y x ct and y x ct can be interpreted as traveling waves going
l r
left and right resp ectively The general solution of the wave equation is a linear
combination of the two traveling waves and it is pictured in Figure This is the
d Alemb ert s solution to the wave equation
The only restriction p osed by the d Alemb ert s solution is that the functions
y x ct and y x ct have to be twice di erentiable in both x and t However
l r
when the linear wave equation is develop ed for the real one dimensional vibrator
the amplitude of the vibration is assumed to be small Physically in the case of
a vibrating string this means that the slop e of the vibrating string can only have
values muchlower than one Similarly vibrating air columns can exhibit only small
variations of pressure around the static air pressure
x ct The digital waveguide is a discretization of the functions y x ct and y
l r
and it is obtained by rst changing the variables
x x mX
m
t t nT
n
where T is the time step X is the corresp onding step in space and m and n are
the new integral valued time and space variables The new variables are related by
X
c
T
Substitution of the new variables to the d Alemb ert s solution of the wave equation
Digital Waveguides
y+ ( n ) y+ ( n - m ) m sample delay line
y( n,k ) y- ( n ) y- ( n + m ) m sample delay line
x= 0 xk= x = mcT
Figure The one dimensional digital waveguide after Smith
yields
x x
m m
y t y x t y t
l n m n r n
c c
mX mX
y nT y nT
r l
c c
y T n m y T n m
r l
Equation can be simpli ed by de ning
y n y nT and
r
y n y nT
l
In this notation the sup erscript denotes the traveling wave comp onent going to
the right and the sup erscript the comp onent going to the left
Finally the mathematical description of the digital waveguide is obtained with
discrete functions y n m and y n m which can be interpreted as the two
m sample delay lines The delay lines are pictured in Figure The output from
the waveguide at point k is obtained by summing the delay line variables at that
point
The solution to the one dimensional wave equation provided by the waveguide
is exact at the discrete points in the lossless case as long as the wa vefronts are
originally bandlimited to one half of the sampling rate Bandlimited interp olation
can b e applied to estimate the values of the traveling waves at non integral p oints of
the delay lines Fractional delay lters provideaconvenient solution to bandlimited
interp olation See Laakso et al and V lim ki for more on fractional
delay lters A number of di erent physical quantities can be chosen as traveling
waves See Smith or for details on conversion between wave variables
Waveguide with Disp ersion and Frequency Dep endent
Damping
t for attenuation of the In real vibrating ob jects physical phenomena that accoun
vibratory motion are always present These phenomena have to be incorp orated
in the mo del to obtain any realistic synthesis In a general case disp ersion is also
present Awave equation that includes b oth the frequency dep endent damping and
Chapter Digital Waveguides and Extended Karplus Strong Mo dels
y+ ( n ) y+ ( n - m ) -k k -(m-k) m-k z G (ω ) z G (ω )
y( n,k ) y- ( n ) y- ( n + m ) k -k m-k -(m-k) G (ω ) z G (ω ) z
x = 0 xk= x = mcT (a)
+ y ( n ) y+ ( n - m ) -k k -(m-k) m-k z G (ω ) Hk(ω ) z G (ω ) Hm-k (ω )
y( n,k ) - y ( n ) y- ( n + m ) k -k H (ω ) -(m-k) m-k Hk(ω ) G (ω ) z m-k z G (ω )
x = 0 xk= x = mcT
(b)
Figure A lossy digital waveguide in a The frequency dep endent gains G
k
are lump ed b efore observation p oints to obtain G in order to get more e cient
implementation In b disp ersion is added in the form of allpass lters approximation
the desired phase delay after Smith
the disp ersion is already presented for the vibrating string in Equation The
complete linear time invariant generalization of the wave equation for the lossy sti
string is describ ed by Smith
A frequency dep endent gain factor G determines the frequency dep endent
attenuation of the traveling wave for one time step For a detailed derivation of
an expression for G from the one dimensional lossy w ave equation see Smith
In the waveguide a gain factor that realizes G would have to be
inserted b etween every unit delay However the system is linear and time invariant
and the gain factors can be commuted for every unobserved p ortion of the delay
line This is illustrated in Figure a where the losses are consolidated b efore
each observation point
When the fourth order derivative with resp ect to displacement y is present in the
t but it is dep endent wave equation the velo city of the traveling waves is not constan
on the frequency This is to say that the wavefront shap e will b e constantly evolving
as the higher frequency comp onents travel with a di erent velo city than the lower
frequency comp onents This physical phenomenon is presentinevery physical string
and it is called disp ersion The disp ersion is mainly caused by sti ness of the string
For derivation of an expression for the frequency dep endent velo city see Smith
The disp ersion can be taken into account in the waveguide mo del by inserting
Digital Waveguides
an allpass lter b efore each observation p oint as is done in Figure b The
allpass lter H z approximates the disp ersion e ect for a delay line of length a
a
Van Duyne and Smith present an e cient metho d for designing the allpass
lter as a series of one p ole allpass lters More recently Ro cchesso and Scalcon
have presented a metho d to design an allpass lter based on analysis of
the disp ersion in recorded sound signals based on an allpass lter design metho d
presented by Lang and Laakso
Applications of Waveguides
The digital waveguide has b een applied to many sound synthesis problems Smith
A short overview of applications in di erent instrument families is given
The rst physics based approach to use digital lters to mo del a musical instru
ment was made for the violin by Smith Ja e and Smith intro duced
several extensions to the Karplus Strong algorithm that enable high quality synthe
sis of plucked strings including an allpass lter in the delaylooptoapproximate the
non integral part of the delay
Since those pioneer works many improvements and further extensions have b een
presented for plucked string synthesis These include Lagrange interp olation for ne
tuning the pitch and pro ducing smo oth glissandi Karjalainen and Laine and
allpass ltering techniques to simulate disp ersion caused by string sti ness Smith
Paladin and Ro cchesso and Van Duyne and Smith Com
muted waveguide synthesis technique is an e cient way to include a high quality
mo del of an instrument body to waveguide synthesis It has b een prop osed by
Smith and Karjalainen et al V lim ki et al have presented
a metho d to pro duce smo oth glissandi with allpass fractional delay lters A pa
rameter calibration metho d based on the STFT was develop ed by Karjalainen et
elab orated by V lim ki et al A similar approach is al and further
also made by Laro che and Jot These works are extended and an automated
calibration system was implemented by Tolonen and V lim ki Multirate
implementations of the string mo del and separate low rate b o dy resonators are pre
sented by Smith V lim ki et al and V lim ki and Tolonen a
b
The plucked string algorithm is also utilized to synthesize electric instrument
tones Sullivan extended the Karplus Strong algorithm to synthesize electric
guitar tones with distortion and feedback Rank and Kubin have develop ed
a mo del for slapbass synthesis
Waveguide synthesis for the piano is presented bySmithandVan Duyne
and Van Duyne and Smith a where a mo del of a piano hammer Van Duyne
and Smith D digital waveguide mesh Van Duyne and Smith a see also
Section and allpass ltering techniques for simulating sti ness of the strings and
the soundb oard are combined together Another development of the piano hammer
mo del is presented by Borin and Giovanni
Chapter Digital Waveguides and Extended Karplus Strong Mo dels
Waveguide synthesis has also b een applied to several wind instruments The clar
inet was one of the rst applications by Smith Hirschman V lim ki
et al b and Ro cchesso and Turra A waveguide mo del for the ute
has been prop osed by Karjalainen and Laine and V lim ki et al a
V lim ki et al prop ose a mo del for the nger holes in wo o dwind b ores Brass
instrument tones have b een simulated with waveguides by Co ok Dietz and
Amir Msallam et al and Vergez and Ro det Co ok has
created a device that can control mo dels of the wind instrument family
SPASM is a DSP program by Co ok to mo del the sound pro cessing mech
anism of a human in real time It also provides a graphical user interface with an
image of the vo cal tract shap e
Waveguide Meshes
The digital waveguide presented in the previous section is very e cient in mo deling
one dimensional vibrators If mo deling of vibratory motion in a D or D ob ject is
desired the digital waveguide can b e expanded to a waveguide mesh Applications of
waveguide meshes can b e found for instance in mo deling membranes soundb oards
cymbals gongs and ro om acoustics
In this section the two dimensional waveguide mesh is discussed Di erent im
plementation of the D waveguide mesh are given byVan Duyne and Smith a
b Fontana and Ro cchesso and Savio ja and V lim ki
Expansion to a three dimensional mesh is relatively straigh tforward as well as to
the mathematically interesting N dimensional mesh An interesting D formulation
not discussed here is the tetrahedral waveguide mesh presented by Van Duyne and
Smith b
The traveling plane wave solution of the two dimensional wave equation is given
as Morse and Ingard
u t x y u t x y u t x y
c
t x y
Z
u t x y f x cos y sin ct d
where denotes the direction of the plane wave The integral involves an in nite
number of traveling waves that are divided into comp onents traveling in the x and
y directions
Scattering Junction Connecting N Waveguides
veguides needs to be To be able to formulate a waveguide mesh a junction of wa
develop ed Connection of waveguides is pictured in Figure where scattering junc
tion S connects N bi directional waveguides with imp edances R i N
i
Waveguide Meshes
R1
RN
R2
S R3
R4
Figure A scattering junction after Van Duyne and Smith b N waveg
uides are connected together with no loss of energy
For the connection to be physically meaningful two conditions are required The
values of the wavevariables e g vibration velo cities or sound pressures havetobe
equal at the pointofthe junction
v v v v
S N
where v is the value of the wave variable in the junction Equation states that
S
the strings move together all the time Second the sum of the forces exerted by the
strings or ows in the tub es must equal to zero
N
X
f
k
k
Recalling from the previous section the de nitions
R v and f R v f f f f v v v
k k k k
k k k k k k k k
the two constraints of Equations and can b e develop ed further as
N N N
X X X
R v R v R v
k k k k
k k
k k k
equals
z
N N N N
X X X X
R v R v R v R v
k k k k
k k k k
k k k k
N
X
R v
k
k
k
Now using v v an expression for the wave variable at the junction is obtained
S k
Chapter Digital Waveguides and Extended Karplus Strong Mo dels
as
N
X
R v
k
k
k
v
P
S
N
R
k
k
as v The outputs of the junction are obtained by applying v v v
S k
k k
v v v
S
k k
Two Dimensional Waveguide Mesh
The rectilinear waveguide mesh formulation of the two dimensional wave equation
consists of delay elements and p ort scattering junctions Such a system is pictured
in Figure The scattering junctions are marked with S where l denotes the
l m
time variable is n index to the x direction and m to the y direction The discrete
The two delay elements between the p orts of consecutive scattering junctions form
a bi directional delay unit
If the medium is assumed isotropic the imp edances R are equal and the junction
k
equations for junction S denoted S for convenience are obtained from Equations
l m
and as
X
v n n v
S
k
k
and
n k n v n v v
S
k k
This formulation can b e interpreted as a nite di erence approximation of the two
dimensional wave equation as shown by Van Duyne and Smith a b
Analysis of Disp ersion Error
The formulation of the waveguide mesh presented ab ove has some drawbacks The
wave propagation sp eed and magnitude resp onse dep end on b oth the direction
of wave motion and frequency This can be illustrated by examining the two
dimensional discrete Fourier transform of the nite di erence scheme The D DFT
pro duces a D frequency space so that each p oint corresp onds to a spatial
frequency
q
The co ordinates and of the D frequency space are taken to corresp ond to x
and y dimensions of the waveguide mesh resp ectively
The ratio of the actual propagation sp eed to the desired propagation sp eed in
the rectilinear waveguide mesh can b e computed as Van Duyne and Smith a
p
p
c b
arctan
c T b
Waveguide Meshes
y-direction
z-1 z-1 z-1 z-1
+ + v v 2 2 + + v v 1 1 z-1 z-1 z-1 S S l,m l+1,m z-1 z-1 z-1 + + v 3 v 3 + v + v 4 4
z-1 z-1 z-1 z-1
+ v + 2 v 2 + v + 1 v 1 z-1 z-1 z-1 S S l,m-1 l+1,m-1 z-1 z-1 z-1 + + v 3 v 3 + v v + 4 4
z-1 z-1 z-1 z-1
x -direction
Figure Blo ck diagram of a D waveguide mesh after Van Duyne and Smith
a
where
b cos T cos T
and T is the sampling interval
The e ect of disp ersion error can be suppressed using di erent typ es of waveg
uide formulations Savio ja and V lim ki prop ose to use an interp olated
waveguide mesh that utilizes deinterp olation to approximate unit delays in the di
agonal directions Fontana and Ro cchesso suggest a tessellation of the ideal
membrane into triangles The ratio of propagation sp eeds is pictured in Figure as
a function of frequency for four di erenttyp es of waveguide formulations In Figure
and Smith a the sp eed ratio of the rectilinear formula a Van Duyne
tion is pictured In b and c the sp eed ratios of a hyp othetical non realizable
directional waveguide and a deinterp olated directional waveguide are depicted
Savio ja and V lim ki In Figure d the sp eed ratio of the triangular
tessellation is illustrated Fontana and Ro cchesso
The distance from center of the plots in Figures a d corresp onds to the
Chapter Digital Waveguides and Extended Karplus Strong Mo dels
1 1 0.8 0.8 0.5 0.5 0.6 0.6 −0.5 0 ω −0.5 0 ω Normalized speed 0 Normalized speed 0 ω 0.5 −0.5 ω 0.5 −0.5 (a) (b)
1 1 0.8 0.8 0.5 0.5 0.6 0.6 −0.5 0 ω −0.5 0 ω Normalized speed 0 Normalized speed 0 ω 0.5 −0.5 ω 0.5 −0.5
(c) (d)
Figure Disp ersion in digital waveguides The wave propagation sp eed is plotted
as a function of spatial frequency and direction for a rectilinear mesh in a for a
hyp othetical directional mesh in b for a deinterp olated directional mesh in c
and for a triangular tessellation in d The spatial frequency is the distance from
the origin and the T and T axis of the horizontal plane corresp ond to x and y
directions
Single Delay Lo op Mo dels
spatial frequency The axis of the horizontal plane are the and axis which
corresp ond to the x and y directions in the mesh resp ectively The contours of
equal ratios are pictured on the b ottom of each gure The dep endence of the
propagation sp eed ratio on b oth the frequency and direction can be seen in a In
other gures the dep endence on the direction can be observed to be less severe It
should b e noted that the mesh of b is not realizable
Single Delay Lo op Mo dels
First extensions to the Karplus Strong algorithm presented in Section were de
rived by Ja e and Smith Even b efore that Smith had develop ed
a mo del for the violin that included a string mo del that is similar to the generic
Karplus Strong mo del in Figure b Those works were the rst to take aphys
ical mo deling interpretation of the Karplus Strong mo del
The digital waveguide presented in the previous chapter can b e develop ed to an
SDL mo del in certain situations In this chapter the SDL mo del is derived for
the guitar as has b een done by Karjalainen et al In this context only
the case of a string with force signal output at the bridge will b e considered This
corresp onds to the construction of a classical acoustical guitar The case of pickup
output which corresp onds to electric guitars is presented by Karjalainen et al
We start with a continuous time waveguide mo del in the Laplace domain
and develop a discrete time mo del which can be identi ed as an SDL mo del
In the discussion to follow the transfer functions of the mo del comp onents are
describ ed in the Laplace transform domain The Laplace transform is an e cient
to ol in linear continuous time systems theory P articularly time domain integration
and derivative op erations transform into division and multiplication by the Laplace
variable s resp ectively The complex Laplace variable s may be changed with j
p
where j is the imaginary unit is the radian frequency f andf is the
frequency in Hz in order to derive the corresp onding representation in the Fourier
transform domain i e the frequency domain For a discrete time implementation
the continuous time system is nally approximated by a discrete time system in the
z transform domain For more information on Laplace Fourier and z transforms
see a standard textb o ok on signal pro cessing such as Opp enheim et al
In the next subsection a waveguide mo del for the acoustic guitar is presented
In the one after that the digital waveguide representation is develop ed into an SDL
mo del
In this do cument the mo dels of a vibrating string consisting of a lo op with single delay line are
called single delay lo op SDL mo dels to distinguish them from b oth the non physical KS algorithm
and the bidirectional digital waveguide mo dels
Chapter Digital Waveguides and Extended Karplus Strong Mo dels
H L1,E1( s ) H E1,R1 ( s ) L1 E1 R1 A 1( s )
Delay line Bridge force ( s ) + 1/2 X 1 R ( s ) Z ( s ) f X ( s ) R ( s ) F ( s ) b s X 2( s ) -
Delay line
L2 E2 R2 A 2 ( s )
H E2,L2( s ) H R2,E2 ( s )
Figure Dual delay line waveguide mo del for a plucked string with a force output
at the bridge Karjalainen et al
Waveguide Formulation of a Vibrating String
In Figure a dual delay line waveguide mo del for an ideally plucked acoustic guitar
string with transversal bridge force as an output is presented The delay lines and
the re ection lters R s and R s form a lo op in which the waveforms circulate
b f
The two re ection lters simulate the re ection of the waveform at the termination
points of the vibrating part of the string at the bridge and at the corresp onding fret
resp ectively The lters are phase inversive i e they have negative signs and they
also contain slight frequency dep endent damping Let us assume for no w that the
delay lines corresp ond to the d Alemb ert s solution of the wave equation for a sti
and lossy string In this case they are disp ersive and they also attenuate the signal
continuously in a frequency dep endent manner
Pluck excitation X s is divided into two parts X s and X s so that X s
X s X s The excitation parts are fed into the waveguides at points E and
E It has been shown by Smith that an ideal pluck of the string can be
approximated by a unit impulse if acceleration waves are used Thus it is attractive
to cho ose acceleration as the wave variable and in this context A s and A s
corresp ond to the values of the right and the left traveling acceleration waves at
p ositions R and R resp ectively
The output signal of interest is the transverse force F s applied at the bridge
ave comp onents A s by the vibrating string It is obtained from the acceleration w
and A s as
F s F s F s Z s V s V s Z s A s A s
s
i e the bridge force F s is the bridge imp edance Z s times the di erence of
the string velo city comp onents V s and V s at the bridge In the last form
of Equation the velo city di erence V s V s is expressed as integrated
A s A s acceleration di erence
s
Single Delay Lo op Mo dels
Figure also includes the transfer functions between the unobserved and un
mo di ed parts of the waveguides H s refers to the transfer function from p oint
A B
A to p oint B These transfer functions are elab orated in the following subsection
where the bi directional digital waveguide mo del is reformulated as an SDL mo del
Single Delay Lo op Formulation of the Acoustic Guitar
In the waveguide formulation pictured in Figure there are four points namely
veguide E E R and R at which either a signal X s and X s is fed to the wa
or the wave variables A s and A s are observed It is immediately apparent
that the formulation can be simpli ed by combining transfer function R s and
f
transfer functions H s and H s of the two parts of the lossy and disp ersive
E L L E
waveguide to the left of the excitation p oint E and E However it is more e cient
to attempt to reduce the numb er of p oints in which the wavevariables are pro cessed
or observed
The explicit input to the lower delay line can be removed by deriving an equiv
alent single excitation at p oint E that corresp onds to the net e ect of the two
excitation comp onents at p oints E and E The equivalent single excitation at E
can be expressed as
X s X s H s R s H s X s
E eq E L f L E
H s X s
E E
H s X s
E
where H s is the left side transfer function from E to E consisting of the two
E E
parts of the lossy and disp ersive delay lines H s and H s and re ection
E L L E
function R s Thus H s is the equivalent excitation transfer function
f E
In a similar fashion one of the explicit output p oints can b e removed in order to
obtain a structure with only single input and output p ositions Since the guitar b o dy
is driven by the force applied by the vibrating string at the bridge it is apparent
that an acceleration to force transfer function is required In Equation output
force F s is expressed in terms of acceleration waves A s and A s This is
further elab orated as
A s A s F s Z s
s
Z s A s R s A s
b
s
Z s R s A s
b
s
H s A s
B
where H s is the acceleration to force transfer function at the bridge Notice that
B
it only dep ends on A s the wave variable of the upp er delay line In a similar
fashion one can derive an expression for F s dep ending only on A s
eq in X s stands for equivalent
E eq
Chapter Digital Waveguides and Extended Karplus Strong Mo dels
Todevelop an expression for A s in terms of the equivalent input X s we
E eq
rst write
A s H s X s H s A s
E R E eq lo op
where
H s R s H s H s H s
lo op b R E E E E R
circulated once around the i e H s is the transfer function when the signal is
lo op
lo op Thus the sum terms of Eq corresp ond to the equivalent excitation
signal X s transfered to p oint R and signal A s transfered once along the
E eq
lo op Solving Equation for A s we obtain
A s H s X s
E R E eq
H s
lo op
H s S s X s
E R E eq
where S s is the string transfer function that represents the recursion around the
string lo op
Finally the overall transfer function from excitation to bridge output is written
as
F s H s
E R
H s H s Z s R s
E B E E b
X s H s s
lo op
or more compactly based on the ab ove notation
H s H s H s S s H s
E B E E R B
which represents the cascaded contribution of each part in the physical string system
At this p oint the continuous time mo del of the acoustic guitar in the Laplace
transform domain is approximated with a discrete time mo del in the z transform
domain This approximation is needed in order to make the mo del realizable in a
Equation in the z transform domain discrete time form We rewrite
H z H z H z S z H z
E B E E R B
where
H z a H z
E E E
b S z
H z
lo op
H z Z s I z R z c
B b
Filter I z is a discrete time approximation of the time domain integration op era
tion
Equation is interpreted by examining a blo ck diagram in Figure It shows
qualitatively the delays and the discrete time approximations of the cascaded lter
Single Delay Lo op Mo dels
Filtering due to excitation position H ( z ) -1 E
1/2 Delay Lowpass Input
Wave propagation, from excitation to bridge H ( z ) E1,R1
Delay Lowpass
String loop S ( z )
Lowpass Delay
Filtering due to bridge coupling H ( z ) B
Lowpass
Output
Integrator
Figure A blo ck diagram of transfer function comp onents as a mo del of the
plucked string with force output at the bridge Karjalainen et al
comp onents in Equation The rst blo ck corresp onding to H z simulates the
E
comb ltering e ect dep ending on the pluck p osition Notice that the phase inversion
of the re ection lter is explicated with the multiplication by The second blo ck
corresp onding to the transfer function from E to R in Figure has a minor e ect
and is usually discarded in the nal implementation of the mo del This reduction
is justi ed by noticing that the gain term G determining the attenuation of the
traveling wave in one time step is extremely close to unity and thus in the short
time it takes for the wave to travel from E to R the attenuation is negligible
The third blo ck in Figure is the string lo op and it simulates the vibration of
the string The delay in the lo op corresp onds in length to the sum of the two delay
lines in Figure the losses of a single round in the lo op are consolidated in the
lowpass lter In the last blo ck the feedforward lter is typically discarded and
only the integrator is implemented In this case the lowpass lter corresp onds to
the opp osite of re ection lter at the bridge and it is very close to unity Thus the
sum of the ltered and the direct signal is approximated as b eing equal to
Notice that the mo del of the acoustic guitar presented in Figure is indeed a
Chapter Digital Waveguides and Extended Karplus Strong Mo dels
single delay lo op mo del with the only lo op in the third blo ck The presented mo del
describ es the vibration of a plucked string It includes the e ects of the plucking
p osition and the output signal corresp onds to the force applied by the string at
the guitar body in a physically relevant manner In the next section this mo del is
extended to include mo dels of the guitar b o dy the two vibration p olarizations and
sympathetic couplings b etween the strings
Single Delay Lo op Mo del with Commuted Bo dy
Resp onse
The sound pro duction mechanism of the guitar can b e divided into three functional
substructures namely the excitation the vibration of the string and the radiation
this functional partition when from the guitar body It is advantageous to retain
developing a computational mo del of the acoustic guitar as suggested by many
studies presented in the literature Smith Karjalainen et al V lim ki
et al Karjalainen and Smith Tolonen and V lim ki V lim ki
and Tolonen a In the previous section a detailed mo del for the vibration
of a single string was describ ed In order to obtain a high quality simulation of
the acoustic guitar the excitation and the body mo dels have to be incorp orated in
the instrument mo del The mo del should be su ciently general to accommo date
suc h e ects as those pro duced by the two vibration p olarizations and sympathetic
couplings b etween the strings
In the virtual instrument the excitation mo del determines the amplitude of
the sound the plucking typ e and the e ect of the plucking point while the body
mo del gives an identity to the instrument i e it determines what typ e of a guitar is
b eing mo deled The b o dy mo del includes the b o dy resonances of the instrumentand
determines the directional prop erties of the radiation The directional prop erties are
not included in the mo del presented here but they can b e added by p ost pro cessing
the synthesized signal Huopaniemi et al Karjalainen et al
In this section the principle of commuted waveguide synthesis CWS Smith
Karjalainen et al is rst discussed for e cient realization of the excita
tion and body mo dels Second a physical mo del that includes the aforementioned
features is presented In this context the synthetic acoustic guitar is only discussed
generally without going into details of the DSP structures
Commuted Mo del of Excitation and Bo dy
The body of the acoustic guitar is a complex vibrating structure Karjalainen et
al have rep orted that in order to fully mo del the resp onse of the body
a digital all p ole lter of order or more is required However this kind of
implementation is impractical since it would b e computationally far to o exp ensive for
real time applications Commuted waveguide synthesis Smith Karjalainen et
al can b e applied to include the b o dy resp onse in the synthetic guitar signal
Single Delay Lo op Mo del with Commuted Bo dy Resp onse
Excitation String Body
δ( n ) E (z ) S (z ) B (z ) y ( n )
δ( n ) B (z ) E (z ) S (z ) y ( n )
x ( n ) y ( n )
exc S (z )
Figure The principle of commuted waveguide synthesis On the top the
instrument mo del is presented as three linear lters In the middle the b o dy mo del
B z is commuted with the excitation and string mo dels E z and S z On the
b ottom the b o dy and excitation mo dels are convolved into a single resp onse x n
exc
that is used to excite the virtual guitar
in a computationally e cient manner It is based on the theory of linear systems
and particularly on the principle of commutation
In CWS the instrument mo del is interpreted as pictured on the top of Figure
i e as the excitation the vibrating string and the radiating b o dy These parts are
presented as linear lters with transfer functions E z S z andB z resp ectively
Since the system is excited with an impulse n the cascaded con guration implies
that the output signal y n is obtained as a convolution of the impulse resp onses
e n s n and b n of the three lters and the unit impulse n i e
y n n e n s n b n e n s n b n
olution op erator de ned as where denotes the conv
X
h n h n h k h n k
k
In the z transform domain Equation is expressed as
Y z E z S z B z
Since weapproximate the b ehavior of the instrument parts with linear lters we
can apply the principle of commutation and rewrite Equation as
Y z B z E z S z
as illustrated in the middle part of Figure In practice it is useful to convolve
the impulse resp onses b n and e n of the b o dy and excitation mo dels into a single
Chapter Digital Waveguides and Extended Karplus Strong Mo dels
C Sympathetic couplings Horizontal polarization to other strings
Sh (z ) m p m o out Pluck and Body H (z ) P (z ) Wavetable e g c 1- m p 1- m o Sympathetic couplings from S v (z ) other strings
Vertical polarization
Figure An extended string mo del with dual p olarization vibration and sym
pathetic coupling Karjalainen et al
impulse resp onse denoted by x n on the b ottom of Figure This signal is used
exc
to excite the string mo del and it can be precomputed and stored in a wavetable
Typically several excitation signals are used for one instrument The excitation
signal is varied dep ending on the string and fret p osition as well as on the playing
style Computation of the excitation signal from a recorded guitar tone is presented
in Section
There are several other ways to incorp orate a mo del of the instrument body in
a realizable form These are discussed by Karjalainen and Smith and they
include metho ds of reducing the lter order using a conformal mapping to warp the
frequency axis into a domain that b etter approximates the human auditory system
and of extracting the most prominent mo des in the body resp onse These mo des
are repro duced to the synthetic signal by computationally cheap lters Karjalainen
and Smith Tolonen
General Plucked String Instrument Mo del
The mo del illustrated in Figure exempli es a general mo del for the acoustic
guitar string employing the principle of commuted synthesis A library of di erent
plucktyp es and instrument b o dies is stored in a wavetable on the left The excitation
signal is mo di ed with a pluck shaping lter H z which brings ab out brightness
e
whichsimulates the e ect of the and gain control and a pluck p osition equalizer P z
excitation p osition to the synthetic signal The pluck p osition equalizer corresp onds
to the transfer function comp onent presented on top of Figure
After the excitation signal is fetched from a wavetable and ltered by transfer
functions H z and P z it is fed to the two string mo dels S z and S z in a
e h v
ratio determined by the gain parameter m The string mo dels simulate the e ect
p
typically of the two p olarizations of the transversal vibratory motion and they are
slightly mistuned in delay line lengths and decay rates to pro duce a natural sounding
synthetic tone The output signal is a sum of the outputs of the two p olarization
mo dels mixed in a ratio determined by m
o
In the instrument mo del of Figure sympathetic couplings b etween the strings
Single Delay Lo op Mo del with Commuted Bo dy Resp onse
1
0 Amplitude −1 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 1
0 Amplitude −1 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 1
0 Amplitude −1 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
Time (s)
Figure An example of the e ect of mistuning the p olarization mo dels S z
h
and S z Top equal parameter values middle mistuned decay rates and b ottom
v
mistuned fundamental frequencies
are implemented by feeding the output of the horizontal p olarization to a connection
matrix C which consists of the coupling co e cients The matrix is expressed as
g c c c
c N
c g c
c
C c c g
c
c g
N cN
where N is the number of dual p olarization strings the co e cients g for k
ck
th
N denote the gains of the output signal to b e sent from the k horizontal
th
string to its parallel vertical string and co e cients c are the gains of the k
mk
th
horizontal string output to be sent to the m vertical string Notice that the gain
th
terms g implement a coupling between the two p olarization in the k string and
ck
the gure With this kind that the co e cient g is also presented explicitly in
c
of structure it is p ossible to obtain a simulation of b oth sympathetic coupling
between string and coupling of the two p olarizations within a string The structure
is inherently stable since there are no feedback paths in the mo del Notice also that
with parameters m and m it is p ossible to change the con guration of the virtual
p o
instrument For instance by setting m the vertical p olarization will act as a
p
resonance string with the only input obtained from the horizontal p olarization
An example of the e ect of mistuning the two p olarization mo dels is shown
in Figure On the top of the gure the mo del parameters are equal and an
exp onential decay is resulted In the middle the fundamental frequencies of the
mo dels are equal but the lo op lter parameters di er from each other and a two
Chapter Digital Waveguides and Extended Karplus Strong Mo dels
1
0.5
0
Amplitude −0.5
−1 0 0.2 0.4 0.6 0.8 1 1
0.5
0
Amplitude −0.5
−1 0 0.2 0.4 0.6 0.8 1 1
0.5
0
Amplitude −0.5
−1 0 0.2 0.4 0.6 0.8 1
Time (s)
Figure An example of sympathetic coupling The output of a tone E played
th
on the string of the virtual guitar is plotted on the top In the middle the sum
of the outputs of the other virtual strings vibrating due to the sympathetic coupling
is illustrated The output of the virtual instrument i e the sum of all the string
outputs is presented on the b ottom
stage decay is pro duced On the b ottom gure the lo op lter parameters are equal
while the frequencies are mistuned to obtain a b eating e ect Another example is
pictured in Figure illustrating the sympathetic couplings between the strings
th
Tone E is played on the string of the virtual instrument and the vibration of
the string is so on damp ed by the player On the top part the output of the plucked
string is depicted In the middle the summed output of the other strings vibrating
sympathetically is illustrated On the b ottom the output of the virtual instrumentis
plotted Notice that the other strings continue to vibrate after the primary vibration
is damp ed
Analysis of the Mo del Parameters
After the instrument mo del has b een constructed b oth to closely simulate the phys
ical behavior of a real instrument and to be e ciently realizable in real time the
mo del parameters have to be derived It is natural to start by recording tones of a
Single Delay Lo op Mo del with Commuted Bo dy Resp onse
real acoustic guitar Since the recordings are treated as acoustic measurements of a
sound pro duction system they have to be p erformed carefully The side e ects of
the environment such as noise and the resp onse of the ro om should b e minimized
An analysis scheme is prop osed by Tolonen In this approach sinusoidal
mo deling is used to obtain the decaying partials of the guitar tone as separate
additive signal comp onents It is shown that the sinusoidal mo deling approach is
well suited for this kind of parameter estimation problem
Chapter Digital Waveguides and Extended Karplus Strong Mo dels
Evaluation Scheme
The sound synthesis metho ds presented in this do cument have b een develop ed to
di erent typ es of synthesis problems Thus it is not appropriate to compare these
metho ds with each other since the evaluation criteria no matter how carefully cho
sen would favor some of the metho ds The purp ose of the evaluation is to give some
guidelines on which metho ds are best suited for a given sound synthesis problem
The metho ds presented in this do cument were divided into four groups based
on a taxonomy presented by Smith to b etter compare techniques that are
closely related to each other The groups are abstract algorithms sampling and
pro cessed recordings sp ectral mo deling synthesis and physical mo deling This divi
sion is based on the premises of each sound synthesis metho d Abstract algorithms
create interesting sounds with metho ds that have little to do with sound pro duction
mechanisms in the real physical world Sampling and pro cessed recordings synthe
sis take existing sound events and either repro duce them directly or pro cess them
further to create new sounds Sp ectral mo deling synthesis uses information of the
prop erties of the sound as it is p erceived by the listener Physical mo deling attempts
to simulate the sound pro duction mechanism of a real instrument
This taxonomy can also b e interpreted as b eing based on tasks generated by the
user of the synthesis system For evaluation purp oses it is helpful to identify these
sound synthesis problems The tasks for which metho ds are b est suited are
Abstract algorithms
creation of new arbitrary sounds
computationally e cient mo derate quality synthesis of existing musical
instruments
Sampling and pro cessed recordings synthesis
repro duction of recorded sounds
merging and morphing of recorded sounds
using short sound bursts or recordings to pro duce new sound events
applications demanding high sound quality
Sp ectral mo dels
Chapter Evaluation Scheme
simulation and analysis of existing sounds
copy synthesis audio co ding
study of sound phenomena
pitch shifting time scale mo di cation
Physical mo dels
simulation and analysis of physical instruments
copy synthesis
study of the instrumentphysics
creation of physically unrealizable instruments of existing instrument
families
applications requiring control of high delity
Typically a sound synthesis metho d can be divided into analysis and synthesis
pro cedures These techniques are evaluated separately as they usually have di erent
requirements In many cases the analysis can be done o line and accuracy can be
gained by the cost of computation time The synthesis part has to b e typically done
have to b e available in real time and exible ways to control the synthesis pro cess
An excellent discussion on the evaluation of sound synthesis metho ds is given
by Ja e Ten criteria prop osed by Ja e are discussed in next section with
some additions These criteria are utilized to create the evaluation scheme used in
this do cument in the last three sections of the chapter In the next chapter the
evaluation scheme is applied to the synthesis metho ds presented in this do cument
The results are collected and tabulated to ease the comparison of the metho ds
The ten criteria address the usability of the parameters the quality diversity
and physicality of sounds pro duced and implementation issues One more criterion
is included in the evaluation scheme of this do cument It considers the suitability
for parallel implementation of the synthesis metho d These criteria are rated p o or
fair or go o d for each synthesis metho d
Usability of the Parameters
Four asp ects of parameters are discussed the intuitivity physicality and the be
havior of the parameters as well as the p erceptibility of parameter changes Ratings
used to judge the parameters are presented in Table
that a control parameter maps to a musical attribute By intuitivityit is meant
or quality of timbre in an intuitive manner With intuitive parameters the user is
easily able to learn how to control the synthetic instrument A signi cant parameter
change should b e p erceivable for the parameter to b e meaningful Such parameters
are called strong in contrast to weak parameters which cause barely audible changes
Ja e The trend is that the more parameters a synthesis system has the
Qualityand Diversity of Pro duced Sounds
weaker they are Ja e However to o strong parameters are hard to control
as a little change on the parameter value has a drastic change on pro duced sound
no matter how intuitivethe parameter is
Physical parameters provide the player of a synthetic instrument with the be
havior of a real world instrument They corresp ond to quantities the player of a real
instrument is familiar with such as string length bow or hammer velo city mouth
pressure in a wind instrument etc The behavior of a parameter is closely related
to the linearity of the parameters A change in a parameter should pro duce a
prop ortional change in the sound pro duced
The criteria presented in this section are tabulated in Table with ratings that
are used in the evaluation
poor fair go o d
Intuitivity
Perceptibility
Physicality
Behavior
Table Criteria for the parameters of synthesis metho ds with ratings used in
the evaluation scheme
Quality and Diversity of Pro duced Sounds
In this section criteria for the prop erties of pro duced sound are being discussed
These include the robustness of the sound s identity the generality of the metho d
and availability of analysis metho ds Ratings used to evaluate these criteria are
presented in Table
The robustness of the sound is determined byhowwell the identity of the sound
is retained when mo di cations to the parameters are presented This is to say
that e g a mo del of a clarinet should sound like a clarinet when played with
di erent dynamics and playing styles or even if the player decides to exp eriment
with the parameter values A general sound synthesis metho d is capable of pro ducing
arbitrary sound events of high quality Every existing sound synthesis metho d has
its short comings in generality and indeed every metho d do es not even attempt to
work at all with arbitrary sounds This criterion is still useful as one would hop e to
have as general metho ds as p ossible for several synthesis problems
For many sound synthesis metho ds an analysis metho d exists to derivesynthesis
parameters from recordings of sound signals This makes the use of the synthesis
metho d easier as it provides for default parameters that can then b e mo di ed to play
the synthetic instrument The analysis part is essential for many of the metho ds to
be useful at all In theory copy synthesis or otherwise optimal parameters can be
derived for most synthesis metho ds using di erent kinds of optimization metho ds
This is not typically desired but instead the analysis part often uses the knowledge
Chapter Evaluation Scheme
of the synthesis system to obtain reliable parameters
In many cases the analysis can b e done o line and typically it will only haveto
b e p erformed once for each instrumenttobemodeled Thus accuracy can b e given
much more weight on the cost of computing time and in this context computing
e ciency will be discarded as a criteria for analysis metho ds In this do cument
the analysis pro cedures of each synthesis system will be judged according to their
accuracy generality and demands for sp ecial devices or instruments
poor fair good
Robustness of identity
Generality
Analysis metho ds
Table Criteria for the quality and diversity of synthesis metho ds with ratings
used in the evaluation scheme
Implemention Issues
The implementation of a synthesis metho d has several imp ortant criteria to meet
E ciency of the techniques is judged latency and the control rate are estimated
Suitability for parallel implementation is also addressed Rating used to evaluate
the criteria concerning implementation issues is presented in Table
E ciency is further divided into three parts computational demands memory
usage and the load caused by control of the metho d In many cases the memory
comp ensated by increasing computational cost Ja e requirements can be
Computational cost is rated good if one or several instances of the metho d can easily
run in real time in an inexp ensive pro cessor fair if only one instance of the metho d
can run in real time in a mo dern desk top computer like PC or a workstation and
poor if a real time implementation is not p ossible without dedicated hardware or a
powerful sup ercomputer
The control stream of the metho d a ects b oth the expressivity and the compu
tational demands Typically more control is p ossible with dense control streams
than with sparse control streams Ja e Pro cessing of the control stream can
be a lot more cost de cient as it can involve I O with external devices or les In
this context the control stream is judged by examining the amount of control made
p ossible by the density of the stream
In real time synthesis systems there will always b e latency present as the system
needs to be causal to be realizable Latency is a problem esp ecially with metho ds
that employ blo ck calculations such as DFT Also with other computationally costly
synthesis metho ds it is sometimes advantageous to run tight lo ops for tens or mayb e
hundreds of output samples to sp eed up the calculation That can be a cause for
latency problems as well In the ratings poor means that the system will have
latency of tens or hundreds of milliseconds or more fair means that if there is not
y extra overhead caused by e g op erating system the latency will practically an
Implemention Issues
not be p erceivable and good indicates that the metho d is tolerantto some altering
overhead
poor fair good
Computational cost
Memory usage
Control stream
Latency
Parallel pro cessing
Table Criteria for the implementation issues of synthesis metho ds with ratings
used in the evaluation scheme
Suitability for parallel implementation can be an imp ortant factor in certain
situations In this context it is assumed that fast communication between parallel
pro cesses is available The system will be rated good on suitability for parallel
pro cessing if it can easily be divided into several pro cesses so that communication
between pro cesses happ ens approximately at the sampling rate level of the system
Rating fair is given if the system can be divided into two pro cesses communicating
at sampling rate level or if it is advantageous to distribute the computation at
higher communication level The metho d will b e judged poor if there is little or no
advantage to parallelize the pro cessing
The synthesis metho ds are rated in Table against the criteria presented in
this section
Chapter Evaluation Scheme
Evaluation of Several Sound
Synthesis Metho ds
In this chapter the sound synthesis metho ds presented in this do cument are eval
uated using the criteria discussed in the previous chapter Ratings are tabulated
for each metho d and they are also collected in Table to enable comparison of
the metho ds It should b e noted that the intention is not to decide which synthesis
metho d is the b est in general for that would be imp ossible Rather the evaluation
should give some guidelines up on which a prop er metho d can b e chosen for a given
sound synthesis problem
For some metho ds there are criteria that we feel we cannot evaluate and those
criteria are not rated
Evaluation of Abstract Algorithms
FM synthesis
The FM synthesis parameters are strong o enders in the criteria of intuitivity phys
icality and behavior as mo dulation parameters do not corresp ond to musical pa
rameters or parameters of musical instruments at all and b ecause the metho d is
highly nonlinear Thus it is rated poor in all these categories Notice however
that the mo dulation index parameter I is directly related to the bandwidth of the
pro duced signal The metho d has strong parameters i e parameters c hanges are
easily audible The rating in Perceptibilityis good
FM synthesis do es not b ehave well when it is used to mimic a real instrument
with varying dynamics and playing styles The parameters of the metho d have to
be changed very carefully in order not to lose the identity of the instrument The
metho d is rated poor for robustness of identity GeneralityofFMsynthesis is good
Analysis metho ds for FM have been prop osed but the metho ds do not apply well
for general cases thus the rating poor for analysis metho ds The interested reader is
referred to a work by Delprat for metho ds of extracting frequency mo dulation
laws by signal analysis
The e cient implementations of FM have made it a p opular metho d It is very
Chapter Evaluation of Several Sound Synthesis Metho ds
cheap to implement uses little memory and the control stream is sparse Minimal
latency makes the metho ds attractive for real time synthesis purp oses The metho d
is rated good for all these criteria FM synthesis is computationally so cheap that
distributing one FM instrument is not feasible Naturally several FM instruments
can be divided to run in several pro cessors
Waveshaping Synthesis
Waveshaping parameters are more intuitive fair than FM parameters esp ecially
when Chebyshev p olynomials are used as a shaping function Scaling of a weighting
gain of a single Chebyshev p olynomial only changes the gain of one harmonic Thus
p erceptible nor physical poor Dep ending on the the parameters are neither very
parameterization the parameters typically b ehave fairly well
Waveshaping is fairly general in that arbitrary harmonic sp ectra are easy to pro
duce By adding amplitude mo dulation after the waveshaping synthesis inharmonic
sp ectra can be pro duced Noisy signals cannot be generated easily Sp ectral analy
sis can easily be applied to obtain the amplitudes of each harmonic This data can
be directly used for gains to the Chebyshev p olynomials The rating for analysis
metho ds is thus good
Just as FM synthesis waveshaping can b e implemented very e ciently and dis
tribution of one instance is not feasible The metho d is rated good for computing
memory and control stream e ciency as well as for latency
Karplus Strong Synthesis
The few parameters of the Karplus Strong synthesis are very intuitive the changes
are easily audible and are well b ehaved Thus the rating for all these criteria is
good In the basic form the metho d only has a parameter for the pitch and one
for determining the typ e of tone e g string or p ercussion The physicality is thus
rated fair
string or a drum even KS synthesis is robust in that it will sound like a plucked
when the parameters are changed thus good for robustness of identity In generality
the metho d fails poorly Analysis techniques for KS synthesis are not available but
for related sound synthesis metho ds they exist see Section
Just like the other abstract algorithms the KS is very attractive to implementin
real time The ratings for implementation issues are the same as with FM synthesis
and waveshaping
Evaluation of Sampling and Pro cessed Recordings
Evaluation of Sampling and Pro cessed Record
ings
Sampling
In sampling synthesis a recording of a sound signal is played back with p ossible
lo oping in the steady state part Sampling is controlled just by note on o and gain
parameters Wehave decided not to give ratings of these trivial parameters in order
not to disturb the evaluation of other synthesis metho ds
Sampling is very general good in that any sound can b e recorded and sampled
The identity of the sound is retained with di erent playing styles and conditions
but at the cost of naturalness Robustness of identity is rated fair Analysis metho ds
for determining the lo oping breakp oints are available and usually give good results
with harmonic sounds
Sampling is computationally very e cient good but it uses lot of memory poor
Control stream is sparse good and latency time small good Distribution of one
sampling instrument is not feasible unless e g a server is utilized as a memory
storage The rating is fair
Multiple Wavetable Synthesis
Multiple wavetable synthesis metho ds can b e parameterized in various ways and the
result of synthesis is highly dep endent of the signals stored in wavetables Thus we
decided not to give ratings on parameters of the metho d or the robustness of sounds
identity
The metho d is general good and analysis metho ds for some implementations
are available fair
The metho d is fairly easily implemented computationally but it uses a lot of
memory poor Control stream is not very costly computationally good and latency
times can b e kept small good Just like with sampling a separate wavetable server
can reduce the memory requirements of a single instance of multiple wavetable
synthesis provided that fast connections are available Suitability for distributed
parallel pro cessing is rated fair
Granular synthesis
ary quite a lot from each other in Granular synthesis is a set of techniques that v
parameterization and implementation Here a general evaluation of the concept is
attempted In the most primitive form the parameters of granular synthesis control
the grains directly The numb er of grains is typically very large and more elab orate
means to control them must be utilized The parameters are thus rated poor in
intuitivity p erceptibility and physicality The system is linear and the b ehavior of
the parameters is good
Chapter Evaluation of Several Sound Synthesis Metho ds
Analysis metho ds for the pitch synchronous granular synthesis exist and they
are also e cient good As the asynchronous metho d do es not attempt to mo del or
repro duce recorded sound signals no analysis to ols are necessary Granular synthesis
metho ds are general good and with the PSGS the robustness of identity is retained
well good
The implementation of the metho d is fairly e cient and also the memory re
quirements are fair as the grains are short and it is typically assumed that the
signals can be comp osed of few basic grains The low level control stream can be
come very dense esp ecially with AGS poor The metho d do es not p ose latency
problems good and the suitability for parallel pro cessing is rated fair
Evaluation of Sp ectral Mo dels
Basic Additive Synthesis
Parameters of the basic additivesynthesis control directly the sinusoidal oscillators
Ways to reduce the control data are available and some of them will b e discussed with
other sp ectral mo deling metho ds In this context only the basic additive synthesis
is discussed The parameters are fairly intuitive in that frequencies and amplitudes
are easy to comprehend The b ehavior of parameters is good as the metho d is linear
Perceptivity and physicalityofthe parameters is poor
Additive synthesis can in theory synthesize arbitrary sounds if an unlimited
number of oscillators is available This so on b ecomes impractical as noisy signals
cannot b e mo deled e ciently and thus the generality is rated fair Analysis metho ds
good based on e g STFT are readily available as additive synthesis is used as
a synthesis part of some of the other sp ectral mo deling metho ds Robustness of
identity is not evaluated as the control of a synthetic instrument would need more
elab orate control techniques
A single sinusoidal oscillator can b e implemented e ciently but in additive syn
is required Computational cost is rated thesis typically a large number of them
fair The control data requires a large memory poor and the control stream is
very dense poor Latency time is small as the oscillators run in parallel good
Parallel implementation can b ecome feasible when the number of oscillators grows
large In distributed implementation the oscillators and corresp onding controls have
to b e group ed The suitabilityfor distributed parallel pro cessing is rated fair
FFT based Phase Vo co der
The parameters of the FFT based phase vo co der are directly related with the STFT
analysis such as the FFT length windowlengthandtyp e and the hop size param
they can be comprehended in the case eter While these are not intuitive directly
of e g time scale mo di cations or pitch shifts Intuitivity is rated fair Parame
ters likethe hop size are relatively strong whereas the windowtyp e might not have
Evaluation of Sp ectral Mo dels
any signi cant e ect except in some sp eci c situations Perceptibility is rated fair
Physicality of the parameters is poor and behavior of the parameters is good if the
changes are taken into account in the analysis stage as well
The metho d retains the identity of the mo deled instrument well esp ecially if
time varying time scale mo di cation is applied Serra a good Generality is
good and it is heavily based on the analysis stage good
relatively e ciently fair by The implementation of the metho d can be done
using the FFT The memory requirements are fair and the control stream is dense
poor as the phase vocoder uses a transform of an original signal Latency time is
large poor b ecause of the blo ck based FFT computation Suitability for parallel
pro cessing is fair as the synthesis stage is mainly comp osed of IFFT It was decided
that all FFT based metho ds are rated fair on suitability for parallel pro cessing since
although the computation of an FFT can be e ciently parallelized it is typically
computed in a single pro cess
McAulay Quatieri Algorithm
The McAulay Quatieri algorithm is based on a sinusoidal representation of signals
It uses additive synthesis as its synthesis part and can thus be interpreted as an
analysis and data reduction metho d for the simple additive synthesis
The control parameters of the MQ algorithm consist of amplitude frequency
and phase tra jectories These tra jectories are interp olated to obtain the additive
synthesis parameters As with the phase vocoder the intuitivity is rated fair the
p erceptibility fair physicality poor and behavior good
The algorithm works with arbitrary signals if the number of sinusoidal oscillators
is increased accordingly but is infeasible for noisy signals fair Analysis metho d is
good and the sound identity is retained fairly well with mo di cations
The implementation is fairly e cient The control stream fair is reduced by
the cost of interp olation of tra jectories The tra jectories take less memory than the
envelop es of the additive synthesis and the memory usage is rated fair Latency of
the synthesis part is better fair than in the phase vocoder as it is related to the
hop size instead of the FFT size Suitability for parallel pro cessing is rated fair as
with additivesynthesis
Source Filter Synthesis
signal The parameters of source lter mo deling include the choice of excitation
fundamental frequency if it exists and the co e cients of the time varying lter
These do not seem very intuitive but when the lter is parameterized prop erly
formants can be controlled easily Intuitivity is thus rated fair Perceptibility also
is rated fair as changes in excitation signal are easily audible The audibility of
lter parameters dep ends again on parameterization When source lter synthesis
is applied to simulate the human sound pro duction system the parameters are fairly
Chapter Evaluation of Several Sound Synthesis Metho ds
physical as the formants corresp ond to the shap e of the vo cal tract In time varying
ltering transition e ects caused by up dating the lter parameters are problematic
and can easily b ecome audible as disturbing artifacts This causes the behavior of
parameters to b e rated poor
Source lter synthesis is general good in that it can in theory pro duce any
sound For example linear prediction o ers an analysis metho d to obtain the lter
co e cients and inverse ltering can b e utilized to obtain an excitation signal good
Robustness of identity fair dep ends on the parameterization but it can be easily
lost if the lter parameters are not up dated carefully
Excitation and ltering are fairly e cient to implement The metho d do es not
require a great deal of memory fair The control stream dep ends on the mo deled
signal for steady state signal it is very sparse but for sp eech the lter co e cients have
to be up dated every few milliseconds fair Latency time of source lter synthesis
is small good Parallel distribution do es not seem to p ose any great advantages as
a large part of the computational cost comes from lter co e cient up dates poor
Sp ectral Mo deling Synthesis
Sp ectral mo deling synthesis uses additive synthesis to pro duce the deterministic
harmonic comp onent and source lter synthesis to pro duce the sto chastic noisy
comp onent of the synthetic signal The parameters consist of amplitude and fre
quency tra jectories of the deterministic comp onen t and sp ectral envelop es of the
sto chastic part These parameters are rated fair in intuitivity For mo di cation of
the analyzed signal to be meaningful higher level controls have to be utilized to
reduce the control data Perceptibility and physicality are thus rated poor The
behavior of the parameters is good
Robustness of identity is rated good as the comp osition of the signal provides
This allows for means to edit the deterministic and sto chastic part separately
b etter control of attack and steady state parts as with e g the phase vo co der
Sp ectral mo deling synthesis is judged to b e good in generality and analysis metho d
Computational cost is reasonable fair The metho d requires more memory than
the MQ algorithm but it is still rated fair in memory usage The control stream is
fairly sparse as the additive source lter parameters are interp olated b etween STFT
frames Latency time is in the order of that of the MQ algorithm fair Additive
and source lter synthesis can b e divided into separate parallel pro cessors and thus
the suitability for distributed parallel pro cessing is fair
Transient Mo deling synthesis
the sp ectral mo deling synthesis in Transient mo deling synthesis is an extension to
that it allows for further pro cessing of the residual signal as separate noise and
transient signals It is fair to say that TMS is more general than SMS and that it
also involves more computation The two metho ds are close enough for the ratings
to b e identical
Evaluation of Sp ectral Mo dels
FFT
FFT is an additive synthesis metho d in the frequency domain that is also capable
of pro ducing noisy signals The parameters consist of frequencies and amplitudes
of partials and of bandwidths and amplitudes of noisy comp onents They are rated
fair in intuitivity The parameters are poorly p erceptible as in a complex signal the
number of signal comp onents can be very large They are not physical poor but
they are linear and b ehave well good
The metho d is general good as it can pro duce harmonic inharmonic and noisy
signals STFT provides a good analysis metho d As with additive synthesis the
robustness of sound identityisnot evaluated for FFT
The implementation of the metho d is e cient good The memory usage is fair
but the control stream can b ecome very dense poor when the number of signal
comp onents increases FFT is a blo ck based metho d and su ers from latency
problems poor As the metho d uses IFFT to in the synthesis stage it is rated fair
for suitability for parallel pro cessing
Formant Wave Function Synthesis
The parameters of FOF synthesis govern the fundamental frequency and the struc
ture of the formants of synthesized sound signals The parameters can be judged
fairly in tuitive and physical esp ecially when the metho d is used for simulation of
the human sound pro duction system Perceptibility is good as there are typically
only a few formants present in sp eech or singing voice signals The parameters are
well b ehaved good
The metho d is fairly general as it can pro duce high quality harmonic sound
signals of singing and musical instruments Linear prediction provides for an analysis
metho d good and the sounds pro duced retain their identitywell good as is proven
by sound examples Bennett and Ro det
The metho d can be implemented e ciently when di erent FOFs are stored in
wavetables good This increases the amount of required memory fair The control
rate is fairly sparse and the latency time is small good Parallel pro cessing of a
single FOF instrument do es not seem feasible as the excitation signal is shared by
all of the FOF generators poor
VOSIM
The parameters of the VOSIM mo del are not very well related to either the sound
sound itself Thus physicality and pro duction mechanism being mo deled or the
intuitivity of the parameters are b oth rated poor The p erceptibility is rated fair as
some of the parameters are strong and some weak The behavior is also rated fair
b ecause although the metho d is linear the e ect of each parameter to the sound
pro duced maynotbe very well b ehaved
Chapter Evaluation of Several Sound Synthesis Metho ds
The metho d is fairly general as it has b een used to mo del the human sound pro
duction mechanism and some musical instruments An e cient analysis metho d was
not found in the literature poor The parameterization suggests that the metho d
is not robust with parameter mo di cation poor VOSIM can be implemented ef
ciently and it only requires a small amount of memory b oth rated good The
control stream is sparse good and latency time can be kept small good There is
little advantage in parallel implementation of the system poor
Evaluation of Physical Mo dels
Finite Di erence Metho ds
The parameters of nite di erence metho ds corresp ond directly to the physical pa
rameters of the mo deled sound pro duction system Thus they are very physical and
intuitive and can also b e rated good for p erceptibility and b ehavior as the vibratory
motion is assumed linear
Although the metho d can be applied in theory to arbitrary sound pro duction
systems a new implementation is typically required as the instrument under study
is changed Thus FD metho ds are rated fair in generality A tuned mo del of an
instrumentbehaves very much like the original and retains the identitywell good
Analysis metho ds are available but although the results can b e very go o d they often
and require a great deal of time and include sp ecialized measurement instruments
e ort fair
FD metho ds are computationally very ine cient poor and they also need a fair
amountofmemory The control stream dep ends on the excitation but is very sparse
with plucked or struck strings and with mallet instruments good The metho d
do es not p ose a problem with latency times if su cient computational capacity is
available good The metho d is well suited good for distributed parallel pro cessing
as signi cant improvements can be achieved by dividing the system into several
substructures running as di erent pro cesses
Mo dal Synthesis
Mo dal synthesis parameters consist of the mo dal data of the mo deled structure and
the excitation or driving data The parameters are not very intuitive fair and a
change in a single mo de can be hard to p erceive if the number of mo des is large
poor They corresp ond directly to physical structures and the physicality and the
behavior are rated good
Analysis metho ds for the system are available and they pro duce reliable and
accurate results However they su er from b eing very complicated and exp ensive
fair The system is general as any vibrating ob ject can b e formulated as its mo dal
sounds related to no physical ob jects are not easily pro duced by data Arbitrary
the mechanism Generalityisrated fair The mo deled structure retains its identity
Evaluation of Physical Mo dels
very well good asitistypically controlled by the excitation signal
The metho d is implemented as a set of parallel second order resonators that are
computationally e cient The number of resonators can grow very large and thus
the computational e ciency is rated fair The mo dal data requires a large amountof
memory poor The excitation signal de nes the control stream for static structures
and it can be rated sparse good The substructures can be e ciently distributed
and pro cessed in parallel good
CORDIS
A clear description of the CORDIS system parameters was not found in the litera
ture For this reason the parameters of CORDIS are not evaluated
uses a physics based description of the system that vibrates it As the metho d
retains the identity of sound well good with di erent meaningful excitation signals
No analysis system was describ ed in the literature poor The generality of the
system is fair as it can in theory mo del arbitrary vibrating ob jects The metho d
do es not provide an easy way to create arbitrary sounds
The metho d is judged to b e computationally fair as although the basic elements
can be computed e ciently a large number of them is needed This accounts also
parame for rating fair for memory requirements The control rate dep ends on the
terization and is not evaluated here The latency time of CORDIS is small good It
app ears that CORDIS maybewell suited for parallel pro cessing good Ro cchesso
Digital Waveguide Synthesis
Digital waveguide synthesis parameters are intuitiveandphysical as they corresp ond
well to physical structures of the instrument and the way it is b eing played b oth
rated good The parameter changes are typically audible good and they b ehave
well with linear mo dels As some of the waveguide mo dels have nonlinear couplings
the behavior is rated fair
The identity of the instrument is retained very well good The metho d can be
used to simulate instruments with one dimensional vibrating ob jects such as string
and wind instruments and it is thus rated fair in generality Automated analysis
metho ds for linear mo dels are e cient but they are not available for nonlinear mo dels
fair
A digital waveguide can b e implemented very e ciently but typically the mo del
also incorp orates other structures that increase the computational requirements
Computational e ciency is rated fair Digital waveguide mo dels require little mem
ory other than the excitation signal A high quality plucked string tone of several
seconds can b e pro duced with only several thousands words of memory good The
control stream dep ends on the instrument b eing mo deled and is here rated fair The
metho d do es not p ose latency problems and esp ecially mo dels with several vibrating
Chapter Evaluation of Several Sound Synthesis Metho ds
structures can b e e ciently divided into substructures that are computed in parallel
b oth rated good
Waveguide Meshes
The parameters of the waveguide meshes are fairly intuitive as they corresp ond to
the excitation and the prop erties of the D or D vibrating system Parameters are
physical and p erceptible and they behave well as the mesh is linear rated good for
all those criteria
Analysis metho ds were not found in the literature poor The metho d is fairly
general as it can b e applied to simulation of D and D ob jects The robustness of
identity is not evaluated
The metho d is computationally exp ensive and requires a large amount of memory
b oth rated poor The control stream is fairly sparse as it consists only of the
excitation information The metho d itself do es not p ose latency problems good
although real time implementations of more complex structures cannot b e achieved
without exp ensive sup ercomputers One of the main advantages of the mo del is that
it can b e divided into arbitrary substructures that are computed in parallel good
Commuted Waveguide Synthesis
Commuted digital waveguides have been used to pro duce high quality synthesis
of instruments that can be describ ed as having linear or linearizable coupling of
excitation to the vibrating structure The parameters are very intuitive p erceptible
aveguide synthesis is rated good for those physical and well b ehaved and commuted w
criteria
The metho d is very good in retaining the identity of the mo deled instrument For
go o d synthesis results parameters need to b e derived by analysis of existing instru
ments The analysis metho ds employ STFT and pro duce good results The metho d
is fairly general as a numb er of p ercussive plucked or struck string instruments can
be mo deled with commuted synthesis
The implementation issues of commuted waveguide synthesis are very close to
digital waveguide synthesis The ratings are the same and they will be rep eated
for convenience Computational e ciency and control stream are rated fair and
memory usage latency and suitability for parallel pro cessing good
Results of Evaluation
Results of Evaluation
The evaluation results discussed in the previous sections are tabulated in Table
It can be observed that the abstract algorithms and sampling techniques are
strongest int the implementation category Sp ectral mo dels are general robust and
analysis metho ds are available They are strongest in the sound category Phys
ical mo deling employs very intuitive parameterization and it is strongest in the
parameter category
Chapter Evaluation of Several Sound Synthesis Metho ds Comm W W FD CORDIS dal Mo Ph V CHANT FFT TMS SMS Source lter MQ Phase A ectral Sp Gran WT Multiple Sampling Sampling KS W FM Abstract OSIM dditiv a a G ysical v v eh ds metho eguide eshaping Meshes ular tdW uted V e ocoder T able G In T t abulated P P erc arameters ev Ph laino h on syn sound the of aluation ys Beha v Robust Sound hssmtod presen ds metho thesis Gen Anal Comp e nti ocumen do this in ted Implemen Mem Con tation tr t Lat P ar
Summary and Conclusions
In this do cument several mo dern sound synthesis metho ds have been discussed
The metho ds were divided into four groups according to a taxonomy prop osed by
Smith Representative examples in each group were chosen and a describtion
of those metho ds was given The interested reader was referred to the literature for
more information on the metho ds
Three metho ds based on abstract algorithms were chosen FM synthesis wave
shaping synthesis and the Karplus Strong algorithm Also three metho ds utilizing
recordings of sounds were discussed These are sampling multiple wavetable syn
thesis and granular synthesis
In the sp ectral mo deling category three traditional linear sound synthesis meth
o ds namely additive synthesis the phase v o co der and source lter synthesis were
rst discussed Second McAulay Quatieri algorithm Sp ectral Mo deling Synthesis
TransientModelingSynthesis and the inverse FFT based additivesynthesis metho d
FFT synthesis were describ ed Finally two metho ds for mo deling the human
voice were shortly addressed These metho ds are the CHANT and the VOSIM
Three physical mo deling metho ds that use numerical acoustics were investigated
First mo dels using nite di erence metho ds were presented Applications to string
instruments as well as to mallet p ercussion instruments were presented Second
mo dal synthesis was discussed Third CORDIS a system of mo deling vibrating
ob jects by mass spring networks was describ ed
Con tinuing in the physical mo deling category digital waveguides were discussed
Waveguide meshes which are D and D mo dels were also presented Extensions
and physical mo deling interpretation of the Karplus Strong algorithm was discussed
and single delay lo op SDL mo dels were describ ed Finally a case study of mo deling
the acoustic guitar using commuted waveguide synthesis was presented
After the metho ds in the four categories were discussed evaluation criteria based
on those prop osed by Ja e were describ ed One additional criterion was
added addressing the suitability of a metho d for parallel pro cessing Each metho d
was evaluated with a discussion concerning each evaluation criterion The criteria
were rated with qualitative measure for each metho d Finally the ratings were tab
ulated in a comparable form It was observed that abstract algorithms and sampling
techniques are strongest in the implementation category Sp ectral mo dels are gen
Chapter Summary and Conclusions
eral robust and analysis metho ds are available They are strongest in the sound
category Physical mo deling algorithms employvery intuitive parameterization and
are strongest in the parameter category
Bibliography
Adrien J M Dynamic mo deling of vibrating structures for sound synthesis
mo dal synthesis Proceedings of the AES th International Conference Audio
Engineering So ciety Toronto Canada pp
Adrien J M The missing link mo dal synthesis in G D Poli A Pic
cialli and C Roads eds Representations of Musical Signals The MIT Press
Cambridge Massachusetts USA pp
Ar b D Digital synthesis of complex sp ectra by means of multiplica
tion of nonlinear distorted sine waves Journal of the Audio Engineering Society
Bate J A The e ect of mo dulator phase on timbres in FM synthesis Com
puter Music Journal
Bennett G and Ro det X Synthesis of the singing voice in M V Mathews
and J R Pierce eds Current Directions in Computer Music Research The MIT
Press Cambridge Massachusetts chapter pp
Borin G and Giovanni D P A hysteretic hammer string interaction
mo del for physical mo del synthesis Proceedings of the Nordic Acouctical Meet
ing Helsinki Finland pp
Borin G De Poli G and Ro cchesso D a Elimination of delay free lo ops
in discrete time mo dels of nonlinear acoustic systems Proceedings of the IEEE
Workshop of Applications of Signal Processing to Audio and Acoustics NewPaltz
New York
Borin G De Poli G and Sarti A b Musical signal synthesis in C Roads
S T Pop e A Piccialli and G De Poli eds Musical Signal Processing Swets
Zeitlinger Lisse the Netherlands chapter pp
Bristow Johnson R Wavetable synthesis a fundamental p ersp ective
the st AES convention in Los Angeles California Proceedings of
Cadoz C Luciani A and Florens J Resp onsive input devices and sound
synthesis by simulation of instrumental mechanisms the CORDIS system Com
puter Music Journal
BIBLIOGRAPHY
Cavaliere S and Piccialli A Granular synthesis of musical signals in
C Roads S T Pop e A Piccialli and G De Poli eds Musical Signal Pro
cessing Swets Zeitlinger Lisse the Netherlands chapter pp
Chaigne A Visco elastic prop erties of nylon guitar strings Catgut Acoustical
Society Journal
Chaigne A On the use of nite di erences for musical synthesis Application
to plucked stringed instruments Journal d Acoustique
Chaigne A and Askenfelt A a Numerical simulations of piano strings I A
struck string using nite di erence metho ds Journal of the physical mo del for a
Acoustical Society of America
Chaigne A and Askenfelt A b Numerical simulations of piano strings II
Comparisons with measurements and systematic exploration of some hammer
string parameters Journal of the Acoustical Society of America
Chaigne A and Doutaut V Numerical simulations of xylophones I Time
domain mo deling of the vibrating bars Journal of the Acoustical Society of Amer
ica
Chaigne A Askenfelt A and Jansson E V Temp oral synthesis of string
instrument tones Quarterly Progress and Status R eport number SpeechTrans
mission Lab oratory Royal Institute of Technology KTH Sto ckholm Sweden
pp
Chowning J M The synthesis of comples audio sp ectra by means of fre
quency mo dulation Journal of the Audio Engineering Society
ReprintedinC RoadsandJ Strawn eds Foundations of Computer Music
Cambridge Massachusetts The MIT Press pp
Co ok P R TBone and interactive waveguide brass instrument synthesis
workb ench for the NeXT machine Proceedings of the International Computer
Music Conference Montreal Canada pp
Co ok P R A meta wind instrument physical mo del and a meta controller
for real time p erformance control Proceedings of the International Computer Mu
sic Conference International Computer Music Asso ciation San Francisco CA
pp
Co ok P R SPASM a real time vo cal tract physical mo del controller
and singer the companion software synthesis system Computer Music Journal
De Poli G A tutorial on digital sound synthesis techniques Computer Music
Journal Also published in Roads C ed The Music Machine
bridge Massachusetts USA pp The MIT Press Cam
De Poli G and Piccialli A Pitch synchronous granular synthesis in G D
Poli A Piccialli and C Roads eds Representations of Musical Signals The
MIT Press Cambridge Massachusetts USA pp
BIBLIOGRAPHY
Delprat N Global frequency mo dulation laws extraction from the Gab or
transform of a signal a rst study of the interacting comp onents case IEEE
Transactions on Speech and Audio Processing
Dietz P H and Amir N Synthesis of trump et tones by physical mo deling
Proceedings of the International Symposium on Musical Acoustics pp
Dolson M The phase vocoder a tutorial Computer Music Journal
Record Dudley H The vo co der Bel l Laboratories
Eckel G Iovino F and Causs R Sound synthesis by physical mo delling
with Mo dalys Proceedings of the International SymposiumonMusical Acoustics
Dourdan France pp
Evangelista G Pitch synchronous wavelet representation of sp eech and music
signals IEEE Transactions on Signal Processing
Evangelista G Comb and multiplexed wavelet transforms and their applica
tions to signal pro cessing IEEE Transactions on Signal Processing
Evangelista G Wavelet representations of musical signals in C Roads
S T Pop e A Piccialli and G De Poli eds Musical Signal Processing Swets
Zeitlinger Lisse the Netherlands chapter pp
Fitz K and Haken L Sinusoidal mo deling and manipulation using Lemur
Computer Music Journal
Flanagan J L and Golden R M Phase Vo co der The Bel l System Technical
Journal
Fletcher N H and Rossing T D The Physics of Musical Instruments
Springer Verlag New York USA p
simulating Florens J L and Cadoz C The physical mo del mo deling and
the instrumental universe in G D Poli A Piccialli and C Roads eds Repre
sentations of Musical Signals The MIT Press Cambridge Massachusetts USA
pp
Fontana F and Ro cchesso D A new formulation of the D waveguide mesh fo
p ercussion instruments Proceedings of the XI Col loquium on Musical Informatics
Bologna Italy pp
Go o dwin M and Gogol A Overlap add synthesis of nonstationary sinusoids
Proceedings of the International Computer Music Conference Ban Canada
pp
Go o dwin M and Ro det X E cient Fourier synthesis of nonstationary si
nusoids Proceedings of the International Computer Music Conference Aarhus
Denmark pp
BIBLIOGRAPHY
Go o dwin M and Vetterli M Time frequency signal mo dels for music analysis
transformation and synthesis Proceedings of the rdIEEESymposium on Time
Frequency and Time Scale Analysis Paris France
Gordon J W and Strawn J An intro duction to the phase vo co der in
J Strawn ed Digital Audio Signal Processing An Anthology William Kau
mann Inc chapter pp
Harris F J On the use of windows for harmonic analysis with the discrete
Fourier transform Proceedings of the IEEE
and Ruiz P a Synthesizing musical sounds by solving the wave Hiller L
equation for vibrating ob jects part Journal of the Audio Engineering Society
Hiller L and Ruiz P b Synthesizing musical sounds by solving the wave
equation for vibrating ob jects part Journal of the Audio Engineering Society
Hirschman S E Digital Waveguide Mo deling and Simulation of Reed Wood
wind Instruments Technical Report STAN M Stanford University Dept of
Stanford California Music
Holm F Understanding FM implementations acallfor common standards
Computer Music Journal
Horner A Beauchamp J and Haken L Metho ds for multiple wavetable
synthesis of musical instrument tones Journal of the Audio Engineering Society
Huopaniemi J Karjalainen M V lim ki V and Huotilainen T Virtual
instruments in virtual ro oms a real time binaural ro om simulation environment
for physical mo deling of musical instruments Proceedings of the International
Computer Music Conference Aarhus Denmark pp
Ja e D A Ten criteria for evaluating syn thesis techniques Computer Music
Journal
Ja e D A and Smith J O Extensions of the Karplus Strong plucked
string algorithm Computer Music Journal Also published in Roads
C ed The Music Machine pp The MIT Press Cambridge
Massachusetts USA
Kaegi W and Temp elaars S VOSIM a new sound synthesis system Journal
of the Audio Engineering Society
Karjalainen M and Laine U K A mo del for real time sound synthesis of
guitar on a oating p oint signal pro cessor Proceedings of the IEEE International
ocessing Vol Toronto Canada ConferenceonAcoustics Speech and Signal Pr
pp
BIBLIOGRAPHY
Karjalainen M and Smith J O Bo dy mo deling techniques for string in
strument synthesis Proceedings of the International Computer Music Conference
Hong Kong pp
Karjalainen M Huopaniemi J and V lim ki V Direction dep endentphys
ical mo deling of musical instruments Proceedings of the International Congress
on Acoustics Vol Trondheim Norway pp
Karjalainen M Laine U K and V lim ki V Asp ects in mo deling and
real time synthesis of the acoustic guitar Proceedings of the IEEE Workshop of
ations of Signal Processing to Audio and Acoustics New Paltz New York Applic
USA
Karjalainen M V lim ki V and J nosy Z Towards high quality sound
synthesis of the guitar and string instruments Proceedings of the International
Computer Music Conference Tokyo Japan pp
Karjalainen M V lim ki V and Tolonen T Plucked string mo dels from
Karplus Strong algorithm to digital waveguides and beyond Accepted for publi
cation in Computer Music Journal
ked string and drum tim Karplus K and Strong A Digital synthesis of pluc
bres Computer Music Journal Also published in Roads C ed
The Music Machine pp The MIT Press Cambridge Massachusetts
Kurz M and Feiten B Physical mo delling of a sti string by numerical
integration Proceedings of the International Computer Music Conference Hong
Kong pp
Laakso T I V lim ki V Karjalainen M and Laine U K Splitting the unit
delay to ols for fractional delay lter design IEEE Signal Processing Magazine
Lang M and Laakso T I Simple and robust metho d for the design of allpass
lters using least squares phase error criterion IEEE Transactions on Circuits and
Systems II Analog and Digital Signal Processingx
Laro che J andDolson M Ab out this phasiness business Proc eedings of the
International Computer Music Conference Thessaloniki Greece pp
Laro che J and Jot J M Analysis synthesis of quasi harmonic sound by
nd
use of the Karplus Strong algorithm Proceedings of the French Congress on
Acoustics Archachon France
Le Brun M Digital waveshaping synthesis Journal of the Audio Engineering
Society
Makhoul J Linear prediction a tutorial review Proceedings of the IEEE
BIBLIOGRAPHY
McAulay R J and Quatieri T F Sp eech analysis synthesis based on a
sinusoidal representation IEEE Transactions on Acoustics Speech and Signal
Processing
Mo ore F R Elements of Computer Music Prentice Hall Englewood Cli s
New Jersey
Mo orer J A The use of the phase vocoder in computer music applications
Journal of the Audio Engineering Society
music Mo orer J A The use of linear prediction of sp eech in computer
applications Journal of the Audio Engineering Society
Mo orer J A Signal pro cessing asp ects of computer music a survey in
J Strawn ed Digital Audio Signal Processing An Anthology William Kau
mann Inc chapter pp
Morrison J and Adrien J MOSAIC a framework for mo dal synthesis
Computer Music Journal
Morse P M and Ingard U K Theoretical Acoustics Princeton University
Press Princeton New Jersey USA
Msallam R Dequidt S Tassart S and Causs R Physical mo del of the
trombone including nonlinear propagation e ects Proceedings of the Institute of
Acoustics Vol pp Presented at the International Symp osium on
Musical Acoustics Edinburgh UK
Nuttall A H Some windows with very go o d sidelob e b ehavior IEEE Trans
actions on Acoustics Speech and Signal Processing
Opp enheim A V Willsky A S and Young I T Signals and Systems
Pren tice Hall New Jersey USA p
Paladin A and Ro cchesso D A disp ersive resonator in real time on MARS
workstation Proceedings of the International Computer Music Conference San
Jose California USA pp
Portno M R Implementation of the digital phase vocoder using the fast
Fourier transform IEEE Transactions on Acoustics Speech and Signal Processing
Rank E and Kubin G A waveguide mo del for slapbass synthesis Proceedings
of the IEEE International ConferenceonAcoustics Speech and Signal Processing
Germany pp Vol Munich
Risset J C Computer music exp eriments Computer Music Journal
Also published in Roads C ed The Music Machine pp
The MIT Press Cambridge Massachusetts USA
BIBLIOGRAPHY
Roads C Asynchronous granular synthesis in G D Poli A Piccialli and
C Roads eds Representations of Musical Signals The MIT Press Cambridge
Massachusetts USA pp
Roads C The Computer Music Tutorial The MIT Press Cambridge Mas
sachusetts USA p
Ro cchesso D Personal communication
Ro cchesso D and Scalcon F Accurate disp ersion simulation for piano strings
Proceedings of the Nordic Acouctical Meeting Helsinki Finland pp
real time sound Ro cchesso D and Turra F A generalized excitation for
synthesis byphysical mo dels Proceedings of the Stockholm Music Acoustics Con
ference Stockholm Sweden pp
Ro det X Time domain formant wave function synthesis Computer Music
Journal
Ro det X and Depalle P a A new additive synthesis metho d using inverse
Fourier transform and sp ectral envelop es in A Strange ed Proceedings of the
International Computer Music Conference pp
Ro det X and Depalle P b Sp ectral envelop es and inverse FFT synthesis
Proceedings of the rd AES convention San Francisco California
Ro det X Potard Y and Barri re J B The CHANT pro ject from synthesis
of the singing voice to synthesis in general Computer Music Journal
Savio ja L and V lim ki V The bilinearly deinterp olated waveguide mesh
Proceedings of the IEEE Nordic Signal Processing Symposium Esp o o Fin
land pp
Savio ja L and V lim ki V Improved discrete time mo deling of multi
dimensional wave propagation using the interp olated digital waveguide mesh Pro
ceedings of the IEEE International Conferenc e on Acoustics Speech and Signal
Processing Vol Munich Germany
Serra M H a Intro ducing the phase vo co der in C Roads S T Pop e
A Piccialli and G De Poli eds Musical Signal Processing Swets Zeitlinger
Lisse the Netherlands chapter pp
Serra X A System for Sound Analysis Transformation Synthesis Based on
a Deterministic plus Stochastic Decomposition PhD thesis Stanford University
California USA p
b Musical sound mo deling with sinusoids plus noise in C Roads Serra X
S T Pop e A Piccialli and G De Poli eds Musical Signal Processing Swets
Zeitlinger Lisse the Netherlands chapter pp
BIBLIOGRAPHY
Serra X and Smith J O Sp ectral mo deling synthesis a sound analy
sis synthesis system based on a deterministic plus sto chastic decomp osition Com
puter Music Journal
Smith J O Techniques for Digital Filter Design and System Identi cation
with Application to the Violin PhD thesis Stanford University California USA
p
Smith J O E cientsimulation of the reed b ore and b ow string mechanisms
Proceedings of the International Computer Music Conference The Hague the
Netherlands pp
Smith J O Music applications of digital waveguides Technical Report STAN
p M CCRMA Dept of Music Stanford University California USA
Smith J O Viewp oints on the history of digital synthesis Proceedings of the
International Computer Music Conference Montreal Canada pp
Smith J O Physical mo deling using digital waveguides Computer Music
Journal
Smith J O E cient synthesis of stringed musical instruments Proceedings
of the International Computer Music Conference Tokyo Japan pp
Smith J O Intro duction to digital waveguide mo deling of musical instru
ments Unpublished manuscript
Smith J O Physical mo deling synthesis up date Computer Music Journal
Smith J O Acoustic mo deling using digital waveguides in C Roads
S T Pop e A Piccialli and G De Poli eds Musical Signal Processing Swets
Zeitlinger Lisse the Netherlands chapter pp
Smith J O and Serra X PARSHL an analysis synthesis program for non
harmonic sounds based on a sinusoidal representation Proceedings of the Interna
tional Computer Music Conference Urbana Champaign Illinois USA pp
Smith J O and Van Duyne S A Commuted piano synthesis Proceedings
of the International Computer Music Conference Ban Canada pp
Stilson T and Smith J Alias free digital synthesis of classical analog wave
forms Proceedings of the International Computer Music Conference Hong Kong
pp
Strawn J Approximation and syntactic analysis of amplitude and frequency
functions for digital sound synthesis Computer Music Journal
Sullivan C S Extending the Karplus Strong algorithm to synthesize electric
guitar timbres with distortion and feedback Computer Music Journal
BIBLIOGRAPHY
Tolonen T Model based Analysis and Resynthesis of Acoustic Guitar Tones
Master s thesis Helsinki UniversityofTechnology Esp o o Finland p Rep ort
Lab oratory of Acoustics and Audio Signal Pro cessing
Tolonen T and V lim ki V Automated parameter extraction for plucked
string synthesis Proceedings of the Institute of Acoustics Vol pp
Presented at the International Symp osium on Musical Acoustics Edinburgh UK
Tomisawa N Tone pro duction metho d for an electronic musical instrument
U S Patent
for c m ratios in frequency mo dulation Truax B Organizational techniques
Computer Music Journal Reprinted in C Roads and J Strawn
eds Foundations of Computer Music Cambridge Massachusetts The MIT
Press pp
Van Duyne S A and Smith J O a The D digital waveguide Proceedings of
the IEEE Workshop of Applications of Signal Processing to Audio and Acoustics
New Paltz New York USA
Van Duyne S A and Smith J O b Physical mo deling with the D digital
Music Conference waveguide mesh Proceedings of the International Computer
pp
Van Duyne S A and Smith J O A simpli ed approach to mo deling dis
p ersion caused by sti ness in strings and plates Proceedings of the International
Computer Music Conference Aarhus Denmark pp
Van Duyne S A and Smith J O a Developments for the commuted pi
ano Proceedings of the International Computer Music Conference Ban Canada
pp
Van Duyne S A and Smith J O b The tetrahedral digital waveguide mesh
Proceedings of the IEEE Workshop of Applic ations of Signal Processing to Audio
and Acoustics New Paltz New York
Van Duyne S A and Smith J O The D tetrahedral digital waveguide
mesh with musical applications Proceedings of the International Computer Music
Conference International Computer Music Asso ciation Hong Kong pp
Vergez C and Ro det X Comparison of real trump et playing latex mo del
of lips and computer mo del Proceedings of the International Computer Music
Conference Thessaloniki Greece pp
Verma T S Levine S N and Meng T H Y Transient mo deling syn
thesis a exible analysis synthesis to ol for transient signals Proceedings of the
International Computer Music Conference Thessaloniki Greece pp
V lim ki V Discrete Time Modeling of Acoustic Tubes Using Fractional
Delay Filters PhD thesis Helsinki University of Technology Esp o o Finland
p
BIBLIOGRAPHY
V lim ki V and Takala T Virtual musical instruments natural sound using
physical mo dels Organised Sound
V lim ki V and Tolonen T a Development and calibration of a guitar syn
rd
thesizer Presented at the Convention of the Audio Engineering Society
Preprint New York USA
V lim ki V and Tolonen T b Multirate extensions for mo del based synthesis
of plucked string instruments Proceedings of the International Computer Music
Conference Thessaloniki Greece pp
V lim ki V Huopaniemi J Karjalainen M and J nosy Z Physical mo d
eling of plucked string instruments with application to real time sound synthesis
Journal of the Audio Engineering Society
V lim ki V Karjalainen M and Laakso T I Mo deing of wo o dwind bores
with nger holes Proceedings of the International Computer Music Conference
pp
V lim ki V Karjalainen M J nosy Z and Laine U K a A real time DSP
implementation of a ute mo del Proceedings of the IEEE International Confer
enceonAcoustics Speech and Signal Processing Vol San Francisco California
pp
V lim ki V Laakso T I and Mackenzie J Elimination of transients in
time varying allpass fractional delay lters with application to digital waveguide
Music Conference Ban mo deling Proceedings of the International Computer
Canada pp
V lim ki V Laakso T I Karjalainen M and Laine U K b A new compu
tational mo del for the clarinet in A Strange ed Proceedings of the Interna
tional Computer Music Conference International Computer Music Asso ciation
San Francisco CA