Vesa Va¨lima¨ki,* Mikael Laurson,† Commuted Waveguide and Cumhur Erkut* *Laboratory of Acoustics and Audio Synthesis of the Signal Processing Helsinki University of Technology Clavichord P.O. Box 3000 FIN-02015 HUT Espoo, Finland {Vesa.Valimaki, Cumhur.Erkut}@hut.fi †Centre for Music and Technology Sibelius Academy P.O. Box 86 Helsinki, Finland Downloaded from http://direct.mit.edu/comj/article-pdf/27/1/71/1853827/01489260360613353.pdf by guest on 26 September 2021 laurson@siba.fi

The clavichord is one of the oldest keyboard instru- instrument is an Anthony Sidey clavichord manu- ments, and it is still often used in performances factured by Heugel in Paris, France, in 1988. This and recordings of Renaissance and Baroque music. clavichord is an unfretted one, so any combination The sound of the instrument is pleasant and ex- of notes can be played. The range of a clavichord is pressive but quiet. Consequently, the instrument anywhere from three octaves to over five octaves. can only be used in intimate performances for Our clavichord has 51 keys ranging from C2 to D6. small audiences. This is the main reason why the The instrument was tuned about a whole tone clavichord was replaced by the harpsichord and fi- lower than the standard modern tuning Hz): the nominal frequency of A4 is 395 440 ס nally by the modern piano, both of which produce (A4 a considerably louder output. Attempts have been Hz. The Werkmeister tuning system was used. made to amplify the sound of the clavichord using For each key of the clavichord, a pair of strings is a piezoelectric pickup (Burhans 1973). tuned in unison, as sketched in Figure 2 (see, for One of our motivations in this research is to give example, Thwaites and Fletcher 1981; Campbell the clavichord a new life in the digital world, and Greated 1987). However, the two strings are al- where the faint sound level of the instrument can ways slightly detuned around the same note, be- be amplified by simply turning a volume knob. The cause exact tuning is impossible manually. Every suggested synthesis model is based on digital wave- key forms one end of a lever that has a tangent at- guide modeling of string instruments (Smith 1992, tached to its other end. When a key is depressed, 1998; Va¨lima¨ki et al. 1996; Karjalainen, Va¨lima¨ki, the tangent hits the string pair and initiates vibra- and Tolonen 1998) and uses the principle of com- tion. One end of the strings has been damped with muted waveguide synthesis where the soundbox’s felt, and the other end goes over a bridge to the response is incorporated in the excitation signal tuning mechanism. Thus, the strings are freely vi- (Smith 1993; Karjalainen and Va¨lima¨ki 1993; Karja- brating between the bridge and the tangent, which lainen, Va¨lima¨ki, and Ja´nosy 1993). Special sam- works as both a hammer and a termination. The pling techniques are also employed. Musical tangent mechanism is rather noisy, as it excites modes of the soundboard but also itself causes examples produced using the proposed synthesizer sound from its moving parts owing to friction. will be included on a forthcoming Computer Music When the key is released, the tangent falls back Journal CD. with the aid of gravity, and the string vibration is allowed to propagate to the felt-covered end of the Acoustics of the Clavichord string, which efficiently damps the vibration. At the end of each note, another knock is heard as the A photograph of the clavichord used for the mea- tangent returns to its resting position. The some- surements in this study is shown in Figure 1. The what mistuned strings of each pair are coupled via Computer Music Journal, 27:1, pp. 71–82, Spring 2003 a non-rigid bridge, and thus both beats and a two- ᭧ 2003 Massachusetts Institute of Technology. stage decay result (Weinreich 1977).

Va¨ lima¨ ki, Laurson, and Erkut 71 Figure 1. Clavichord used Figure 2. Tangent mecha- in this study. nism of one key of the clavichord and a string pair associated with it. Downloaded from http://direct.mit.edu/comj/article-pdf/27/1/71/1853827/01489260360613353.pdf by guest on 26 September 2021

The first thing that people usually notice about Felt the clavichord is that the sound level is very low. Tangent The maximum sound pressure level at 1 meter is only about 50 dB or 60 dB, depending on the indi- String vidual construction of the instrument. This makes pair this ancient instrument a sound source less effi- cient than a human speaker. There are many rea- sons for the weak output (see Campbell and Lever Bridge Key Greated 1987; Fletcher and Rossing 1991). The Tuning strings are thin and their tension is low, and they pins radiate sound inefficiently. The soundboard is small and light, and it cannot amplify the sound much. A particularly interesting feature in the clavi- chord is the mechanical aftertouch known as Be- bung. When the player increases pressure on a key, phonic aftertouch, which would really be needed, is the tangent is raised more, which in turn increases only available in high-end keyboard controllers. the tension of the string pair, resulting in a raise of As the key pressure is changed, the tangent pitch. This enables continuous control of vibrato, causes a slightly different change in the tension of which is a used performance style. Aftertouch has the two strings of a pair. This feature affects the been part of keyboard controllers since the 1980s, timbre of clavichord tones by introducing a flang- and it is thus easy to include this control in the ing effect. It is heard during and after the attack, clavichord synthesizer. However, a fully poly- particularly in the bass register.

72 Computer Music Journal Figure 3. (Top) Waveform tone at 1.1 sec, and (bot- Figure 4. Envelopes of the (dashed line), 4th (dash- of a clavichord tone (A3) tom) the time history of its lowest partials of the tone dot line), and 5th (dotted showing the irregular over- fundamental frequency shown in Figure 1: 1st line). all decay pattern and the (nominal fundamental fre- (solid line with circles), thump at the end of the quency is 197.5 Hz). 2nd (solid line), 3rd

1 Ϫ80

Ϫ100 0 Level Ϫ120 Magnitude (dB) Ϫ1 Downloaded from http://direct.mit.edu/comj/article-pdf/27/1/71/1853827/01489260360613353.pdf by guest on 26 September 2021 0 0.2 0.4 0.6 0.8 1 1.2 0 0.2 0.4 0.6 0.8 Time (sec)

200

199 lower than the threshold of audibility for this ef-

F0 (Hz) fect. According to Ja¨rvela¨inen and Va¨lima¨ki (2001), 198 the threshold at 196 Hz (the nearest frequency 197 tested) is 4.4 Hz. In the case of the clavichord used 0 0.2 0.4 0.6 0.8 1 1.2 Time (sec) in our measurements, during normal playing the pitch glide can be inaudible for most listeners. Au- dible pitch glides can be generated easily, however. Figure 5 gives an example of a tone that contains Properties of Single Clavichord Tones an exaggerated pitch vibrato, which should be obvi- ous for all listeners. Note that the vibrato width is Figure 3 shows the envelope of a recorded clavi- typically about 2 Hz or less, but can occasionally chord tone. The irregular, non-exponential decay of peak higher. The vibrato rate is about 7 Hz in this the tone can be observed. The key mechanism of case, which is slightly faster than what would be the clavichord generates a loud knock at the begin- usual in a musical context. The above fundamental ning and end of a tone, which is characteristic to frequency measurements reflect the effective pitch, the sound of the instrument. In Figure 3, a burst lo- where the contributions of both strings are present. cated at 1.1 sec corresponds to the thump caused Impulsively driven string instrument sounds are by release of the key, at the same time the signal always at least weakly inharmonic (Fletcher and starts to decay quickly. The envelope curves of the Rossing 1991). This is caused by the stiffness of the first five harmonics are presented in Figure 4. Note string material. It is well known that piano tones, that regular exponential decay (i.e., linear decay on particularly in the bass range, are highly inhar- a dB scale) is rare: pronounced beating and other ir- monic, which affects both the timbre and the tun- regularities are observed in many harmonics (e.g., ing. Measurements of inharmonicity have not been the beating of the fourth partial). published previously for the clavichord, so we con- The fundamental frequency of clavichord tones ducted them ourselves. This was needed to decide varies over time, as illustrated by an example in whether we had to implement the inharmonicity the lower part of Figure 3. This may be partly using an allpass filter, as is customary to do in caused by tension modulation (Tolonen, Va¨lima¨ki, waveguide synthesis (Jaffe and Smith 1983; Bank and Karjalainen 2000), which is a purely physical 2000). We used our parameter estimation software phenomenon in vibrating strings, but also by the to extract partial frequencies of clavichord tones as pressure of the player’s finger (i.e., the mechanical a function of time. Using a least-squares fit, we es- aftertouch), which directly controls string tension timated from the data the inharmonicity coeffi- during playing. A time-domain finite-difference cient. We found that the clavichord used in our simulation suggests that the tension modulation ef- measurements produces almost perfectly harmonic fect is negligible compared to that caused by the tones. (The inharmonicity coefficient does not ex- mechanical aftertouch (Va¨lima¨ki et al. 2000). In ceed 10–6.) Thus, the effect of inharmonicity can be Figure 3, the pitch glide is about 2 Hz, which is considered inaudible (Ja¨rvela¨inen, Va¨lima¨ki, and

Va¨ lima¨ ki, Laurson, and Erkut 73 Figure 5. (Top) Waveform Figure 6. Magnitude spec- bridge (bottom). Results of of clavichord tone A3 with trum of the soundboard two measurements (solid strong vibrato (mechanical excited with an impulse and dashed line) are plot- aftertouch) and (bottom) hammer at the high- ted one on the other for its fundamental frequency frequency (top) and at the both cases. as a function of time. low-frequency end of the

1 60

0 40 Level Downloaded from http://direct.mit.edu/comj/article-pdf/27/1/71/1853827/01489260360613353.pdf by guest on 26 September 2021 Magnitude (dB) 20 Ϫ1 0 0.5 1 1.5 2 2.5 0 100 200 300 400 500

202 60

200 40

F0 (Hz) 198

196 Magnitude (dB) 20 0 0.5 1 1.5 2 2.5 Time (sec) 0 100 200 300 400 500 Frequency (Hz)

Karjalainen 2001), and perfectly harmonic synthesis Synthesis Model for the Clavichord is adequate. Next, we describe the synthesis model for clavi- chord tones. The commuted waveguide synthesis Soundbox method invented by Smith (1993) and Karjalainen An essential part of clavichord tones is the re- and colleagues (Karjalainen and Va¨lima¨ki 1993; sponse of the soundbox. The box is strongly excited Karjalainen, Va¨lima¨ki, and Ja´nosy 1993) is applied every time a key is pressed. This reverberant re- by using inverse-filtered clavichord signals as exci- sponse will ring for about five seconds, and it tation for the synthesis model. Inverse filtering stands out particularly in staccato notes, where the here refers to the processing of a signal with the in- string vibration itself is damped quickly after the verted transfer function of a waveguide string attack. model. In this case, the inverse filtering essentially We have conducted a series of acoustic measure- cancels the partials of a recorded tone. The com- ments on the soundbox of the instrument to iden- muted waveguide synthesis method has been for- tify the most prominent modes. Figure 6 shows the merly used successfully for the synthesis of the magnitude spectrum of the soundboard that was acoustic guitar (Karjalainen, Va¨lima¨ki, and Ja´nosy hit with an impulse hammer when the strings were 1993; Va¨lima¨ki et al. 1996; Laurson et al. 2001) and carefully damped with soft material. Results of two the piano (Smith and Van Duyne 1995). separate measurements are plotted on top of each Figure 7 shows the structure of the synthesis other. The differences between these curves are model developed in this work. The coupling of the quite small, verifying the repeatability of the mea- two basic strings models Sl(z) and S2(z) is realized surement. Many narrow peaks are visible in Figure with an unconditionally stable technique suggested 6. They correspond to long ringing resonance by Karjalainen et al. (1998): the output of only one modes. It can also be seen that the excitation of of the string models is fed to the input of the other, modes depends on the location of excitation. For and hence there is no feedback. In practice, the example, the 90 Hz mode is much stronger in the coupling coefficient gc is selected to have a small top part of the figure than in the lower part. A value, but this is not required, because there can be similar phenomenon occurs in the case of the mode no stability problems. The input and output signals at about 340 Hz. of the string models are scaled by constant multi-

74 Computer Music Journal Figure 7. Clavichord syn- Figure 8. Block diagram of ling the sharpness of thesis algorithm for one (a) the standard and (b) attack using parameter key consists of three data- modified string algorithm. gattack. bases of samples and two The latter allows control- basic string models that are coupled.

Trigger at release time g End knocks Trigger at attack time sb Soundbox responses Downloaded from http://direct.mit.edu/comj/article-pdf/27/1/71/1853827/01489260360613353.pdf by guest on 26 September 2021 grelease

gi1 go1 Timbre S Sound Excitation control 1 signals gc g gi2 o2

S2

plying coefficients gi and go, respectively. In addi- tion, two sample databases are needed for realistic simulation of the soundbox response and the per- ϪL F(z) z H l (z) cussive noise caused by key release. In the follow- ing, we discuss the string algorithm, model parameters and their control, and the sample data- (a) bases. gattack

Modified String Algorithm Ϫ L F(z) z H l (z) The strings of the clavichord can be simulated us- ing digital waveguide string models. In principle, two digital waveguide models should be used for (b) each string, since they have two polarizations of vi- bration (i.e., horizontal and vertical with respect to H (z) is a loop filter with the following transfer func- the soundboard), but at present we only use one l tion (Va¨lima¨kiet al. 1996): digital waveguide per string. For a pair of clavi- aםchord strings, we thus have two digital waveguide 1 (g loop (2 ס (string models, not four. This choice results in an H (z 1מazם1loop1 efficient algorithm and still generates some beating loop in the synthetic tones. Ͻ Ͻ Ͻ Ͻ with 0 gloop 1 and –1 aloop 0. The values of Usually, a digital waveguide string model has the these parameters can be estimated for each string following transfer function (Jaffe and Smith 1983; using previously developed methods (Va¨lima¨kiet Va¨lima¨ki et al. 1996), as also illustrated in Figure al. 1996; Erkut et al. 2000). 8(a), While testing an early version of the clavichord synthesizer, we noticed that the contribution of the 1 ס tangent knock was not sufficiently prominent. In (1) מ (S(z F(z)H (z)z Lמ1 1 particular, the attack of high notes sounded too 3– ם 2– ם 1– ם ס where F(z) h0 h1z h2z h3z is a third- soft. To fix this deficiency, we modified the above order Lagrange interpolation filter (Laakso et al. string algorithm to allow the attack sharpness to be 1996), L is the (integer) length of the delay line, and controlled with one parameter. The block diagram

Va¨ lima¨ ki, Laurson, and Erkut 75 Figure 9. Estimated values dashed line) together with of string model parameters their straight-line approxi- gloop (top) and aloop (bottom) mations (solid lines). (dots connected with

1 of the acoustic guitar. Details can be found else- where (Va¨lima¨ki et al. 1996; Erkut et al. 2000). 0.995 Figure 9 shows the gloop and aloop parameters for 1oop

g all 51 keys of the clavichord. These values were es- 0.99 timated from recorded clavichord tones. The sec-

0.985 ond string model uses the same parameters but Downloaded from http://direct.mit.edu/comj/article-pdf/27/1/71/1853827/01489260360613353.pdf by guest on 26 September 2021 0 10 20 30 40 50 slightly different delay lines lengths for mistuning. 0 The fundamental frequencies of the lowest (#1) and highest (#51) key are 58.7 Hz and 1056 Hz, respec- tively. Note that the loop gain parameters g have Ϫ loop 0.2 a nearly linear trend as a function of key index (i.e., 1oop a logarithm of frequency) with some variations and a Ϫ0.4 few outliers. Ja¨rvela¨inen and Tolonen (2001) have investigated the tolerance of parameters value vari- 0 10 20 30 40 50 40%ם Key index ation. Their results show that –30% to changes in the time constant (which depends on

gloop) are inaudible. Also, in the aloop parameter, vari- -cannot be per 16%ם of the new algorithm is shown in Figure 8b. Its ations between –17% and transfer function can be written as ceived. It appears that the gloop parameters presented in Figure 9 can be replaced with values LמF (z) H(z) z :l obtained by linear regression ם g ס (S(z Lמ מ attack ם F (z) Hl (z) z (3) Ϸ 1 gloop 0.989 0.000189k (4) Lמg ) F (z) H (z) zמ1)ם g is the key number starting from 51. . .1 ס attack attack l where k ס LמF (z) H (z) z מ1 This .(51 ס and continuing up to D6 (k ,(1 ס l C2 (k ס Note that with gattack 1, this algorithm is identi- straight-line fit is illustrated in Figure 9 (top) with a cal to the standard plucked-string algorithm of solid line.

Equation 1. A value larger than unity, such as The values of parameter aloop may be approxi- ס gattack 4, may be used to imitate the aggressive at- mated by a low-order polynomial as well. However, tack of acoustic tones. In the current version, the a linear regression to the data shown in the lower value of the attack sharpness parameter increases part of Figure 9 yields intolerably large errors. To linearly with key number, so that for C2 it is 2.0 reduce the error, we attempt a piecewise linear re- and for D6 it is 4.0. The linear trend and the actual gression with a knee point between key indices 23 values were adjusted by informal listening. There is and 24: 0.0176k, for k Յ 23 ם 0.514מ -no physical motivation for this new model struc a Ϸ Ά (5) 0.00225k, for k Ն 24 ם 0.175מ ture, other than that it enables separate control of loop two different components of the tone: the tangent knock and the vibrating string sound. This two-part straight-line fit is shown in Figure 9 (bottom) with solid lines, one for the low end of the keyboard and another for the high end. This ap-

proximation for the aloop values introduces minor String Model Parameters changes in timbre that can be perceived, but it does not necessarily render the sound quality worse. In Parameter values for the synthesizer are obtained fact, the overall tone color behaves more uniformly from analysis of recorded signals. We use a method with the linear approximations than with the un- based on the short-time Fourier transform and si- processed parameter data. nusoidal modeling of the clavichord tones. These The damping of the string vibrations as a result methods have proven successful earlier in the case of releasing a key is simulated by changing the loop

76 Computer Music Journal filter characteristics (e.g., by setting loop gain gloop metals, that is, the tip of the hammer and a tuning equal to 0.1 momentarily). A similar method has pin. This is easily done with a fast fade-in. Note been used successfully to simulate damping of a that this editing is not extremely critical, because guitar string by finger (Erkut et al. 2000; Laurson et the impact noise from the tangent mechanism, al. 2001). which is included in the excitation signal, domi-

nates during the attack and masks the beginning of Downloaded from http://direct.mit.edu/comj/article-pdf/27/1/71/1853827/01489260360613353.pdf by guest on 26 September 2021 the soundbox response. For each key, a soundbox Sample Databases sample should be chosen, which is obtained close to the point where the strings are attached. How- Excitation signals for the synthesis have been ob- ever, it is practically impossible to excite the bridge tained by processing anechoic recordings of clavi- of the clavichord between the tuning pins, and the chord tones using the methods described by Erkut only clean samples that we could obtain are from et al. (2000). The procedure consists of sinusoidal both ends of the bridge (see Figure 6). This leaves analysis, subtracting partials, equalizing the resid- us with possibilities to use one of the samples for ual, and truncating the resulting signals with the the left half of the keyboard and the other for the right half of a Hanning window. One such excita- right one, or to interpolate between the high- and tion signal is used for each of the 51 keys. Cur- low-end samples along the keyboard. Although the rently, all excitation signals are 20,000 samples sampling-based reverberation modeling is clearly long (about 0.45 sec at a 44.1 kHz sampling rate). simplified, it yields a natural-sounding ambience in It would be possible to reduce their length further, the synthetic output and is very cheap to imple- but we have found it unnecessary in our current ment because it requires no filtering. implementation, since there is no lack of memory. The knock database for note endings consists of a The reverberation caused by the soundbox is in- few short samples (duration less than 0.5 sec). corporated by triggering a soundbox response sam- These samples may be obtained from recorded long ple at a low level each time any note is played (see tones, where the sound of string vibration is al- Figure 7). This sample must be long enough (about lowed to decay below the threshold of audibility five seconds) so that it provides the reverberant before the key is released. Now the isolated thump character of the instrument. This is particularly can be simply edited from the recording. One prac- important for short notes, such as staccato playing, tical problem was caused by the poor signal-to- for which the output signal would otherwise stop noise ratio of the release samples. It was possible to suddenly in an unnatural manner. Note that in hear when a key release occurred from the in- principle we could instead use very long excitation creased hiss during synthesis. This led us to apply a signals for each note, in which case the full sound- de-noising algorithm to suppress the background box response would automatically sound with each noise from the key release recordings, which solved note. However, we prefer using short excitation sig- the problem. nals, because we can avoid certain problems related It is unnecessary to record a release sample for to the inverse filtering (e.g., ringing and beating due each key. Because the tangent mechanism for every to imperfectly cancelled partials). This also saves key is identical and the resulting sound is similar, memory, because we use fewer soundbox samples it is plausible to use a single sample for all syn- than excitation signals. Using a separate soundbox thetic notes. To avoid a boring similarity, as a com- sample also allows adjusting the relative levels of promise a small collection of release samples may soundbox reverberation and string sound, which is be used. We have observed that the largest varia- an advantage. tion among different tangent releases is caused by The soundbox samples can be obtained by hitting the velocity of release, which slightly changes the the bridge of the clavichord with an impulse ham- rhythm of different noises within the knock. Thus, mer at various points. The response must be edited one choice is to gather a collection of tangent re- to remove the sharp attack caused by contact of leases for different release velocities, and select one

Va¨ lima¨ ki, Laurson, and Erkut 77 Figure 10. Signals involved sample, (d) end noise sam- in the synthesis of a single ple (triggered at about 1 clavichord tone: (a) excita- sec), and (e) synthetic tion signal, (b) string algo- tone, which is sum of sig- rithm output, (c) soundbox nals (b) to (d).

0.5 (a) that this is a simpler way of producing a time- 0 varying pitch than the tension-modulation tech- Ϫ0.5 nique Tolonen et al. (2000) used previously for the 0.5 (b) synthesis of the kantele (Va¨lima¨ki et al. 1999; Er- 0 kut et al. 2002) and the tanbur (Erkut and Va¨lima¨ki Ϫ0.5 2000). The tension-modulation algorithm would re- Downloaded from http://direct.mit.edu/comj/article-pdf/27/1/71/1853827/01489260360613353.pdf by guest on 26 September 2021 0.1 (c) quire computing the elongation of the string in real 0 time and controlling the delay-line length of the Ϫ0.1 string model according to it. 0.1 (d) For the mechanical aftertouch, a more sophisti- 0 cated control signal is required. The pitch contour Ϫ0.1 of a note is generated with a help of a pitch scaler 0.5 (e) in the same fashion as in Laurson et al. (2001). If 0 there is neither an initial pitch glide nor vibrato, Ϫ0.5 0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 the scaler value is 1. The initial pitch glide is im- Time (sec) plemented as a ramp starting from a value larger than 1 and ending at 1 after around 0.1 sec. (The initial value depends on the dynamic level of the randomly or based on some rule, such as the play- current note so that forte playing yields a higher ing tempo. We are currently using two release sam- value than piano playing.) The vibrato control, in ples, and we slightly randomize the release times of turn, is realized with separate parameters for rate different keys. More sophisticated but still fairly and maximum depth, and a temporal envelope for simple synthesis models employing physically in- depth (for more details, see Laurson et al. 2001). formed randomization have been proposed by Cook A specific feature of the clavichord synthesizer is (1997). that the pitch scaler affects a detuning parameter of the two string models dynamically. Thus, a more drastic pitch drift results in a tone that is more out Delay Line Variations and the Flanging Effect of tune. This mimics the imperfect mechanism of the tangent that pulls the two strings unequally. A Although the variation of the fundamental fre- database of detuning parameters was generated by quency in clavichord sounds can be inaudible as trial and error. An alternative way would be to discussed earlier, it is also easy to cause pitch measure detuning parameters from the original re- changes with mechanical aftertouch. Because any cordings, which turns out to be difficult. High- change in the string tension will be different for the pitched tones tend to be more detuned than low two strings of a pair, a flanging-like effect can ap- ones. The sound resulting from the time-varying pear also in tones where the pitch change is negli- detuning is suggestive of the flanging effect, which gible. Variation of the fundamental frequency can brings warmth and variation to the synthesis. be simulated in the following way: a decaying con- trol function, for example, a scaled impulse re- sponse of a leaky integrator (a one-pole filter), is Synthesis Example subtracted from the delay-line length of both string models. This imitates the change of pressure on the Figure 10 presents the components for the synthe- key by the player’s finger right after depressing the sis of a single clavichord tone. The excitation sig- key (or the decrease of tension modulation depth nal shown in Figure 10a is inserted into the two while the string vibration begins to decay), and string algorithms. The name ‘‘commuted synthe- brings about the progression of the fundamental sis’’ comes from the principle that this excitation frequency similar to that shown in Figure 3. Note signal includes both the contribution of the excita-

78 Computer Music Journal tion mechanism (the tangent) and the soundbox re- Exaggerated detuning of the two string models for sponse. However, as can be seen, in our each voice leads to a low-quality—or possibly a implementation the input signal contains only the very old—clavichord. As a consequence of the very beginning of the soundbox response. The tail above parametric modifications, the synthetic of the soundbox response is produced separately us- tones will still be reminiscent of the clavichord. ing the sampling-based method. Extending the range of an instrument is fascinat- Downloaded from http://direct.mit.edu/comj/article-pdf/27/1/71/1853827/01489260360613353.pdf by guest on 26 September 2021 The summed output signals of the string models ing and useful in the case of the acoustic guitar. constitute the basic vibrating string component of Design of a synthetic ‘‘super guitar’’ that has a the synthetic tone, which is displayed in Figure wide range, almost like that of a grand piano, has 10b. The key release occurs at 0.97 sec; note how been documented recently (Laurson, Va¨lima¨ki, and the string tone in Figure 10b decays quickly after Erkut 2002). While an arbitrary change of pitch is that. Figure 10c shows the beginning of a five- easy in a waveguide synthesizer (simply vary the second-long soundbox sample, which has been length of a delay line), the loop filter parameters edited from the impulse response of the low- and excitation signals should be extracted from re- frequency end of the bridge. A tangent release sam- cordings, but for non-existing fundamental frequen- ple, which is started at the release time, is given in cies, they are unavailable. The extension of the Figure 10d. Finally, Figure 10e presents the sum of pitch range thus involves extrapolation problems the signals shown in Figures 10b, 10c, and 10d, and that must be solved somehow. We are planning it is the output of the clavichord synthesizer. tests on extending the range of the proposed clavi- Although remaining almost invisible in Figure chord synthesizer. 10e, the soundbox sample rings several seconds af- Naturally, it is also possible to turn the clavi- ter the string tone and the key release sample have chord synthesizer into a previously unheard virtual died out. The soundbox sample will also be clearly instrument. For instance, modifying or replacing audible only after the string tone has been ringing the signals in the sample databases allows dramatic and decaying for some time, and particularly after variations. Exploring all such transformations was the key has been released. We may note that iden- not included in the goals of this study, and the po- tity resynthesis is impossible using the proposed tential of these modifications thus remains mostly synthesis model and parameter estimation meth- unknown. ods. Nevertheless, the obtained similarity is con- sidered to be sufficient for high-quality synthesis. This is confirmed by musical examples produced Software Implementation of the Synthesizer with the synthesizer. The real-time implementation of the proposed clav- ichord synthesizer has been realized using a Musically Interesting Modifications PatchWork (Laurson 1996) user-library called PWSynth (Laurson and Kuuskankare 2001). The The proposed synthesizer structure allows the user high-level part of PWSynth is based on Lisp and to modify the timbre in numerous ways. Many of CLOS, but the low-level and time-critical DSP- the possible variations have a clear intuitive inter- routines have been implemented in C. The system pretation. For example, varying the gain gloop of the can be played either from musical notation and loop filter affects the decay time, but otherwise control software called Expressive Notation Pack- the timbre remains unchanged. However, varying age (ENP, described in Kuuskankare and Laurson the loop filter coefficient aloop changes the decay 2001; Laurson et al. 2001), or from a MIDI key- time at different frequencies, resulting in a curious board. evolution of the spectrum over time. The attack The starting point for the clavichord implemen- sharpness parameter and the gain of the note-off tation was to use a guitar synthesizer implemented noise also enable meaningful and useful controls. earlier (Laurson et al. 2001). These systems have

Va¨ lima¨ ki, Laurson, and Erkut 79 Figure 11. ENP screenshot from the final cadence of Par le regard de vos beaux yeux by Guillaume Dufay. Downloaded from http://direct.mit.edu/comj/article-pdf/27/1/71/1853827/01489260360613353.pdf by guest on 26 September 2021

several features in common. Both use basically the Figure 11 shows the three last measures of a same dual-polarization string model, although in piece by Guillaume Dufay (1400–1474). Besides the case of the clavichord, it models a pair of pitch and rhythm information, the piece includes strings and not the orthogonal polarizations of a several expression markings that control both vi- single string. The database of samples—needed for brato (see the labels starting with ‘‘vb’’) and articu- the excitation signals and the noise effects in the lation (see labels starting with ‘‘st,’’ which stands clavichord simulation—could be used in both cases for staccato). The numbers after the expression in a similar way. markings denote how much vibrato or staccato The main difference between the implementa- should be applied to a given note. This example has tions is the fact that the clavichord consists of 51 also a special expression containing a tempo func- strings, while the classical guitar has only six. To tion (see the function above the staff marked with create a synthesizer that could be used in real time, ‘‘ritardando’’ in Figure 11), which controls the we designed a voice-allocation algorithm that al- tempo of the piece. lows us to effectively play typical pieces of the Our current MIDI keyboard implementation is clavichord repertoire. We found that a synthesizer fairly rudimentary and serves only as a basic tool for six simultaneous strings could be played in real for testing. The control parameters of the MIDI time on a Macintosh G3 400 MHz portable com- keyboard include key number, key velocity, attack puter. This is sufficient for a large percentage of old time, and end time of notes. Furthermore, we use keyboard music. New, faster computers will allow the MIDI aftertouch parameter to simulate the fin- playing more voices simultaneously. ger vibrato effect of the clavichord. A more usable The control information for the clavichord syn- MIDI implementation would require a keyboard thesizer is calculated from an ENP input score. The with polyphonic aftertouch. Also, we found that user enters first the basic musical information (i.e., the MIDI keyboard at our disposal was not respon- pitches and rhythms) of the piece using the graphi- sive enough when simulating the finger vibrato. A cal front-end of ENP. After this, the user typically playable keyboard simulation of the clavichord adds expressions and tempo functions that allow model requires more sensitive hardware than what fine tuning of the timing information in the score. is commonly available today. Also, one can include expressions specific to the clavichord, such as the amount of vibrato to be ap- plied to a given note. Furthermore, a special rule Conclusions and Future Plans modifies the end time of notes (i.e., the time when the player lifts the fingers from the keyboard). This A simplified physical model was proposed for the was done to make the performance more lively. synthesis of the clavichord. It follows the principle The characteristic noise burst of the instrument of commuted waveguide synthesis, but also uses when the fingers are released was found to be es- many samples obtained by editing and processing sential for the realism of the synthesis. recordings. The excitation signal can be obtained

80 Computer Music Journal by inverse filtering and truncating a recorded clavi- ments, Virtual Musical Instruments and their Con- chord tone. Two coupled digital waveguide string trol,’’ and the work of C. Erkut is part of the project models are used for synthesizing each voice. A mi- ‘‘Technology for Speech and Audio Processing.’’ nor modification to the plucked string model was Both projects are funded by the Academy of Fin- introduced to enable control over attack sharpness, land. The authors are grateful to Dr. Tero Tolonen which was found necessary during testing of the for his contributions in the initial phases of this Downloaded from http://direct.mit.edu/comj/article-pdf/27/1/71/1853827/01489260360613353.pdf by guest on 26 September 2021 synthesis algorithm. The reverberation caused by study, to Mr. Paulo A. A. Esquef for his help in the the soundbox that has many resonant modes was de-noising of the sample databases, and to Mr. implemented in a computationally economic way Jonte Knif for his help in the production of musical by triggering a soundbox response sample with examples using the clavichord synthesizer. Special each note. The ending of a note requires another thanks to Professor Matti Karjalainen who provided sample to be played, because there must be a digital photographs of the clavichord and our mea- ‘‘knocking’’ sound, which is characteristic to the surement session. clavichord. The mechanical aftertouch of the clavi- chord can be implemented by varying the delay line lengths of the waveguide string models over time. When these changes are different for the two References string algorithms modeling a pair of strings, a flang- ing effect is generated, which is also heard in the Bank, B. 2000. Physics-Based Sound Synthesis of the Pi- real clavichord. ano. Report no. 54, Helsinki University of Technology, In the future, we plan to develop another version Laboratory of Acoustics and Audio Signal Processing, of the clavichord synthesizer using the direct Espoo, Finland. Available on-line at physical modeling approach, where the excitation, www.acoustics.hut.fi/publications/. strings, and the soundbox have a counterpart in the Burhans, R. W. 1973. ‘‘Clavichord Amplification.’’ Jour- model. To accomplish this, the sampling-based im- nal of the Audio Engineering Society 21(6):460–463. plementation of the reverberant response of the Campbell, M., and C. Greated. 1987. The Musician’s soundboard must be replaced with a soundbox Guide to Acoustics. New York: Schirmer Books, model that filters the output of the string models. pp. 234–236. The soundbox can be considered a small reverber- Cook, P. R. 1997. ‘‘Physically Informed Sonic Modeling ant room and be simulated with an artificial rever- (PhISM): Synthesis of Percussive Sounds.’’ Computer beration algorithm (e.g., Gardner 1998; Bank 2000). Music Journal 21(3):38–49. Erkut, C., et al. 2000. ‘‘Extraction of Physical and Expres- In addition, further measurements of the real in- sive Parameters for Model-Based Sound Synthesis of strument should be taken. Particularly, the acceler- the Classical Guitar.’’ Proceedings of the AES 108th ation of the tangent needs to be registered during Convention. New York: Audio Engineering Society. the attack of the tone, because this information is Erkut, C., and V. Va¨lima¨ki. 2000. ‘‘Model-Based Sound needed for an algorithm simulating the tangent ac- Synthesis of Tanbur, a Turkish Long-Necked Lute.’’ tion, similar to piano hammer simulations (Bank Proceedings of the IEEE International Conference on 2000). This might also reveal the importance of the Acoustics, Speech, and Signal Processing, Vol. 2. Pis- tension modulation effect in the sound production cataway, New Jersey: Institute of Electrical and Elec- of the clavichord. The tangent modeling algo- tronics Engineers, pp. 769–772. rithm—which will be different from physical mod- Erkut, C., et al. 2002. ‘‘Acoustical Analysis and Model- els available for the piano hammer, as Hall (1993) Based Sound Synthesis of the Kantele.’’ Journal of the has pointed out—should be developed to be able to Acoustical Society of America 112(4):1681–1691. control the excitation using physical parameters. Fletcher, N. H., and T. D. Rossing. 1991. The Physics of Musical Instruments. New York: Springer-Verlag. Acknowledgments Gardner, W. G. 1998. ‘‘Reverberation Algorithms.’’ In M. Kahrs and K. Brandenburg, eds. Applications of Digital The work of M. Laurson is part of the project Signal Processing to Audio and Acoustics. Dordrecht, ‘‘Sounding Score—Modeling of Musical Instru- The Netherlands: Kluwer, pp. 85–131.

Va¨ lima¨ ki, Laurson, and Erkut 81 Hall, D. E. 1993. ‘‘String Excitation: Piano, Harpsichord Lisp-based Bridge between Computer Assisted Compo- and Clavichord.’’ Proceedings of the 1993 Stockholm sition and Model-based Synthesis.’’ Proceedings of the Music Acoustics Conference. Stockholm, Sweden: 2001 International Computer Music Conference. San Royal Swedish Academy of Music, pp. 309–314. Francisco: International Computer Music Association, Jaffe, D. A., and J. O. Smith. 1983. ‘‘Extensions of the pp. 127–130. Karplus–Strong Plucked-String Algorithm.’’ Computer Laurson, M., V. Va¨lima¨ki, and C. Erkut. 2002. ‘‘Produc- Downloaded from http://direct.mit.edu/comj/article-pdf/27/1/71/1853827/01489260360613353.pdf by guest on 26 September 2021 Music Journal 7(2):76–87. tion of Virtual Acoustic Guitar Music.’’ Proceedings of Ja¨rvela¨inen, H., and T. Tolonen. 2001. ‘‘Perceptual Toler- the AES 22nd International Conference on Virtual, ances for Decay Parameters in Plucked String Synthe- Synthetic, and Entertainment Audio, New York: Au- sis.’’ Journal of the Audio Engineering Society 49(11): dio Engineering Society, pp. 249–255. 1049–1059. Smith, J. O. 1992. ‘‘Physical Modeling Using Digital Ja¨rvela¨inen, H., V. Va¨lima¨ki, and M. Karjalainen. 2001. Waveguides.’’ Computer Music Journal 16(4): 74–91. ‘‘Audibility of the Timbral Effects of Inharmonicity in Available on-line at www-ccrma.stanford.edu/ϳjos/ Stringed Instrument Tones.’’ Acoustics Research Let- pmudw/. ters Online 2(3):79–84. Smith, J. O. 1993. ‘‘Efficient Synthesis of Stringed Musi- Ja¨rvela¨inen, H., and V. Va¨lima¨ki. 2001. ‘‘Audibility of Ini- cal Instruments.’’ Proceedings of the 1993 Interna- tial Pitch Glides in String Instrument Sounds.’’ Pro- tional Computer Music Conference. San Francisco: ceedings of the 2001 International Computer Music International Computer Music Association, pp. 64–71. Conference. San Francisco: International Computer Available on-line at www-ccrma.stanford.edu/ϳjos/cs/. Music Association, pp. 282–285. Smith, J. O. 1998. ‘‘Principles of Digital Waveguide Mod- Karjalainen, M., and V. Va¨lima¨ki. 1993. ‘‘Model-Based els of Musical Instruments.’’ In M. Kahrs and K. Bran- Analysis/Synthesis of the Acoustic Guitar.’’ Proceed- denburg, eds. Applications of Digital Signal Processing ings of the 1993 Stockholm Music Acoustics Confer- to Audio and Acoustics. Dordrecht, The Netherlands: ence. Stockholm, Sweden: Royal Swedish Academy of Kluwer, pp. 417–466. Music, pp. 443–447. Smith, J. O., and S. A. Van Duyne. 1995. ‘‘Commuted Pi- Karjalainen, M., V. Va¨lima¨ki, and Z. Ja´nosy. 1993. ‘‘To- ano Synthesis.’’ Proceedings of the 1995 International wards High-Quality Sound Synthesis of the Guitar and Computer Music Conference, San Francisco: Interna- String Instruments.’’ Proceedings of the 1993 Interna- tional Computer Music Association, pp. 319–326. tional Computer Music Conference. San Francisco: In- Thwaites, S., and N. H. Fletcher. 1981. ‘‘Some Notes on ternational Computer Music Association, pp. 56–63. the Clavichord.’’ Journal of the Acoustical Society of Karjalainen, M., V. Va¨lima¨ki, and T. Tolonen. 1998. America 69(5):1476–1483. ‘‘Plucked-String Models: From the Karplus–Strong Al- Tolonen, T., V. Va¨lima¨ki, and M. Karjalainen. 2000. gorithm to Digital Waveguides and Beyond.’’ Com- ‘‘Modeling of Tension Modulation Nonlinearity in puter Music Journal, 22(3):17–32. Plucked Strings.’’ IEEE Transactions on Speech and Kuuskankare, M., and M. Laurson. 2001. ‘‘ENP: Musical Audio Processing 8(3):300–310. Notation Library based on Common Lip and CLOS.’’ Va¨lima¨ki, V., et al. 1996. ‘‘Physical Modeling of Plucked Proceedings of the 2001 International Computer Mu- String Instruments with Application to Real-Time sic Conference. San Francisco: International Computer Sound Synthesis.’’ Journal of the Audio Engineering Music Association, pp. 131–134. Society 44(5):331–353. Laakso, T. I., et al. 1996. ‘‘Splitting the Unit Delay— Va¨lima¨ki, V., et al. 1999. ‘‘Nonlinear Modeling and Syn- Tools for Fractional Delay Filter Design.’’ IEEE Signal thesis of the Kantele—a Traditional Finnish String In- Processing Magazine 13(1):30–60. strument.’’ Proceedings of the 1999 International Laurson, M. 1996. PATCHWORK: A Visual Program- Computer Music Conference. San Francisco: Interna- ming Language and Some Musical Applications. Doc- tional Computer Music Association, pp. 220–223. toral dissertation, Sibelius Academy, Helsinki, Va¨lima¨ki, V., et al. 2000. ‘‘Model-Based Synthesis of the Finland. Clavichord.’’ Proceedings of the 2000 International Laurson, M., et al. 2001. ‘‘Methods for Modeling Realis- Computer Music Conference. San Francisco: Interna- tic Playing in Acoustic Guitar Synthesis.’’ Computer tional Computer Music Association, pp. 50–53. Music Journal 25(3):38–49. Weinreich, G. 1977. ‘‘Coupled Piano Strings.’’ Journal of Laurson, M., and M. Kuuskankare. 2001. ‘‘PWSynth: A the Acoustical Society of America 62(6):1474–1484.

82 Computer Music Journal