View metadata, citation and similar papers at core.ac.uk brought to you by CORE provided by CiteSeerX 1040 IEEE TRANSACTIONS ON AUDIO, SPEECH, AND LANGUAGE PROCESSING, VOL. 14, NO. 3, MAY 2006 Automated Classification of Piano–Guitar Notes Dimitrios Fragoulis, Member, IEEE, Constantin Papaodysseus, Mihalis Exarhos, George Roussopoulos, Thanasis Panagopoulos, and Dimitrios Kamarotos Abstract—In this paper, a new decisively important factor in tems. First attempts on computer identification included a very both the perceptual and the automated piano–guitar identification limited number of instruments and note ranges. Kaminsky and process is introduced. This factor is determined by the nontonal Materka [11] used features derived from the note’s energy enve- spectral content of a note, while it is, in practice, totally inde- pendent of the note spectrum tonal part. This conclusion and lope and a neural network classifier to discriminate guitar, piano, all related results are based on a number of extended acoustical marimba and accordion tones over an one-octave band. De Poli experiments, performed over the full pitch range of each instru- and Tonella [12] used a Self Organizing Map to classify sounds ment. The notes have been recorded from six different performers with a procedure similar to Grey’s. Cosi et al. [13] used fea- each of whom played a different instrument. Next, a number of tures based on an auditory model followed by a neural network powerful criteria for the classification between guitar and piano is proposed. Using these criteria, automated classification between to classify instruments. Most of the recent musical instrument 754 piano and guitar test notes has been achieved with a 100% identification systems have already shown a respectable level success rate. of performance. However, they haven’t demonstrated the ability Index Terms—Musical instrument classification, noontonal spec- of generalization i.e., the ability of the system to perform suc- trum, timbre identification, timbre recognition. cessful timbre identification among instrument recordings dif- ferent from those used during the training procedure. Martin [14], [15] presented a system that operates on single isolated I. INTRODUCTION notes played over the full pitch ranges of 15 orchestral instru- HE term timbre refers to the attributes of sound that play ments and uses a hierarchical classification framework. Recog- T an important role in the perception of speech and music. nition was performed using temporal features calculated from According to the American National Standards Institute, timbre the outputs of a log-lag correlogram. Brown [16] used cepstral is defined as “that attribute of auditory sensation in terms of coefficients calculated from oboe and saxophone samples and which a listener can judge that two sounds similarly presented managed to develop a hierarchical classifier. Herrera et al. [17] and having the same loudness and pitch are dissimilar” [1]. have presented a very informative review on the techniques that In many of the early studies on instrument timbre, perceptual have been so far proposed for automatic classification of mu- experiments have been performed to relate acoustic perception sical instruments. with several spectral and temporal characteristics of acoustic In this paper we have tackled the problem of piano and guitar signals. Clark et al. [2] found that timbre is associated with the timbre determination and classification. We have chosen to deal attack transient, using modulation during the steady state of a with these two instruments since their discrimination presents note. Strong and Clark [3], [4] interchanged spectral and tem- serious difficulties for the following reasons. poral envelopes of sounds produced by wind instruments and • The timbre of those two instruments is quite similar. In found that the results are instrument dependent. many instances even an experienced auditor cannot decide In addition, a considerable amount of research has been done whether the note he is listening to comes from a piano or in order to find the perceptual dimensions of musical instrument a guitar. timbre. Grey [5], Grey and Gordon [6], Grey and Moorer [7], • Piano and guitar have overlapping frequency ranges and used Multidimensional Scaling to put in evidence the main per- their sound production mechanism is based on string vi- ceptual dimensions of timbre. Similar results were presented by bration. Moreover, in many instances, both instruments’ Krumhansl [8] and McAdams [9]. Although many studies have notes show similar patterns of time decay. been presented emphasizing the importance of the note onset • A fully successful automated classification of piano and for instrument identification, Kendall [10] demonstrated that in guitar timbre has not, so far, been achieved, especially musical phrases, properties of the steady state are at least as im- when numerous notes coming from various instruments portant as transient properties. are considered. Systems presented so far have not demon- The results of the aforementioned studies have been applied strated the ability of generalization. One main reason for to the development of musical instrument identification sys- this may be the fact that these systems are based on several acoustical characteristics, which are not associated with Manuscript received May 20, 2003; revised December 21, 2004. The Asso- the kernel of the timbre. ciate Editor coordinating the review of this manuscript and approving it for pub- lication was Dr. Michael Davies. • The question where exactly the timbre lies for both piano The authors are with the School of Electrical and Computer Engineering, and guitar has not been answered so far. Division of Communications, Electronics and Information Systems, Na- In fact, a series of original experiments is presented that allow tional Technical University of Athens, GR-15773, Athens, Greece (e-mail: [email protected]). for the discrimination of piano and guitar timbre. Based on these Digital Object Identifier 10.1109/TSA.2005.857571 experiments, a very important factor in the timbre identification 1558-7916/$20.00 © 2006 IEEE FRAGOULIS et al.: AUTOMATED CLASSIFICATION OF PIANO–GUITAR NOTES 1041 process, both perceptual and automated, is introduced, namely typical guitar and piano envelopes, in both time and fre- the nontonal spectral content of a note. The term “nontonal” part quency domain. The resulted signals acoustically approx- of the spectrum is used to describe the frequency components imated naturally produced notes by guitar and piano, re- located between the main peaks. spectively, but not always at a satisfactory level and some- As it will be shown hereafter, this dimension can provide a times poorly. Similar results are obtained if one places the number of powerful criteria for the classification of musical in- harmonic series’ frequencies not at integral multiples of struments. The efficiency of these criteria is demonstrated in the the pitch frequency, but on the partials’ exact frequency case of piano–guitar discrimination, and a system for the auto- positions of the naturally produced note, whose envelope mated classification between piano and guitar is presented that is used. We have employed this alternative method espe- exhibits a success rate of 100%, independently of the choice of cially in the case of piano notes, since it is well known the training/test set. that their partials are not exactly harmonic. This experi- ment does not aim at synthesizing guitar or piano notes. II. PERFORMED EXPERIMENTS TO STUDY On the contrary, it demonstrates that the time envelope of PIANO AND GUITAR TIMBRE a piano or guitar note, alone, does not allow for a perfect A. An Experimental Material and Evaluation Group recognition of the identity of the instrument. 3) We have proceeded a bit further than experiment No2 by All related experiments have been performed on 1538 iso- performing the following: Firstly, we have reduced all lated notes sampled at 44 100 Hz over the full pitch range of notes to a common length, say 65 536 samples. Consider, each instrument. We remind that the fundamental frequencies next, a tonal peak of the DFT magnitude of an arbi- of piano notes range from 27 Hz to 4160 Hz, while those of trary note, located say at frequency . Moreover consider guitar notes range from 90 Hz to 990 Hz. From the gathered all spectral information located around in the interval note samples 612 were isolated guitar notes, while 926 were iso- , where is the greatest in- lated piano ones. The average note duration is around 1.8 sec. teger smaller than such that , All of them have been recorded from six different performers and is the smallest integer greater than such playing a different instrument each, i.e., six different pianos and that . We call all this spectral in- six different guitars. About 30% of the guitar notes were gen- formation “the lobe”. The constant a corresponds to a erated by plectrum while the rest by a human finger. Recorded suitably chosen small value, say . notes of half of the performers and half of the instruments have Subsequently, we have taken numerous pairs of notes been used as a training set, while those of the other performers of the same pitch, one of a guitar and the other of a piano. playing the remaining instruments as the test set. In this way, a For each such pair, we spotted the main peaks of the DFT training set has been obtained consisting of 483 piano and 301 magnitude and the corresponding lobes. Next, we guitar sample notes, as well as a test set consisting of 443 piano have created artificial signals by means of the following and 311 guitar sample notes. Notice that a number of piano notes procedure. have been produced by the performer so as to bear timbre char- For each peak of the two notes at the same or nearby acteristics, which resemble the guitar ones, and vice-versa. Fi- frequency, we have exchanged the corresponding lobes, nally, the acoustic experiments’ evaluators were five persons, namely in the guitar note DFT, we have replaced the guitar two professors of musicology and musicians as well, one pro- lobes with the piano lobes of the same or nearby fre- fessional musician and two amateur music lovers.