Percussion Classification in Polyphonic Audio Recordings Using Localized Sound Models
Total Page：16
File Type：pdf, Size：1020Kb
PERCUSSION CLASSIFICATION IN POLYPHONIC AUDIO RECORDINGS USING LOCALIZED SOUND MODELS Vegard Sandvold Fabien Gouyon Perfecto Herrera University of Oslo Universitat Pompeu Fabra Universitat Pompeu Fabra Oslo, Norway Barcelona, Spain Barcelona, Spain ABSTRACT as e.g. voice, piano, guitars etc. Even worse, simultaneous occurrences of several classes of percussion instruments This paper deals with automatic percussion classification (e.g. kick + hihat or snare + hihat) may be encountered. in polyphonic audio recordings, focusing on kick, snare To deal with this issue, existing literature advocates di- and cymbal sounds. We present a feature-based sound verse research directions. Some advocate source separa- modeling approach that combines general, prior knowl- tion techniques as Independent Subspace Analysis [1, 2] edge about the sound characteristics of percussion instru- or signal models as ‘Sinusoidal + Residual’ (assuming that ment families (general models) with on-the-fly acquired drums are in the residual component) [7, 11]. Noise re- knowledge of recording-specific sounds (localized mod- duction techniques, as RASTA [10], are also thinkable. els). This way, high classification accuracy can be ob- Another option is to build sound models from a large col- tained with remarkably simple sound models. The accu- lection of labeled “noisy” percussion instrument sounds racy is on average around 20% higher than with general extracted from polyphonic audio recordings [9]. The main models alone. assumption in this method is that, in average on the train- ing database, the noise shows considerably higher vari- ability than the drum sounds. 1. INTRODUCTION The approach by [4, 13] also assumes that percussion This paper deals with automatic symbolic transcription of sound characteristics show less variability than surround- percussion mixed in polyphonic audio signals. That is, ing noise; however, this assumption is not made at the given a multi-timbral audio signal, the goal is twofold: to scope of a training database, but rather at the smaller scope automatically classify its percussion sounds and to auto- of individual audio recordings. They design very simple, matically determine their positions on the time axis. general sound templates for each percussive sound (ac- Snare drum sounds, for instance, can show large varia- tual waveform templates [4, 13], at the difference with tions in timbral characteristics. In automatic isolated sound the sound models previously mentioned) and find the sev- classification [8], this is typically dealt with from a ma- eral sub-segments in the audio recording at hand that best chine learning perspective: a sound model (roughly, thresh- match those templates (by means of a correlation func- olds for specific relevant signal features) is built from a tion). This process is iterated several times and sound large, diverse collection of labeled snare drum sounds. templates are gradually refined by time-domain averaging This model is subsequently used to assign labels to un- of the best-matching segment in the very audio recording known instances. at hand. However, in our framework, the temporal boundaries Our approach is to combine general, prior knowledge of the sounds to classify are unknown. A list of potential about the sound characteristics of percussion instrument percussion sound occurrences must be first extracted from families with on-the-fly acquired knowledge of recording- the audio recording. Different rationales have been pro- specific sounds. Instead of pursuing universally valid sound posed to solve this issue. For instance, one may assume models and features [8, 9], unique, localized sound mod- that percussion sounds are bound to occur in fixed-length els are built for every recording using features that are lo- regions around specific time-points, either sharp onsets cally noise-independent and give good class separation. [3, 4] or beats at the tatum level [5, 11, 1]. Instead of actually synthesizing new waveform templates Dealing with polyphonic recordings raises an additional from the audio signal [4, 13], we tailor (in a gradual fash- issue: percussion sounds are superposed with, and sur- ion) feature spaces to the percussion sounds of each record- rounded by, a high level of “noise”, i.e. other instruments ing. Therefore, an onset detector yields N potential drum sound occurrences that are subsequently processed as fol- Permission to make digital or hard copies of all or part of this work for lows: personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that 1. Classification using general drum sound models copies bear this notice and the full citation on the first page. c 2004 Universitat Pompeu Fabra. 2. Ranking and selection of the M < N most reliably classified instances Bagging or boosting approaches has turned out to yield better results when compared to other more traditional 3. Feature selection and design of localized models us- machine learning algorithms [9]. ing those M instances 4. Classification of the N segments using the localized 2.3. Instance ranking and selection models The instances classified by the general models must be As it turns out, our attempts at the automatic rank- parsed in order to derive the most likely correctly classi- ing and selection has not yet provided satisfactory results. fied subset. Several rationales are possible. For instance, Therefore, we corrected manually the output of steps 1 we can use instance probability estimates assigned by some and 2 and provided only correct percussion sound instances machine learning algorithms as indicators of correct clas- to the feature selection step. Consequently, in this pa- sification likelihood. Another option is to use clustering per, we present a more focused evaluation of the local- techniques. Instances of the same percussion instrument, ized sound model design, as well as a proper comparison which we are looking for, would form the most populated between general sound model and localized sound model and compact clusters while other drum sounds and non- performances. Using the corrected instance subsets, we percussive instances would be outliers. investigate how the performance of the localized models However, as mentioned above, we went for a “safe” op- evolve as increasingly smaller proportions of the data is tion: manually parsing the output of the general classifica- used for feature selection and training. tion schemes. Using the corrected output, we investigated Automatic techniques for instance ranking are currently how the performance of the localized models evolved as being evaluated. Together with the evaluation of the fully increasingly smaller proportions of the instances selected automatic system they are the object of a forthcoming pa- from a recording were used to classify the remaining sound per. instances of the recording. Since dependence between training and test data sets is known to yield overly op- 2. METHOD timistic results, these test were performed by doing ran- domized, mutually exclusive splits on the complete col- 2.1. Data and features lection of instances for each recording. The training data set for general model building consists of 1136 instances (100 ms long): 1061 onset regions taken 2.4. Feature selection from 25 CD-quality polyphonic audio recordings and 75 isolated drum samples. These were then manually anno- A collection of correctly classified instances from a record- tated, assigning category labels for kick, snare, cymbal, ing are then used to build new, localized sound models. kick+cymbal, snare+cymbal and not-percussion. Other Relevant features for the localized sound models are percussion instruments like toms and Latin percussions selected using a Correlation-based Feature Selection (CFS) were left out. Cymbals denote hi-hats, rides and crashes. algorithm that evaluates attribute subsets on the basis of Annotated test data consists of seventeen 20-second ex- both the predictive abilities of each feature and feature cerpts taken from 17 different CD-quality audio record- inter-correlations [6]. This method yields a set of features ings (independent from the training data set). The total with recording-specific good class separability and noise number of manually annotated onsets in all the excerpts is independence. The localized models may differ from the 1419, average of 83 per excerpt. general models in two respects: they may be based on Training and test data are characterized by 115 spec- 1) a different feature subset (feature space) and 2) differ- tral features (averages and variances of frame values) and ent threshold values (decision boundaries) for specific fea- temporal features (computed on the whole regions), see tures. As a general comment, the features showed signifi- [9] and [12] for details. cantly better class-homogeneity for localized models than The experiments described in the remainder of this pa- for the general models. per were conducted with Weka . 1 2.5. Classification with localized models 2.2. Classification with general models Finally, the remaining instances must be classified with In order to design general drum sound models, we first the recording-specific (localized) models. propose to reduce the dimensionality of the feature space For this final step, we propose to use instance-based by applying a Correlation-based Feature Selection (CFS) classifiers, such as 1-NN (k-Nearest Neighbors, with k = algorithm (Section 2.4) on the training data. From the total 1). Instance-based