Electronic Theses and Dissertations UC San Diego Peer Reviewed Title: Physical and perceptual aspects of percussive timbre Author: Brent, William Acceptance Date: 2010 Series: UC San Diego Electronic Theses and Dissertations Degree: Ph. D., UC San Diego Permalink: http://www.escholarship.org/uc/item/5bx4j1fj Local Identifier: b6851118 Abstract: This dissertation explores relationships between perceptual dimensions of percussive timbres and measurements produced by several signal analysis algorithms. The literature of psychophysical timbre experiments since 1941 is reviewed with respect to two contrasting approaches. The earliest attempts at unraveling the interdependent aspects of timbre perception employed multiple adjective scales intended to describe various sonic features. Following developments in the technique of multidimensional scaling (MDS) in the 1960s, several researchers began to apply scaling techniques to data sets of timbre similarity judgments. At present, the majority of timbre studies are based on MDS. In spite of such advancements, the range of musical timbres has only begun to be explored from a perceptual viewpoint, and a significant gap exists in the literature for percussive instruments. The signal analysis algorithms employed in this research are introduced in the context of timbreID--a timbre analysis software library written by the author. The library's adaptability is illustrated with respect to several musical research applications in Pure data. This flexibility is shown to be beneficial in the case of two percussive instrument classification tests, in which the effectiveness of perceptually weighted spectral features like mel- and Bark-frequency spectrum are evaluated alongside other standard analysis techniques from the music information retrieval literature. In the final chapter, a perceptual experiment involving 30 diverse percussion timbres is carried out. The study confirms the importance of spectral centroid and attack duration as predictors of perceptual dimensions, and reveals two additional dimensions that may be unique to percussive timbres: "dryness" and "noisiness." A predictive model is generated using multiple linear regression, and results indicate that the noisiness dimension cannot be predicted as accurately as dimensions relating to spectral center of gravity and attack time. Thus, there is a clear need for an effective measure of perceptual noisiness for accurate description of percussive timbre eScholarship provides open access, scholarly publishing services to the University of California and delivers a dynamic research platform to scholars worldwide. Copyright Information: All rights reserved unless otherwise indicated. Contact the author or original publisher for any necessary permissions. eScholarship is not the copyright owner for deposited works. Learn more at http://www.escholarship.org/help_copyright.html#reuse eScholarship provides open access, scholarly publishing services to the University of California and delivers a dynamic research platform to scholars worldwide. UNIVERSITY OF CALIFORNIA, SAN DIEGO Physical and Perceptual Aspects of Percussive Timbre A dissertation submitted in partial satisfaction of the requirements for the degree Doctor of Philosophy in Music by William Brent Committee in charge: Professor Miller Puckette, Chair Professor David Borgo Professor Diana Deutsch Professor Shlomo Dubnov Professor Shahrokh Yadegari 2010 Copyright William Brent, 2010 All rights reserved. The dissertation of William Brent is approved, and it is acceptable in quality and form for publication on micro- film and electronically: Chair University of California, San Diego 2010 iii DEDICATION For my father. iv EPIGRAPH One of the most striking paradoxes concerning timbre is that when we knew less about it, it didn't pose much of a problem. |Philippe Manoury v TABLE OF CONTENTS Signature Page . iii Dedication . iv Epigraph . .v Table of Contents . vi List of Figures . ix List of Tables . xii Acknowledgements . xiii Vita and Publications . xv Abstract of the Dissertation . xvi Chapter 1 Introduction . .1 Chapter 2 Historical Overview of Timbre Studies . .7 2.1 Experimental Design . .7 2.2 Verbal Attribute Studies . 10 2.2.1 Von Bismarck . 10 2.2.2 Kendall & Carterette . 13 2.2.3 Freed . 15 2.3 Multidimensional Scaling . 17 2.3.1 Grey . 20 2.3.2 Iversen & Krumhansl . 24 2.3.3 McAdams . 27 2.3.4 Lakatos . 32 2.4 Summary . 34 Chapter 3 Objective Analysis . 37 3.1 Low level features . 38 3.1.1 Spectral Centroid . 38 3.1.2 Spectral Spread . 39 3.1.3 Spectral Skewness . 39 3.1.4 Spectral Kurtosis . 40 3.1.5 Spectral Brightness . 41 3.1.6 Spectral Rolloff . 41 3.1.7 Spectral Flatness . 41 vi 3.1.8 Spectral Irregularity . 42 3.1.9 Spectral Flux . 43 3.1.10 Zero Crossing . 44 3.1.11 Log attack time . 44 3.1.12 Features for harmonic spectra . 44 3.2 High level features . 45 3.2.1 Cepstral Analysis . 47 3.2.2 Mel Frequency Cepstrum . 56 3.2.3 Critical Bands and the Bark Scale . 59 3.3 Interpreting BFCCs . 63 3.4 Summary . 65 3.5 Acknowledgements . 66 Chapter 4 timbreID . 67 4.1 Introduction . 67 4.2 Feature Extraction Objects . 68 4.2.1 Available Features . 70 4.2.2 Open-ended analysis strategies . 70 4.2.3 Details of Analysis Algorithms . 72 4.3 The Classification object . 73 4.3.1 timbreID settings . 75 4.4 Applications . 75 4.4.1 Plotting Cepstrograms . 75 4.4.2 Percussive Instrument Recognition . 77 4.4.3 Vowel Recognition . 78 4.4.4 Target-based Concatenative Synthesis . 81 4.4.5 Timbre ordering . 83 4.4.6 Mapping sounds in timbre space . 85 4.5 Conclusion . 89 4.6 Acknowledgements . 89 Chapter 5 Classification Performance Evaluation . 90 5.1 Examining Percussive Timbres . 90 5.2 Method . 91 5.2.1 Instruments . 93 5.2.2 Analysis Strategies . 95 5.3 Results . 98 5.3.1 30 Diverse Timbres . 98 5.3.2 30 Similar Timbres . 105 5.3.3 Signal distortion . 113 5.4 Conclusions . 115 vii Chapter 6 A Perceptual Timbre Space for Percussive Sounds . 118 6.1 Method . 118 6.1.1 Participants . 121 6.1.2 Apparatus . 121 6.1.3 Stimulus Materials . 122 6.2 Procedure . 123 6.3 Results . 124 6.3.1 Consistency of Ratings . 124 6.3.2 Adjective correlations . 127 6.3.3 Physical Correlates of Perceptual Judgments . 129 6.3.4 Principal Components Analysis . 132 6.3.5 A Predictive Model . 136 6.4 Conclusions . 137 Appendix A Spectra of Timbre Sets . 140 Appendix B 100 Adjectives . 145 Bibliography . 147 viii LIST OF FIGURES Figure 2.1: Participants' PMH ratings as a function of mallet identity. 16 Figure 2.2: Spectra for the steady state of 9 instrument tones used in [WG72]. 19 Figure 2.3: Grey's three-dimensional timbre space. 21 Figure 2.4: Spectral characteristics of the trumpet and trombone are ex- changed. 23 Figure 2.5: The three-dimensional timbre space produced by McAdams et al. 28 Figure 2.6: Two-dimensional synthesis parameter and perceptual spaces, from Caclin et al. 31 Figure 2.7: Two-dimensional timbre space for the \combined" stimulus set. 34 Figure 3.1: Bongo (left) and metal bowl (right) spectra, with spectral cen- troids of 926 Hz and 2858 Hz. 39 Figure 3.2: A tambourine spectrum with flatness value of 0.42. 42 Figure 3.3: Spectrogram of a bass drum strike. 46 Figure 3.4: Magnitude spectrum of a 440 Hz sawtooth wave. 48 Figure 3.5: A cepstral quefrency peak resulting from a 440 Hz sawtooth wave. 50 Figure 3.6: Quefrency peaks resulting from 165 Hz (top) and 220 Hz (bot- tom) sung vowels. 53 Figure 3.7: Magnitude spectra for 440 Hz (left) and 880 Hz (right) sawtooth waves................................. 55 Figure 3.8: Cepstral coefficients 1 through 30 for a voiced vowel sung at 220 Hz (left) and 165 Hz (right). 55 Figure 3.9: Hz plotted against mel units, from [SVN37]. 57 Figure 3.10: A mel-spaced triangular filterbank, from [DM80]. 58 Figure 3.11: Critical bandwidths and related units vs. frequency, from [ZF90]. 60 Figure 3.12: Mels (top) and Barks (bottom) plotted against linear frequency. 62 Figure 3.13: The first six cosine transform basis functions. 64 Figure 4.1: Generating a mixed feature list. 71 Figure 4.2: Generating a time-evolving feature list. 72 Figure 4.3: timbreID in a training configuration. 73 Figure 4.4: Cepstrogram of three glockenspiel tones. 76 Figure 4.5: Cepstrogram of two nipple gong tones. 77 Figure 4.6: An instrument recognition and sample mapping patch. 78 Figure 4.7: Sending training snapshots and continuous overlapping cepstral analyses to timbreID. 79 Figure 4.8: Fifty-one percussion sounds ordered based on a user-specified weighting of 5 features. 84 Figure 4.9: Speech grains mapped with respect to the 2nd and 3rd BFCC. 85 Figure 4.10: String grains mapped with respect to amplitude and fundamen- tal frequency. 86 ix Figure 4.11: Sixty percussion samples colored by cluster. 87 Figure 4.12: Grains from a 20-20,000 Hz frequency chirp plotted with respect to spectral centroid and the 2nd BFCC. 88 Figure 5.1: Training and testing instances of tam tam strikes. 92 Figure 5.2: Scores for individual low level features, combined low level fea- tures, and high level features. 99 Figure 5.3: Accuracy vs. coefficients for all high level features. 100 Figure 5.4: CR vs. OD vs. accuracy for BFCCs. 101 Figure 5.5: Scores for individual low level features, combined low level fea- tures (CLL), and high level features using multiple frame analysis.102 Figure 5.6: Accuracy vs. coefficients for all high level features using multiple- frame analysis. ..