Perspectives on Memory for Musical Timbre Kai Kristof Siedenburg Music Technology Area Department of Music Research Schulich School of Music McGill University Montreal, Canada March 2016 A thesis submitted to McGill University in partial fulfillment of the requirements for the degree of Doctor of Philosophy. c 2016 Kai Siedenburg 2016/03/29 bong bing bang bung bäng für Eliza ii Contents Abstract/Résumé .................................. vi Acknowledgments ................................. x Contribution of authors .............................. xii List of Figures ................................... xvii List of Tables .................................... xx List of Acronyms .................................. xxi 1 Introduction 1 1.1 Ideas and questions ............................. 1 1.2 Methods ................................... 8 1.3 Thesis outline ................................ 9 I Background 13 2 Three conceptual distinctions for timbre 15 2.1 Introduction ................................. 15 2.2Soundeventvs.timbre........................... 16 2.3 Qualitative vs. source timbre ........................ 18 2.4 Timbre on different scales of detail .................... 19 2.5 Conclusion .................................. 21 3 A review of research on memory for timbre 23 3.1 Introduction ................................. 23 3.2 Basic concepts in memory ......................... 25 3.3 Key findings in auditory memory ..................... 28 Contents iii 3.3.1 Auditory sensory memory ..................... 29 3.3.2 Memory for noise .......................... 30 3.4 Memory for timbre ............................. 31 3.4.1 Timbre in memory for melodies .................. 32 3.4.2 Neurophysiological perspectives and long-term memory ..... 33 3.4.3 Short-term timbre recognition ................... 35 3.5 Summary and open questions ....................... 42 II Timbre sequences 45 4 Short-term recognition of timbre sequences 47 4.1 Introduction ................................. 48 4.1.1 Timbre recognition in the literature ................ 49 4.1.2 Retention of pitch and timbre and musical training ....... 50 4.1.3 Similarity effects .......................... 52 4.1.4 The present study .......................... 53 4.2 Experiment 1: Group, length, pitch variability, and timbre dissimilarity 54 4.2.1 Method ............................... 54 4.2.2 Results ................................ 57 4.2.3 Discussion .............................. 60 4.3 Experiment 2: Length and pitch variability ................ 61 4.3.1 Method ............................... 62 4.3.2 Results ................................ 63 4.3.3 Discussion ............................. 64 4.4 Experiment 3: Similarity and position ................... 65 4.4.1 Method ............................... 65 4.4.2 Results ................................ 67 4.4.3 Discussion .............................. 68 4.5 Experiment 4: Facets of similarity ..................... 70 4.5.1 Method ............................... 70 4.5.2 Results ................................ 71 4.5.3 Discussion .............................. 72 4.6 Response choice and timbre dissimilarity ................. 72 iv Contents 4.7 General discussion ............................. 74 5 Auditory and verbal memory in North Indian tabla drumming 79 5.1 Introduction ................................. 80 5.1.1 The North Indian tabla ....................... 81 5.1.2 Voice superiority effects ...................... 82 5.1.3 Familiarity and chunking ...................... 84 5.1.4 The present experiment ...................... 86 5.2 Methods ................................... 87 5.2.1 Participants ............................. 87 5.2.2 Stimuli ................................ 88 5.2.3 Presentation and apparatus .................... 91 5.2.4 Procedure and design ........................ 91 5.3 Results .................................... 92 5.4 Discussion .................................. 95 III Source categories and familiarity 101 6 Acoustic and categorical dissimilarity of musical timbre 103 6.1 Introduction ................................. 104 6.1.1 Sound source categories and similarity .............. 106 6.1.2 Similarity and categorization .................... 108 6.1.3 The present study .......................... 109 6.2 Experiment 1: Identification and familiarity ............... 110 6.2.1 Method ............................... 110 6.2.2 Results ................................ 113 6.2.3 Discussion .............................. 115 6.3 Experiment 2: Timbre dissimilarity of acoustic recordings and synthetic transformations ............................... 116 6.3.1 Method ............................... 117 6.3.2 Results ................................ 119 6.3.3 Discussion .............................. 123 6.4 Dissimilarity models and analyses ..................... 125 v 6.4.1 Approach .............................. 126 6.4.2 Acoustic model ........................... 129 6.4.3 Including categorical variables ................... 130 6.4.4 Discussion .............................. 134 6.5 Conclusion .................................. 136 7 Familiarity and attentional maintenance in memory for timbre 139 7.1 Introduction ................................. 140 7.2 Experiment 1: Material and delay ..................... 143 7.2.1 Methods ............................... 143 7.2.2 Results ................................ 149 7.2.3 Summary and discussion ...................... 152 7.3 Experiment 2: Material, suppression, and group ............. 154 7.3.1 Methods ............................... 154 7.3.2 Results ................................ 157 7.3.3 Summary and discussion ...................... 159 7.4 Questionnaire data ............................. 161 7.5 General discussion ............................. 163 7.6 Appendix: Transformation and selection of sounds ............ 166 IV Conclusion 169 8 Facets of memory for musical timbre 171 8.1 Summary .................................. 171 8.2 Factors that affect timbre cognition .................... 174 8.2.1 Overview .............................. 174 8.2.2 Discussion .............................. 175 8.3 Processes and principles in memory for timbre .............. 183 8.3.1 Processes .............................. 183 8.3.2 Principles .............................. 188 8.4 Remarks on timbre in theories of music listening ............. 190 References 199 vi Abstract This thesis studies musical timbre cognition from the perspective of short-term recog- nition and dissimilarity rating tasks. Four independent experimental studies provide in-depth investigations into the role of a) timbre similarity and concurrent pitch vari- ability in short-term memory for timbre, b) the impact of sound source categories and familiarity of tones and sequences in timbre recognition and dissimilarity ratings, c) the musical experience of participants and their memory maintenance strategies. A first theoretical part proposes three conceptual distinctions for the notion of mu- sical timbre and outlines a comprehensive map of previous research on memory for mu- sical timbre. The second part describes experiments on memory for timbre sequences. Using a serial-matching task, the first study shows that musicians do not differ from nonmusicians on sequences with constant pitch, but are better than nonmusicians in recognizing sequences that featured concurrent pitch variability. The perceptual dis- similarity of timbres is shown to be the major determinant of participants’ response choices, suggesting parallels of perceptual representation and short-term storage. A musical case study then explores auditory and verbal memory for tabla, a pair of drums central to North Indian classical music. Recognition memory of tabla students is compared to naïve controls, using sequences composed of drum sounds, as well as verbal sounds from a dedicated “tabla solfège” (bols) in which syllables stand for spe- cific tabla strokes. The results suggest a partial dissociation of memory for verbal and instrumental sounds. The third portion of this thesis explores the ways in which sound source categories retrieved from long-term memory affect timbre cognition. Considering timbre dissim- ilarity ratings for groups of tones from familiar acoustic instruments and unfamiliar digital transformations, the third study reports rating asymmetries that cannot be explained on acoustical grounds. Descriptors related to sound source categories signif- vii icantly improve an acoustic model of timbre dissimilarity. The fourth study highlights the finding that musicians and nonmusicians better recognize familiar acoustic timbres than unfamiliar (“non-lexical”) transformations of them. Detrimental effects of verbal and visual interference further suggest that short-term memory for timbre relies on attention-based maintenance of the auditory trace. This research synthesizes and advances the current knowledge on timbre cognition. Several links to hallmark effects of verbal memory are established, including acous- tic similarity, sequential chunking, lexicality, and active memory maintenance. The misleading notion that timbre should only be conceived of as a “surface feature” is countered by the demonstration that it can be deeply ingrained in human memory— which explains why it occupies a central role in music listening. viii Résumé Cette thèse aborde la question de la cognition du timbre musical du point de vue de la reconnaissance à court terme et de tâches de jugement de dissemblance. Quatre séries d’expériences indépendantes ont étudié