Profiles of Pitch Classes Circularity of Relative Pitch and Key – Experiments, Models, Computational Music Analysis, and Perspectives vorgelegt von Diplom-Mathematiker Hendrik Purwins aus Münster von der Fakultät IV – Elektrotechnik und Informatik der Technischen Universität Berlin zur Erlangung des akademischen Grades Doktor der Naturwissenschaften – Dr. rer. nat. – genehmigte Dissertation Promotionsausschuss: Vorsitzender: Prof. Dr. B. Mahr Berichter: Prof. Dr. K. Obermayer Berichter: Prof. Dr. W. Auhagen Tag der wissenschaftlichen Aussprache: 15. August 2005 Berlin 2005 D 83 Abstract The doubly-circular inter-relation of the major and minor keys on all twelve pitch classes can be depicted in toroidal models of inter-key relations (TOMIR). We demonstrate convergence of derivations on the explanatory levels of a) an experiment in music psychology, b) geomet- rical considerations in music theory, and c) computer implementation of musical listening scenarios. Generalizing Shepard [1964] to full overtone spectra, circular perception of relative pitch is experimentally verified and mathematically modeled as the spectrum of pitch differences, derived from virtual pitch [Terhardt, 1998]. Musical examples of circular pitch, tempo, and loudness are analyzed. For each pitch class calculating the intensity in a musical recording, our constant quo- tient (CQ-) profile method is a) consistent with psychological probe tone ratings, b) highly efficient, c) computable in real-time, d) stable with respect to sound quality, e) applicable to transposition, f) free of musical presupposition, except approximately equal temperament, and g) sensitive to substantial musical features (style, composer, tendency towards chromati- cism, and major/minor alteration) in a highly compact reduction. In Bach, Chopin, Alkan, Scriabin, Hindemith, and Shostakovich, the latter features are extracted from overall CQ- profiles by classification (support vector machine [SVM], regularized discriminant analysis) and clustering. Their inter-relations are visualized by a technique called Isomap. Leman [1995] models acquisition of inter-key relations. Artificial cadences are preprocessed through modeling the auditory periphery. Then, in the toroidal self organizing feature map (SOM) a TOMIR is arrived at. We extend this approach by a) using a great many actual performances of music as input, and/or b) not presupposing toroidal topology a priori. Visu- alizing CQ-profiles from Bach’s WTC by correspondence analysis (CA) and Isomap reveals the circle of fifths. TOMIRs evolve from a) average CQ-profiles of Chopin’s Préludes and toroidal SOMs, b) overall annotated duration profiles of Bach’s WTC I from score and CA, and c) formalization of music theoretical derivation from Weber [1817]’s key chart by the topographic ordering map introduced here. These results are consistent with Krumhansl and Kessler [1982]’s visualization of music psychology ratings. As auxiliary results, we discuss fuzzy distance, spatial and systematized synesthetic vi- sualization in conjunction with beat weight detection on multiple time scales and suggested applications to automatic tone center tracking. Furthermore, statistics on the key preference of various composers are collected. Based on the latter, CA visualizes composer inter-relations. This thesis substantially contributes to content retrieval (MPEG-7), automated analysis, interactive audio-based computer music, and musicology. Acknowledgments I am very grateful towards my supervisor Klaus Obermayer for his support. His vivid Neural Information Processing (NI) Group provided an effective infrastructure and an open and pleasant atmosphere of scientific adventure. I want to thank some people that made substantial contributions to this research and that have been especially inspiring and encouraging. First of all I want to express my gratitude towards Benjamin Blankertz. We worked together on auditory models, the constant Q profil technique, toroidal models of inter-key relations, and machine learning. I collaborated with Immanuel Normann on experiments and models of circular perception of relative pitch in harmonic complex tones. Thore Graepel and I worked on correspondence analysis. Always helpful, Christian Piepenbrock set up the initial software infrastructure for the NI group. Thanks to Wilhelm Köhler and Uli Pralle for maintaining a huge network of workstations. I am grateful to a couple of computer wizards, mostly NI group members, who supported me: Robert Anniés, Hauke Bartsch, Anca Dima, Thomas Knop, Roman Krepki, Nabil Lashin, Johannes Mohr, André Przy- wara, Stephan Schmitt, Holger Schöner, Susanne Schönknecht, Lars Schwabe, Frank Segtrop, Michael Sibila, Ilya Souslov, and Peter Wiesing. I have to thank the audio experts Kamil Adiloglu, Michael Natterer, and Marcus Verwiebe. Cornelius Weber and Oliver Beck were helpful “room mates” at the office. I owe a lot to Thomas Noll. Several times he pointed out to me new fruitful directions in my research. In addition, he always let me use the equipment of the research group Mathematical Music Theory. Jörg Garbers also was helpful. I learned much from Ulrich Kock- elkorn’s well-founded statistical insight. Thanks for his support in solving statistical problems. I want to express my gratitude towards the people from the studio for electroacoustic music at TU Berlin for letting me use the equipment and for support: Gerhard Behles, Onnen Bock, Folkmar Hein, Manfred Krause†, Axel Roebel, and anonymous subjects of the listening tests. At Stanford I was warmly welcomed by Jonathan Berger, Stefan Bilbao, Fabien Gouyon, Stefan Harmeli, Heidi Kugler, Fernando Lopez-Lezcano, Max Matthews, Eleanor Selfridge-Field, Julius Orion Smith. I had a very fruitful exchange with Craig Sapp. I had inspiring discussions with Marina Bosi, Takuya Fujishima, Dan Gang, David Huron, Dan Levitin, Bob Sturm, and Harvey Thornburg. At McGill Auditory Research Laboratory, I have to thank Al Bregman for thorough discussions. Pierre Ahad was very helpful with setting up my working environment. I had an inspiring discussion with Jim Ramsay. Antonie Budde, Christopher W. Jones, Hans Peter Reutter, and Kotoka Suzuki advised me in music analysis. Mark Lindley shared his insight into temperament with me. Young-Woo Lee played many piano samples. Joo-Kyong Kim recorded for me. Thanks to Wolfgang Auhagen for evaluating this thesis. I am indebted to Stefan Bilbao for proof reading the major part of the thesis and to Thomas Gonsior, Christo- pher W. Jones, Timour Klouche, Ingo Lepper, Dennis Livingston, Thomas Noll, and Joshua Young for proof reading sections thereof. In addition I have to thank Matthias 4 Bode, Joachim Buhmann, Matthias Burger, Gunnar Eisenberg, Georg Hajdu, Sepp Hochreiter, Petr Janata, Carsten Kruse, Marc Leman, Markus Lepper, Fabien Lévy, Guerino Mazzola, Helga de la Motte-Haber, Hans Neuhoff, Miller Puckette, Ingo Schießl, Sambu Seo, Manfred Stahnke, Baltasar Trancon-y-Widemann, Gregor Wen- ning, Daniel Werts, and Ioannis Zannos. Since so many people supported me, I would like to ask pardon for forgetting someone. I am obliged to my parents who supported me in the most comprehensive sense of the word but also very specially related to this thesis: my father, who triggered my interest in physics, and my mother, who provided me with music education from early on. Last not least I have to thank my friends. I am grateful for financial support by the German National Academic Foundation (Studienstiftung des deutschen Volkes) and Axel Springer Stiftung. 5 6 Contents Is Music Reducible to Numbers, Mechanics, and Brain States? 13 AMachinistIntrudingMusicology . 13 Profiles from Theory, Score, and Sound . 14 Inter-KeyRelations. 16 Infinity – Construction of “Paradoxes” . 20 Interdisciplinary Perspectives on Music . 21 Surface Level or Underlying Deep Structure? . 21 Sister Sciences Mathematics and Music . 22 TheBrainComprehended? . 24 Machine“learning” . 26 Ground Truth through Inquiry of the Listener . 27 Organization of Thesis . 28 1 From Pitch Criticism to Tone Center Geometry 31 1.1 FuzzyFoundationsofPitch . 31 1.1.1 The Psychoacoustic Nature of Pitch . 31 1.1.2 SumsofSineTones–NaturalSounds . 32 1.1.3 FrequencyScaling .......................... 33 1.1.4 Loudness Summation of Frequency Components . 34 1.1.5 Singling Out Frequency Components . 36 1.1.6 Few Frequencies – Several Tones . 37 Beating, Roughness, and Combination Tones . 37 VirtualPitch.......................... 38 1.1.7 Several Frequencies – One Tone . 40 1.2 MusicalLandscapes ............................. 43 1.2.1 Tonnetz, Temperaments, and Torus . 45 CriticismoftheNotionofTone. 47 1.2.2 PitchClasses ............................. 48 1.2.3 Pitch Decomposition into Pitch Class and Brightness . 48 1.2.4 ChordsandHarmony . 51 TheKeytoTonality ..................... 51 Categorization of Synchronous Pitch Classes . 52 SyntaxandMusic....................... 53 1.2.5 KeyProximity............................. 54 7 Contents 1.2.6 ChartofToneCenters . 54 1.2.7 Torus by Synchromatic and Enharmonic Identification . 56 1.2.8 KeyCharacter............................. 59 1.2.9 Key–ReductiontoanEmpiricalProfile. 61 ProbeToneRatings. 61 A TOMIR from Probe Tone Ratings . 63 Limitations as a Psychological Reference . 64 1.2.10 Spatial and Planar Views of Tones, Triads, and Keys . 64 1.2.11 SequencesofToneCenters . 67 TemporalGranularity . 67 Modulation .......................... 67 2 Signals, Ear Models, Computer Learning 69 2.1 ConstantQTransform ............................ 69 2.1.1 DerivingFilterParameters