A Quantitative, Parametric Model of Musical Tension by MASSACHUSETTS INSTITUTE Morwaread M. Farbood OF TECHNOLOGY A.B. Music MAR 0 1 2006 Harvard College (1997) M.S. Media Arts and Sciences LIBRARIES Massachusetts Institute of Technology (2001) Submitted to the Program in Media Arts and Sciences, School of Architecture and Planning, in partial fulfillment of the requirements for the degree of Doctor of Philosophy !ROTCH}', at the MASSACHUSETTS INSTITUTE OF TECHNOLOGY February, 2006 @ Massachusetts Institute of Technology 2006. All rights reserved. Author.. -Program in Media Arts and Sciences January 2006 Certified by. Tod Machover P of Music and Media Thesis Supervisor Accepted by........ Andrew B. Lippman L9<air, Departmental Committee on Graduate Students A Quantitative, Parametric Model of Musical Tension by Morwaread M. Farbood Submitted to the Program in Media Arts and Sciences, School of Architecture and Planning, on January 2006, in partial fulfillment of the requirements for the degree of Doctor of Philosophy Abstract This thesis presents a quantitative, parametric model for describing musical tension. While the phenomenon of tension is evident to listeners, it is difficult to formalize due to its subjective and multi-dimensional nature. The model is therefore derived from empirical data. Two experiments with contrasting approaches are described. The first experi- ment is an online test with short musical excerpts and multiple choice answers. The format of the test makes it possible to gather large amounts of data. The second study requires fewer subjects and collects real-time responses to musical stimuli. Both studies present test subjects with examples that take into account a number of musical parameters including harmony, pitch height, melodic ex- pectation, dynamics, onset frequency, tempo, and rhythmic regularity. The goal of the first experiment is to confirm that the individual musical parame- ters contribute directly to the listener's overall perception of tension. The goal of the second experiment is to explore linear and nonlinear models for predicting tension given descriptions of the musical parameters for each excerpt. The resulting model is considered for potential incorporation into computer- based applications. Specifically, it could be used as part of a computer-assisted composition environment. One such application, Hyperscore, is described and presented as a possible platform for integration. Thesis supervisor: Tod Machover Title: Professor of Music and Media Thesis Committee /1 Thesis supervisor......... ....... v-.................-- --- - - - -- - - / .. .... .... ... ... Tod Machover Professor of Music and Media MIT Program in Media Arts and Sciences Thesis reader.................v/...........- ---------------------...............- Fred Lerdahl Fritz Reiner Professor of Music Columbia University Thesis reader.. .................M... .............................. Frangois Pachet Research Scientist Sony Computer Science Laboratory, Paris Acknowledgments This work couldn't have been done without the help and support of many friends, colleagues, and mentors. I'd like to thank the following people in par- ticular: My advisor Tod Machover, who has given me so many great opportunities and supported me all these years. My thesis committee members Fred Lerdahl and Frangois Pachet, both of whose work has inspired me. Colleagues and friends who contributed to my thesis work: Tristan Jehan, Josh McDermott, Joe Presbrey, and Bernd Schoner. Friends: Tristan Jehan, Mike Hawley, Junne Kamihara, Ariane Martins, An- thony Beuriv6, J. C., and many others from MIT, Harvard, and beyond. Fellow students, past and present, from the Opera of the Future group. Jeanne Bamberger, Marcus Thompson, David Deveau, and the rest of the MIT music department. Bernd and the Schoner family. My family: Mom, Dad, and Mina. Acknowledgments 7 8 Acknowledgments To the memory of David Lewin (1933-2003) Acknowledgments 9 10 Table of Contents 1 Introduction 23 1.1 Musical parameters. ... ... ... .. .. ... ... ... 24 1.2 Objective and approach.. ... ... ... ... ... .. .. 24 1.3 M otivation . ... ... ... ... ... ... .. ... ... ... 25 1.4 Overview of Thesis ... .. .. ... .. .. ... .. .. ... .. 26 2 Prior Work 29 2.1 Schenkerian analysis.. .... .. ... .. .. .. ... .. 29 2.2 Generative theory of tonal music .. ... .. .. ... .. .. .. 30 2.3 Lerdahl's tonal tension model . .. .. .. .. .. .. .. .. .. 31 2.3.1 Perception of tonal hierarchies . .. .. .. .. .. ... 32 2.3.2 Lerdahl's criteria for tonal tension.. .. .. .. .. 33 2.4 Melodic structures . .. .. ... .. .. .. .. .. .. .. ... 35 2.4.1 Implication-Realization theory . .. .. .. .. .. .. .. 35 2.4.2 Margulis' model of melodic expectation . .. .. .. .. 36 2.4.3 Melodic contour and tension .. .. .. .. .. .. 38 2.5 Some comments on tension outside the tonal context . .. .. 39 2.6 Modeling expressive characteristics of music ... .... ... 40 2.7 The role of prior work in defining a new model . .... ... 42 3 Method and Experimentation 43 3.1 Experiment 1: Web-based study . ... ... .... ... ... 43 3.1.1 Procedure . .. ... .... ... ... ... ... .. 43 3.1.2 Selection of musical examples . ... ... ... .... .. 44 3.1.3 Design of musical examples .. ... ... .. ... ... 47 3.1.4 Demographics .. ... .. ... .. .. ... .. ... .. 48 3.2 Experiment 2 .. ... .. ... .. .. ... .. .. .. ... .. 50 3.2.1 Test format .. ... .. .. ... .. .. ... .. .. ... 50 4 Model Results 55 4.1 Experiment 1 analysis .. .. .. .. .. .. .. .. .. .. .. .. 55 4.1.1 Pitch height . .. .. .. .. .. .. .. .. .. .. .. .. 55 4.1.2 Loudness .. .. .. .. .. .. .. .. .. .. .. .. .. 58 4.1.3 Onset frequency .. .. .. .. .. .. .. .. .. .. .. 59 4.1.4 Harmony .. .. .. .. .. .. .. .. ... -- - . -. 60 4.1.5 Rhythmic irregularity . .. .. .. .. .. .. .. .. .. 63 4.1.6 Interaction of features .. .. .. .. .. .. .. .. .. .. 64 4.1.7 Three corresponding features opposed by a fourth feature 67 4.1.8 Musicians versus non-musicians . .. .. .. .. .. .. 73 4.1.9 Summary .. .. .. .. ... .. .. .. .. .. -. 75 11 Contents 4.2 Experiment 2 Analysis..... .. .. .... 4.2.1 Feature graphs .... .... ..... 4.2.2 Linear correlation of individual features with empirical data 4.2.3 Musical experience ... .... .... 4.2.4 The goal: a predictive model . .... 4.3 Note on implementation of analysis tools ... 5 Applications 93 5.1 The original tension line in Hyperscore . ... ... .. .. ... 94 5.2 Later versions . .... ... ... ... ... .. .. .. .. .. 94 5.3 Final Hyperscore version ... ... .. ... .... ... .... 97 5.3.1 Harmonic tension line . ... .. .. ... .. .. ... 100 5.4 The future of Hyperscore .. ... ... .. ... .. .. ... 104 5.5 A new tension line . .. ... ... .. ... ... ....... 104 6 Conclusions 111 6.1 Summary ...... ............................... 111 6.1.1 Experimental results .. .. .. ... .. .. .. .. .. .. 111 6.1.2 The model... .. .. .. .. .. .. .. .. .. .. 112 6.1.3 Musicians vs. non-musicans . .. .. .. .. .. .. .. 113 6.1.4 Applications . .. .. .. .. .. .. .. .. .. 113 6.2 Discussion .. .. .. .. .. .. .. .. .. .. .. 114 6.3 Contributions . .. .. .. .. .. .. .. .. .. 115 Contents 13 6.4 Future directions and final remarks ... ... .... .... ... 115 Appendix A Musical excerpts from Experiment 2 117 Appendix B Feature analysis from Experiment 2 121 Bibliography 125 IA Contents List of Figures 2-1 Middleground analysis of Bach Prelude in C Major, Well-Tempered Clavier Bk. I [Forte and Gilbert, 1982] .. .. ... .. .. ... .. ... .. 30 2-2 GTTM analysis of a Bach chorale showing the prolongational tree, metrical grid, and grouping analysis [Lerdahl, 2001]. .. .. .. ... .. ... .. 32 3-1 Response choices for Experiment 1. .. ... .. .. .. .. .. .. .. 45 3-2 (a) Initial sample question in Experiment 1. All parameters are clearly increasing in tension. (b) Assumed "correct" answer: the graphical response indicating tension is increasing. .. .. ... .. .. .. .. ... .. .. 45 3-3 Three musical examples from group 001. .. .. .. .. .. .. .. 46 3-4 Example where two parameters, harmony and tempo, appear to be conflict- ing in their contribution to tension. .. .. .. .. .. .. .. .. .. 49 3-5 The first question in Experiment 1. All parameters are moving in the same direction: the harmony is relaxing, the tempo is slowing down, the volume is decreasing, and the pitch contour is falling consistently..... .. .. 49 3-6 Map showing the origin of subjects in Experiment 1. Numbers shown in- clude only the people who completed both the first and final surveys (image courtesy of Rich Chan and Google). .. .. .. .. .. ... .. 49 3-7 Top twelve countries of origin for subjects in Experiment 1. .. .. 50 3-8 Experiment 2: countdown to prepare subjects for excerpt playback. .. 51 3-9 Screen shot of Experiment 2 interface in data recording/subject listening mode. .. .. .. .. .. .. - --. .. .. 52 4-1 (a) Example where pitch height is the only feature changing. (b) The "cor- rect" answer corresponding to pitch height for (a). .. .. ... .. .. 56 4-2 Graph showing subjects' responses to changes in pitch height. .. ... .. 56 4-3 Four examples with varying degrees of change in pitch height. .. .. .. 57 4-4 Subjects' responses to musical examples shown in Figure 4-3. .. ... 57 4-5 (a) An example designed to measure subjects' responses to loudness. (b) Answer corresponding to loudness for example shown in (a). .. .. .. 58 4-6 Graph showing subjects' responses to all examples that have loudness as the only changing feature. .. .. .. .. .. .. .. .. .. .. .. 58 4-7 (a) An example designed to test subjects' responses to