Measurement and Time Series Analysis of Emotion in Music
Total Page:16
File Type:pdf, Size:1020Kb
Measurement and Time Series Analysis of Emotion in Music Emery Schubert BE, BA (Hons) In Two Volumes Volume 1 A thesis submitted to the University of New South Wales in partial fulfilment of the requirements for the degree Doctor of Philosophy 1999 I hereby declare that this submission is my own work and to the best of my knowledge it contains no material previously published or written by another person, nor material which to a substantial extent has been accepted for the award of any other degree or diploma at UNSW or any other educational institution, except where due acknowledgement is made in the thesis. Any contribution made to the research by others, with whom I have worked at UNSW or elsewhere, is explicitly acknowledged in the thesis. I also declare that the intellectual content of this thesis is the product of my own work, except to the extent that assistance from others in the project's design and conception or in style, presentation and linguistic expression is acknowledged. Signed ___________________________ Date _________________ Emery Schubert - ii - Abstract This thesis examines the relations among emotions and musical features and their changes with time, based on the assertion that there exist underlying, culturally specific, quantifiable rules which govern these relations. I designed, programmed and tested a computer controlled Two-Dimensional Emotion Space (2DES) which administered and controlled all aspects of the experimental work. The 2DES instrument consisted of two bipolar emotional response (ER) dimensions: valence (happiness-sadness) and arousal (activeness-sleepiness). The instrument had a test- retest reliability exceeding 0.83 (p < 0.01, N = 28) when words and pictures of facial expressions were used as the test stimuli. Construct validity was quantified (r > 0.84, p < 0.01). The 2DES was developed to collect continuous responses to recordings of four movements of music (N = 67) chosen to elicit responses in all quadrants of the 2DES: “Morning” from Peer Gynt, Adagio from Rodrigo’s Concierto de Aranjuez (Aranjuez), Dvorak’s Slavonic Dance Op 42, No. 1 and Pizzicato Polka by Strauss. Test- retest reliability was 0.74 (p < 0.001, N = 14). Five salient and objectively quantifiable features of the musical signal (MFs) were scaled and used for time series analysis of the stimuli: melodic pitch, tempo, loudness, frequency spectrum centroid (timbral sharpness) and texture (number of different instruments playing). A quantitative analysis consisted of: (1) first order differencing to remove trends, (2) determination of suitable, lagged MFs to keep as regressors via stepwise regression, and (3) regression - iii - of each ER onto selected MFs with first order autoregressive adjustment for serial correlation. Regression coefficients indicated that first order differenced (∆) loudness and ∆tempo had the largest correlations with ∆arousal across all pieces, and ∆melodic pitch correlated with ∆valence for Aranjuez (p < 0.01 for all coefficients). The models were able to explain up to 73% of mean response variance. Additional variation was explained qualitatively as being due to interruptions, interactions and collinearity: The minor key and dissonances in a tonal context moved valence toward the negative direction; Short duration and perfect cadences moved valence in the positive direction. The 2DES measure and serial correlation adjusted regression models were, together, shown to be powerful tools for understanding relations among musical features and emotional response. - iv - Acknowledgements During the course of my doctoral work it was necessary to venture into several academic disciplines. I am delighted and honoured to acknowledge the various people from these areas who have contributed to making my study a most enriching experience. In the discipline of psychology I am grateful to Dr. Kate Stevens, Prof. Denis Burnham and Peter Keller. I am further appreciative of the establishment of AMPS (Australian Music and Psychology Seminars) by Dr. Stevens, for this provided me with a listening space and testing ground among old and new-found colleagues to whom I am also grateful. For expert guidance in statistics and time series modelling, and for generously sharing thoughts and precious time with me, I am indebted to Prof. William Dunsmuir. Prof. Dunsmuir possesses the perfect blend of knowledge and clarity of explanation of which I was thrilled to be a beneficiary. Frances Lovejoy provided limitless assistance in operating SPSS and shared much information on writing dissertations with me. In acoustics I received generous assistance from Assoc. Prof. Joe Wolfe and from Densil Cabrera. Prof. Wolfe has taken a great interest in my work and continuously provided me with invaluable feedback. Densil has generously made available to me his software in order to code several psychoacoustic musical features. He also spent considerable time providing insightful comments about my drafts. My harmonic analysis of music was kindly and enthusiastically checked and aided by Colin Watts. Several librarians at the - v - University of New South Wales were particularly helpful, among whom were Julie Nolan and Rita Keller. For tips and tricks I acquired on computer related matters I acknowledge Paul Sluis and the staff at Professional Development, and the staff at Vislab at the University of Sydney. It would be difficult to name all the people who encouraged me and contributed to my work, however I must include the following people: the staff at the School of Music and Music Education at the University of New South Wales; Dr. Carol Richardson; Dorottya Fabian; Prof. Jane Davidson; Emeritus Prof. Robert Gregson; Prof. Sandra Trehub; Dr. Bernice Laden; Michael Holack; Alfonso Egan; Nicola and Rosemary Bryden; Stephanie Wilson and Jennifer Christianson. Special mention is due to Assoc. Prof. Eric Sowey for the time he spent in making perceptive comments and helpful suggestions about my drafts. I also thank my amazingly patient and encouraging friends: Christina Mimmocchi; Jan Howe; Belinda Lawson; my mother, Agnes; and Puss (Motchko), whose versatile keyboard skills aided me with the challenge of minimising typographical errors. Nicole Cooper has supported me immensely and provided me with the benefit of her literary intellect, including the translation of an article from French. I am deeply grateful to Nicole and her family — Martin, Lois and Sooty — for their kindness, friendliness and words (and barks) of wisdom. It is an understatement to say that working with my supervisor, Assoc. Prof. Gary McPherson, has been an inspiring experience. I express my most genuine thanks for his tireless support, direction, encouragement and belief in my work. I cannot conceive how I might have completed this dissertation - vi - without him. Thanks are also due to Gary’s family for allowing me to take so much of his time and for allowing me to keep their phone engaged. Over 100 people took part in various pilots, preliminary investigations and main experiments as participants. Many of these people gave comments and advice freely and beyond my expectations, and most people gave over an hour of their time voluntarily. To all these people, I am grateful and regret the impracticality of listing all their names and contributions here. With the benefit of all these people, the job of making significant contributions to the research community has been an enlightening and most pleasurable experience. I hope that the readers of this dissertation are able to share in the depth of these experiences above and beyond the requirements of this dissertation as much as I did. Finally, I gratefully acknowledge the financial assistance of the Australian Postgraduate Award and the Faculty Resources Allocation Centre of the Faculty of Arts and Social Sciences at the University of New South Wales. - vii - Table of Contents Volume 1 Volume 1 i Abstract iii Acknowledgements v List of Figures xvii List of Tables xix Chapter 1 Introduction 1 Historical Overview..............................................................................................2 Musical Features..................................................................................................10 The Nature of Emotion.......................................................................................11 The Structure of Emotion....................................................................................16 Purpose of the Study...........................................................................................22 Plan of Methodology...........................................................................................23 Limitations of the Study......................................................................................24 Limitation 1: Romantic, Western Tonal Music............................................24 Limitation 2: Valence and Arousal of Emotion...........................................26 Limitation 3: Level of Explanation................................................................26 Limitation 4: Cognitivist Response...............................................................27 Limitation 5: Self-Report Measures...............................................................27 Limitation 6: Musicological Investigation....................................................30 Chapter 2 Measuring Emotions 32 Philosophical Preamble.......................................................................................33 Structure of the Review.......................................................................................33