A Cross-Cultural Investigation of the Vocal Correlates of Emotion
Total Page:16
File Type:pdf, Size:1020Kb
A cross-cultural investigation of the vocal correlates of emotion Alison A. Tickle A thesis submitted in fulfillment of the requirements for the degree of Doctor of Philosophy School of Education, Communication and Language Sciences Newcastle University March 2015 Declaration of originality The material presented in this thesis is the original work of the candidate except as otherwise acknowledged. It has not been submitted previously in part or whole, for any award, at any university, at any other time. This copy has been supplied on the understanding that it is copyright material and use thereof requires proper acknowledgement. The right of Alison A. Tickle to be identified as author of this work has been asserted by her in accordance with the Copyright, Designs and Patents Act 1988. ii Abstract Universal and culture-specific properties of the vocal communication of human emotion are investigated in this balanced study focussing on encoding and decoding of Happy, Sad, Angry, Fearful and Calm by English and Japanese participants (eight female encoders for each culture, and eight female and eight male decoders for each culture). Previous methodologies and findings are compared. This investigation is novel in the design of symmetrical procedures to facilitate cross-cultural comparison of results of decoding tests and acoustic analysis; a simulation/self-induction method was used in which participants from both cultures produced, as far as possible, the same pseudo- utterances. All emotions were distinguished beyond chance irrespective of culture, except for Japanese participants’ decoding of English Fearful, which was decoded at a level borderline with chance. Angry and Sad were well-recognised, both in-group and cross- culturally and Happy was identified well in-group. Confusions between emotions tended to follow dimensional lines of arousal or valence. Acoustic analysis found significant distinctions between all emotions for each culture, except between the two low arousal emotions Sad and Calm. Evidence of ‘In-Group Advantage’ was found for English decoding of Happy, Fearful and Calm and for Japanese decoding of Happy; there is support for previous evidence of East/West cultural differences in display rules. A novel concept is suggested for the finding that Japanese decoders identified Happy, Sad and Angry more reliably from English than from Japanese expressions. Whilst duration, fundamental frequency and intensity all contributed to distinctions between emotions for English, only measures of fundamental frequency were found to significantly distinguish emotions in Japanese. Acoustic cues tended to be less salient in Japanese than in English when compared to expected cues for high and low arousal emotions. In addition, new evidence was found of cross-cultural influence of vowel quality upon emotion recognition. ii Early versions of a portion of this work were presented at conferences and published in proceedings: Tickle, A., (1999). Cross-language vocalisation of emotion: methodological issues. In: J.J. Ohala, Y, Hasegawa, M. Ohala, D. Granville, & A.C. Bailey (Eds.) Proceedings of 14th International Congress of Phonetic Sciences (ICPhS), August 1-8, 1999, San Francisco, USA, 305-308. Tickle, A. (2000). English and Japanese speakers’ emotion vocalisation and recognition: A comparison highlighting vowel quality. In: R. Cowie, E. Douglas-Cowie, & M. Schröder (Eds.), Proceedings of ISCA Workshop on Speech and Emotion. September 5- 7, 2000, Newcastle, Northern Ireland: International Speech Communication Association, 104-109. iii To my parents, and to Nana iv Acknowledgements My thanks go to my supervisor, Gerry Docherty for all of his help, support and encouragement during the writing of this thesis. I should also like to thank Barry Heselwood of Leeds University for his support and great sense of humour and am grateful to Leendert Plug, also of Leeds University. This thesis would not have been possible without the kind voluntary participation of Japanese and English people who gave up their time to take part in the experiments conducted for this study. My gratitude also goes to all those at Newcastle University and INTO Newcastle University who have given me support in diverse ways over the years spent writing this thesis. Thanks to David Clark from Newcastle University Music Department for early conversations on the theme of this thesis. I am grateful to Simon Kometa for helpful advice on the statistical analysis in this thesis and to Doug Cudmore and Chris Letts for technical advice. I should also like to thank previous colleagues at Barcelona University for their encouragement when I first had the idea for this thesis and to Albert for the conversations over trifásicas on the Rambla. Thanks to the musicians who have helped me escape when I needed to and to return with inspiration. The reliable support of friends has been heart-warming. I am grateful to Lynne for being there over many years – and no, I doubt they never did go shopping! Thanks very much to Jadwiga Billewicz, Lynne Pickles, Jo Craggs, Joan Tickle and Dave Porthouse for their proof-reading assistance; it is very much appreciated. Any remaining errors are of course my own. There are bottles in the fridge and on the mantelpiece. It seems a long time since the early days of discussions around the area of emotion in speech, in a sushi bar and various other venues in San Francisco. However, those conversations provided invaluable lasting encouragement for the writing of this thesis. My appreciation goes to Sylvie Mozziconacci, Marc Schröder, Ellen Douglas Cowie, Roddy Cowie, Veronique Auberge, Louis ten Bosch and Olivier Piot, and to all those working within the area of emotion in speech with whom I have had conversations, including those at the distant ISCA Conference in Newcastle, Northern Ireland, where many ideas on emotion in speech began to germinate. v Special thanks go to my parents, Joan and Ron Tickle for their unwavering love and support. I am grateful to all of my family for their stalwart support: to Annie Reynolds, who was and is always there; and in no particular order, to May and Roy for their considerable practical and moral support; to Audrey and Jack for helping me to keep my feet on the ground; to Vince and Sarah for their support and their haven of escape on Lake Geneva; to Joe, Hannah, Jack and Sam (in order of age!) for their sense of fun and for helping me to retain a sane perspective whilst writing this thesis; to Alan, Thomas, Holly and Oscar for the early morning walks which geared me up for writing and to all of my friends and family for their many kinds of support, all invaluable and much appreciated. Thanks in particular to Dave Porthouse for his love, help and support. Whilst this is an objective study, as far as any human observer can make an objective study, I feel I have experienced all of the emotions investigated here during the course of completing this thesis and many others besides. A conglomeration of inspiration has contributed to the writing of this thesis. Thanks to all those who were there. vi As soon as human beings began to make systematic observations about one another’s languages, they were probably impressed by the paradox that all languages are in some fundamental sense one and the same, and yet they are also strikingly different from one another. Ferguson, C. A. (1978). Historical background of universals research. In J.H. Greenberg, C.A. Ferguson, E.A. Moravcsik (Eds.) Universals of human language, Vol. One: Method and theory, 7–33. Stanford, CA: Stanford University Press. p.9. vii Contents Page Abstract ii Acknowledgements v Contents viii List of Tables xii List of Figures xiii Introduction 1.1 Background and aims 1 1.2 Structure 6 Chapter One. The nature of emotion - theoretical overview 1.1 Introduction 8 1.2 Universal versus social constructivist theories of emotion 9 1.3 Basic emotions 11 1.4 Innate and cultural effects upon the psycho-physiological processes influencing vocal correlates of emotion 16 1.5 Induction of emotion 19 1.5.1 The facial feedback hypothesis 19 1.5.2 Theory of emotional contagion 21 1.6 The Brunswikian Lens Model applied to vocal correlates of emotion 22 1.7 Summary 24 Chapter Two. The vocal correlates of emotion – previous studies 2.1 Introduction 26 2.2 Vocal cues of basic emotions 28 2.2.1 Cross-cultural similarities and differences in vocal cues of Happy 37 2.2.2 Cross-cultural similarities and differences in vocal cues of Sad 39 2.2.3 Cross-cultural similarities and differences in vocal cues of Angry 40 2.2.4 Cross-cultural similarities and differences in vocal cues of Fearful 42 2.2.5 Vocal cues of Calm 43 2.3 Summary 43 Chapter Three. Encoding and decoding the vocal correlates of emotion – previous studies 3.1 Introduction 46 3.2 Language as an indicator of cultural differentiation 48 3.3 Variety of cultures studied 49 3.4 Number of encoders/decoders 50 3.5 Balance and symmetry 52 3.6 Classification of emotion 56 3.6.1 Cultural bias of emotions studied and emotion labels 59 3.7 Methods used to encode vocal emotion 59 viii 3.7.1 Ecologically valid data 60 3.7.2 Induction and self-induction 61 3.7.3 Simulation by human portrayal 63 3.7.4 Synthesis and computer manipulation 66 3.7.5 Multimodal studies 67 3.8 Isolating vocal cues from verbal cues 68 3.9 Findings of previous studies 70 3.9.1 Recognition accuracy 70 3.9.2 In-Group Advantage 71 3.9.3 The Cultural Proximity Hypothesis 74 3.9.4 Recognition rates for specific emotions 76 3.9.5 Verbal and vocal cues 78 3.9.6 Vowel quality 78 3.9.7 Utterance length 79 3.9.8 Gender and age 80 Chapter Four.