Linköpings universitet/Linköping University | Department of Computer and Information Science Bachelor thesis, 18 hp | Cognitive Sciences Spring term 2021 | LIU-IDA/KOGVET-G--21/028--SE The effects of emotional prosody on perceived clarity in degraded speech Rasmus Lindqvist Supervisor: Carine Signoret Examinator: Michaela Socher Copyright The publishers will keep this document online on the Internet – or its possible replacement – for a period of 25 years starting from the date of publication barring exceptional circumstances. The online availability of the document implies permanent permission for anyone to read, to download, or to print out single copies for his/hers own use and to use it unchanged for non- commercial research and educational purpose. Subsequent transfers of copyright cannot revoke this permission. All other uses of the document are conditional upon the consent of the copyright owner. The publisher has taken technical and administrative measures to assure authenticity, security and accessibility. According to intellectual property law the author has the right to be mentioned when his/her work is accessed as described above and to be protected against infringement. For additional information about the Linköping University Electronic Press and its procedures for publication and for assurance of document integrity, please refer to its www home page: https://ep.liu.se/. © 2021 Rasmus Lindqvist ii Abstract The ability to hear is important to communicate with other people. People suffering from hearing loss are more likely to also suffer from loneliness and depression (Mener et al., 2013; Mo et al., 2005). To understand how degraded speech is recognized, the pop-out effect has been studied. The pop-out effect is the moment when a listener recognizes the meaning of degraded speech. Previous research on the pop-out effect in perception of speech has predominantly been focused towards top-down processes, such as form-based priming and semantic coherence in sentences. The purpose of this study was to research the relationship between emotional prosody and the perception of speech in varying levels of degraded speech. The participants were presented sentences with angry, neutral or happy prosody in varying levels of noise vocoding. The participants were then asked to rate the perceived amount of noise for each sentence, and if the prosody was perceived as positive, neutral or negative for each sentence. The results suggest that the participants' ability to perceive positive prosody in the sentences decreased more rapidly than negative as the amount of noise increased. The result did not show any statistically significant evidence that emotional prosody had any effect on the perceived amount of noise. Future research should further investigate emotional prosody together with emotional semantics, as an emotionally coherent spoken sentence, and the influences on speech perception in adverse listening conditions, in order to further investigate the factors contributing to the pop-out effect. Keywords: Emotional prosody, Speech perception, Degraded speech, Noise vocoding, Pop-out effect iii Acknowledgement First and foremost, I want to express my gratitude to Carine Signoret for taking the role as my supervisor. Thank you for your insightful ideas, feedback, and for guiding my back on track all those times where I felt lost and confused. I would also like to say a special thank you to Mattias Ekberg for providing me with the recorded audio material used in this study, and for sharing valuable and relevant articles of the field. Lastly, I would like to thank my friends and family for supporting me and helping me throughout the whole process. Linköping in June 2021 Rasmus Lindqvist v Table of Contents Copyright .......................................................................................................................... ii 1. Introduction ..................................................................................................................... 1 2. Theory ............................................................................................................................... 3 2.1 Pop-out effect ........................................................................................................... 3 2.2 Emotions and emotional categories ......................................................................... 4 2.3 Emotional prosody ................................................................................................... 4 2.4 Noise vocoding......................................................................................................... 5 2.5 Emotions and perception of speech .......................................................................... 6 2.6 About this study ....................................................................................................... 6 3. Method .............................................................................................................................. 7 3.1 Participants ............................................................................................................... 7 3.2 Material .................................................................................................................... 7 3.2.1 HINT ............................................................................................................... 7 3.2.2 Emotional Prosody material ........................................................................... 8 3.2.3 Test .................................................................................................................. 8 3.2.4 Task ................................................................................................................. 9 3.2.5 Procedure ....................................................................................................... 9 3.3 Data analysis ............................................................................................................ 9 4. Result .............................................................................................................................. 11 4.1 Ability to perceive emotional prosody ................................................................... 11 4.2 Perceived amount of noise ..................................................................................... 11 5. Discussion ....................................................................................................................... 13 5.1 Result discussion .................................................................................................... 13 5.1.1 Ability to perceive emotional prosody .......................................................... 13 5.1.2 Perceived amount of noise ............................................................................ 14 5.2 Implications ............................................................................................................ 15 5.3 Method discussion .................................................................................................. 15 5.4 Ethics ...................................................................................................................... 17 5.5 Conslusion and Future............................................................................................ 17 References ................................................................................................................................ 18 Appendix .................................................................................................................................. 23 A.1 List of the Swedish version of HINT sentences ..................................................... 23 A.2 Consent form .......................................................................................................... 23 A.3 Test 24 A.4 Descriptive data for Ability to perceive emotional prosody .................................. 29 A.5 Durbin-Conover for Ability to perceive emotional prosody .................................. 30 A.6 Descriptive data for Amount of perceived noise .................................................... 33 vii List of Abbreviations Abbreviation Meaning ERP Event related potential NV Noise vocoding STG Superior temporal gyrus STS Superior temporal sulcus RMS Root mean square viii 1. Introduction Our ability to hear is one of our most important senses. Auditory perception allows us to relate to the world in a variety of very significant purposes, and has played an integral role in the survival of our species (Heffner & Heffner, 1992). The ability to perceive sounds and to locate those sounds has enabled us to either approach or evade other animals, and to direct the attention of our other senses towards other valued sources of sound. However, one of the most important aspects of hearing is that hearing enables us to communicate and connect with other people in a way that the other senses can not. As said by the american author and activist, Helen Keller, “Blindness cuts us off from things, but deafness cuts us off from people”. Our ability to communicate with others is highly reliant on the ability to perceive speech. Speech is among the most complex sounds that we have to perceive, and therefore, our ability to perceive speech is fragil to hearing loss, lack of contextual awareness and background noise. Many of the patients seen by hearing healthcare providers seem to suffer from anxiety in some way (Carmen & Uram, 2002).