A Dyadic Conversation Dataset on Moral Emotions
Total Page:16
File Type:pdf, Size:1020Kb
A dyadic conversation dataset on moral emotions Citation for published version (APA): Heron, L., Kim, J., Lee, M., El Haddad, K., Dupont, S., Dutoit, T., & Truong, K. (2018). A dyadic conversation dataset on moral emotions. In 13th IEEE International Conference on Automatic Face and Gesture Recognition (pp. 687-691). Institute of Electrical and Electronics Engineers. https://doi.org/10.1109/FG.2018.00108 DOI: 10.1109/FG.2018.00108 Document status and date: Published: 05/06/2018 Please check the document version of this publication: • A submitted manuscript is the version of the article upon submission and before peer-review. There can be important differences between the submitted version and the official published version of record. People interested in the research are advised to contact the author for the final version of the publication, or visit the DOI to the publisher's website. • The final author version and the galley proof are versions of the publication after peer review. • The final published version features the final layout of the paper including the volume, issue and page numbers. Link to publication General rights Copyright and moral rights for the publications made accessible in the public portal are retained by the authors and/or other copyright owners and it is a condition of accessing publications that users recognise and abide by the legal requirements associated with these rights. • Users may download and print one copy of any publication from the public portal for the purpose of private study or research. • You may not further distribute the material or use it for any profit-making activity or commercial gain • You may freely distribute the URL identifying the publication in the public portal. If the publication is distributed under the terms of Article 25fa of the Dutch Copyright Act, indicated by the “Taverne” license above, please follow below link for the End User Agreement: www.tue.nl/taverne Take down policy If you believe that this document breaches copyright please contact us at: [email protected] providing details and we will investigate your claim. Download date: 29. Sep. 2021 See discussions, stats, and author profiles for this publication at: https://www.researchgate.net/publication/325635344 A Dyadic Conversation Dataset on Moral Emotions Conference Paper · May 2018 DOI: 10.1109/FG.2018.00108 CITATIONS 0 7 authors, including: Jaebok Kim Minha Lee University of Twente Technische Universiteit Eindhoven 25 PUBLICATIONS 76 CITATIONS 6 PUBLICATIONS 1 CITATION SEE PROFILE SEE PROFILE Kevin El Haddad Stéphane Dupont Université de Mons Université de Mons 26 PUBLICATIONS 64 CITATIONS 159 PUBLICATIONS 2,134 CITATIONS SEE PROFILE SEE PROFILE Some of the authors of this publication are also working on these related projects: Interactive Inflatables View project Biomanufacturing View project All content following this page was uploaded by Kevin El Haddad on 11 June 2018. The user has requested enhancement of the downloaded file. A Dyadic Conversation Dataset On Moral Emotions Louise Heron* Jaebok Kim* Minha Lee* Kevin El Haddad* University of Bath Human Media Interaction group Human-Technology Interaction group University of Mons Bath, UK University of Twente Technical University of Eindhoven Mons, Belgium [email protected] Enschede, Netherlands Eindhoven, Netherlands [email protected] [email protected] [email protected] Stephane´ Dupont Thierry Dutoit Khiet Truong University of Mons University of Mons Human Media Interaction group Mons, Belgium Mons, Belgium University of Twente [email protected] [email protected] Enschede, Netherlands [email protected] Abstract—In this paper, we present a dyadic conversation a conversation, it is harder to draw a clear line between dataset involving topics related to moral emotions which are “speakers” and “listeners”, since a “listener” could utter ethically relevant. To the best of our knowledge, it is the verbal or nonverbal sounds as feedback to the “speaker” and first dataset where the main focus is moral emotions. This dataset also focuses on speaker-listener reactions during a the speaker could be interrupted verbally by the “listener”. dyadic conversation. Although some of the currently available This is why we decided to build our recording scenario by datasets contain dyadic conversations, they were not conceived assigning clearly the “speaker” and “listener” roles to the with the idea of focusing on the speaker-listener setup. Thus subjects while keeping the interaction naturalistic. making it difficult to use them to study reactions related to The second purpose of the dataset was to look into moral speakers and listeners. Some preliminary analyses of the data are presented as well as our thoughts on future work related emotions, which are emotions that are ethically relevant to this dataset. [3]. The emotional databases currently available contain categorical annotations of mainly the six basic emotions Keywords-Moral emotions; Dataset; Dyadic interaction; Non- verbal expressions; Multimodal data; Affective computing [4], or with codes largely based on the Circumplex Model of emotions [5]. To add diversity to emotion research, we focused on moral emotions. The eNTERFACE workshop 1 I. INTRODUCTION gave us the opportunity to collect our own dyadic conver- Human-agent interactions (HAI) systems are emerging in sation database [6]. To build this database, participants took our daily lives. It is important that these agents learn not turns as speakers and listeners, the latter asking questions to only our verbal, but also nonverbal way of communicating. the former about memories of emotional states. Questions This would allow them to better “understand” underlying were on negative (i.e. guilt and shame) and positive (i.e. messages and make it more comfortable to interact with pride and compassion) emotions, Which are considered to them since they would have more human-like behavior. be moral emotions [3]. In the following subsections, the data In this paper, we present a corpus of spontaneous dyadic recording setup will be described in Section II. We will then conversations with a rich amount of verbal and nonverbal go through the dataset content in Section III and our first expressions. Our contributions are two-fold: analyses will be presented in Section IV. We will finally 1) focusing on listener-speaker reactions during a dyadic present conclusions in Section V. conversation II. SETUP AND EXPERIENCE DESCRIPTION 2) obtaining data related to moral emotions A. Procedure Indeed, it is important for the agent to have the proper attitude as a speaker, but also as an attentive listener. Participants were firstly asked to read the informed con- The current dyadic interaction datasets such as: the Cardiff sent form, which stated that their participation was voluntary Conversation Database (CCDB), IFA Dialog Video corpus and unpaid. The consent form stated that moral emotions (IFADV) [1] and IEMOCAP [2] do not explicitly focus on will be discussed by the participants. At the start of the the separation between speaker and listener. Although the experiment, participants were told that they will randomly be roles of “speaker” and “listener” are implicitly present in assigned to the role of speaker or listener, then switch roles. 1 *Authors with equal contribution eNTERFACE 2017 took place at the Centre of Digital Creativity (CCD), Escola das Artes, Universidade Catolica Portuguesa, in Porto, Portugal. More information is available at http://artes.ucp.pt/enterface17/. 978-1-5386-2335-0/18/$31.00 c 2018 IEEE Figure 1. Pictorial depiction of the experimental setup from (A) side and (B) aerial views. Figure 2. (A) A side on view of the experimental setup from a third party observer and (B) the setup from the participants perspective. The speaker answered questions about moral emotions, whilst the listener listened to the speakers answers and asked the videos and corresponding audio files as well as the any follow up questions as necessary. The instructions were interlocutors’ data with each other. So far, most of the purposively vague to ensure that the dyadic interaction was data has been annotated and separated in topics (gratitude, as natural as possible. Indeed, the listener was free to discuss compassion, guilt and shame) and with respect to the role and/or interrupt the speaker just like in any normal conversa- of the subject visible in the video (speaker or listener). tion. The order in which participants discussed the emotions We are currently working on segmenting and annotating was randomised: listeners chose one of the two moral the dataset in more detail. Indeed, as mentioned earlier, emotion options (positive, negative) from question prompts although the roles were split into speakers and listeners, on a table. Each interaction started with the listener asking a we did not prevent the listener from responding. Thus, question that varied in the emotion category: When was the segmenting the dataset into speakers and roles in each of last time you experienced gratitude/compassion/guilt/shame? the separated audio/video files will be our next goal. Other Can you describe the event and your feelings? The speaker annotations will be added in the near future to help us responded to each question. The interaction lasted until study these interactions better. First, non-linguistic nonverbal the interlocutors both indicated to experimenters that the conversation expressions such as laughs, smiles, eyebrow conversation was finished. movements and head movements will be annotated. This will help us model the nonverbal interactions between the B. Experimental Setup: Video/Audio Acquisition interlocutors. Then, the dialogs will be transcribed in order Video and audio were recorded in a soundproof room to obtain the semantic content related to the topics of moral at the Catholic University of Porto, Portugal. Two Canon emotions. Cameras: EOS 550D and EOS 6D were used to record the interactions. Camera A (beside Speaker 1, recorded Listener III. CONTENT 1/Speaker 2) and Camera B (beside Speaker 2, recorded This database was designed to mimic, as much as possible, Speaker 1/Listener 2).