Final Project Write-Up for CSCI 534: Affective Computing “Developing
Total Page:16
File Type:pdf, Size:1020Kb
Final Project Write-Up for CSCI 534: Affective Computing “Developing Group Emotion-Recognition Software to Investigate Group vs. Individual Emotion” - ChengYuan Cheng, Drishti Saxena, Prakhar Deep, Ritesh Sinha University of Southern California 5th Dec, 2018 1 TABLE OF CONTENTS I. Introduction…………………………………………………………… 2 II. Related Approaches………………………………………………….. 2 III. Theoretical Perspectives……………………………………………… 3 IV. Developing the Software………………………………………………3 V. Empirical Test: The Experiment…………………………………….... 6 VI. Interesting Insights………………………………………………….... 12 VII. Next Steps, Implications & Conclusions………………………………13 VIII. Division of Labor………………………………………………………14 IX. References…………………………………………………………….. 15 X. Appendix……………………………………………………………… 16 2 I. Introduction Area Of Interest: Emotion recognition is currently one of the most researched areas in artificial intelligence. We aimed to build software to detect a person’s emotion based on facial expressions and then to use it to explore use-case “How group effect individual’s emotion”. Our general area of interest was emotion recognition and to use results to test our hypothesis that group elevates the individual’s emotional response. Problem Statement: To develop emotion recognition software for the comparative study of a person’s emotions when seeing some stimuli individually or in a group. “Individual vs Groups” – as there’s a saying that misery loves company, we tried to check our hypothesis that person’s emotion gets heightened if exposed to similar stimuli in a group as compared to that of in solitude. We build emotion recognition software to detect group/individual response from recorded video. We then showed similar intensity videos to both individual and group and checked the intensity of their emotions. II. Related Research and Approaches: ● In 1991, Alan J. Fridlund conducted a similar experiment with 64 people to check how emotions vary when a person is alone or when a person is in a group. He explored four areas: when the candidate is (a) alone, (b) alone but with the belief that a friend nearby was otherwise engaged, (c) alone but with the belief that a friend was viewing the same videotape in another room, or (d) when a friend was present. He used electrodes to measure facial muscle movement to detect the emotion.[1] ● The Ripple Effect: Emotional Contagion and Its Influence on Group Behavior by Sigal G. Barsade University of Pennsylvania. In 2002, they performed the experiment on how one person can influence the group towards the targeted emotion. They conducted their experiment on 98 MBA students divided into a group of four. They placed a confederate in each group, who will lead discussion towards a targeted emotion. The analysis was made using self-reports, questionnaires filled before and after the experiment, and by observing the recorded video.[2] ● Individual and group-level factors for students' emotion management in online collaborative group work - Jianxia Du, Chuang Wang, Mingming Zhou, Jianzhong Xu, Xitao Fan & Saosan Lei (2017) – They examined the group trust and dynamics of people in online collaborative group work. Four hundred eleven university students from 103 groups in the United States responded to survey items on online collaboration, interactivity, communication media, and group trust. They used group chat and survey form for building results. Results revealed that trust among group members had a positive and powerful influence on online collaboration.[3] 3 III. Theoretical Perspectives Our endeavor to investigate the difference in emotional experiences when responding to similar stimuli individually and in groups is not new. Historically, the expression of emotion is a primarily personal experience and the individual phenomenon has been contested by supporters of the Behavioral Ecology view of expressions such as Alan Fridlund. This view was supported by considerable amounts of research. In a study by Fridlund (1991), it was demonstrated that when made to watch pleasant/ funny videos, participants’ smiling systematically increased as the “sociality” or extent to which participants believed they had company increased, as measured by facial electromyography. However, the sociality had no such effect on self-reported emotion. This finding thus corroborated Fridlund’s view that facial expressions are weakly connected to the true emotion felt and are not only shaped by the social context but also serve primarily as social information that is intended to be communicated implicitly or explicitly to another person. As discussed in the lecture to the Affective Computing class on Oct 17, 2018, by Professor Gratch, there is thus an evident divide between such a view and Paul Ekman’s Basic Emotion Theory view which strongly contends that emotions may be masked by purposeful means, yet true emotion is reflected in one’s facial expressions and micro-expressions. His theory draws support from empirical evidence that showed the universality of emotion across cultures (e.g. Ekman (1972)) and that voluntarily making an expression can lead to the subjective experience of an emotion (Ekman, Levenson & Friesen, 1983). An alternative “Emotional Contagion” view states that the emotional experience of one individual can be mutually shared and distributed across the members of a group, which may lead the group to amplify the intensity of emotion felt in contrast to the experience they may have had alone (Barsade, 2002). Yet another perspective comes from a fairly recent study by Shteynberg et al. (2014) which corroborated the finding that it may actually be mere group attention as opposed to emotional contagion that may intensify the experience of an emotional experience in response to a specific stimulus. The study found that group attention augmented the intensity of fear and happiness felt when watching scary and positive advertisements respectively. This suggests that the mere collective experience of attending to a stimulus elicits intensified effect in groups. Such contrasting views on the affective experience of stimuli in groups led us to investigate it further by employing an automated recognition approach. We hoped to gain deeper insight by using advanced recognition techniques to uncover more sophisticated answers to the questions that such theoretical perspectives pose. Next, we explain how the software used in the experiment was developed. IV. Developing the Software ● Emotion Recognition Microsoft provides Emotion APIs which takes video or image as an input and returns the confidence scores across a set of emotion for the faces detected in the frame. It can detect 8 different emotions and follows Ekman’s basic emotion model - i.e. happiness, sadness, surprise, anger, fear. Contempt, disgust or neutral. 4 ● Subscriptions We need to get API keys for the Vision APIs. It is needed in order to integrate our code with the emotion APIs. For video frame analysis, the applicable APIs are: • Computer Vision API • Emotion API • Face API ● Interpreting Results The result returned by the Emotion API is in the JSON structure. For interpreting results from the Emotion API, the emotion detected should be interpreted as the emotion with the highest score, as scores are normalized to sum to one. ● Analyzing Videos We performed analysis on frames taken from a video stream. The basic components in the system are: 5 • Acquire frames from a video source • Select which frames to analyze • Submit these frames to the API • Consume each analysis result that is returned from the API call ● A Producer-Consumer Design We used "producer-consumer" system as suggested by the Microsoft Emotion documentation, we have a producer thread that puts the tasks (consuming analysis results) into a queue to keep track of them. We have created consumer thread, which takes tasks off the queue and waits for them to finish. It can either display the result or raise the exception that was thrown. Usage of queue ensures that results get consumed one at a time, and in the correct order, without limiting the maximum frame-rate of the system. 6 V. Empirical Test: The Experiment Having developed the software, we then designed our experiment to gain further insights into our research question: How can our understanding of affective consequences improve by 7 using emotion-recognition software to study the emotional expression of individuals alone and in groups when they attend to emotionally evocative stimuli? Based on the discrepant research findings, we decided to conduct an exploratory study through a structured experiment with the following two-tailed (bidirectional) hypotheses: ● Hypothesis H1: The fear-output scores from software analysis would be significantly different when participants watch the scary video in a group of peers than when they watch it alone. ● Hypothesis H2: The happiness-output scores from software analysis would be significantly different when participants watch the funny video in a group of peers than when they watch it alone. Independent Variable: The presence of peers in the room as manipulated by leaving participants alone in the room in the individual condition or with 3 other peers in group condition. Dependent Variable: The fear-emotion output from the emotion-recognition software was considered for the scary video analyses and the happiness-emotion output was considered for the funny video analyses. Method ● Participants: The participants were sampled using convenience sampling, our friends and classmates formed an easily available pool of participants