Faculteit Letteren & Wijsbegeerte Sarah Messiaen Automatic detection of crisis situations on social media: an analysis of the tweets posted after the crash of Germanwings Flight 9525 Masterproef voorgedragen tot het behalen van de graad van Master in de Meertalige Communicatie 2015 Promotor Prof. Dr. Véronique Hoste Vakgroep Vertalen Tolken Communicatie PREFACE First and foremost, I would like to show my sincerest gratitude to my supervisor Prof. dr. Véronique Hoste, who has guided me from the initial to the final level of this master’s dissertation. She was a valuable help to me and she always took her time to meet and review nearly anything. Furthermore, I thank Cynthia Van Hee for helping me to perform all experiments needed to complete this paper’s study. Third, I would like to thank dr. Klaar Vanopstal for correcting some parts of this master’s dissertation. I would also like to thank my boyfriend and my friends for their moral support. And finally, I owe my deepest gratitude to my parents for supporting my throughout my studies over the past few years. 3 TABLE OF CONTENTS LIST OF GRAPHS AND TABLES ........................................................................................... 5 ABSTRACT ............................................................................................................................... 7 1. INTRODUCTION .............................................................................................................. 7 2. THEORETICAL FRAMEWORK ..................................................................................... 9 2.1 CRISIS COMMUNICATION ..................................................................................... 9 2.1.1 Crisis defined ........................................................................................................ 9 2.1.2 The three-stage approach to crisis management .................................................. 9 2.1.2.1 Pre crisis ……………………………………………………………...10 2.1.2.2 Crisis event …………………………………………………………...10 2.1.2.3 Post crisis ……………………………………………………………..10 2.1.3 Stealing thunder .................................................................................................. 11 2.1.4 Coombs’ Situational Crisis Communication Theory ......................................... 11 2.1.4.1 SCCT's attribution theory roots ………………………………………12 2.1.4.2 Factors that shape the reputational threat …………………………….13 2.1.4.3 Crisis types …………………………………………………………...14 2.1.4.4 Crisis response strategies ……………………………………………..15 2.1.5 Emotions in crisis communication ..................................................................... 17 2.2 SOCIAL MEDIA ....................................................................................................... 18 2.2.1 Social media defined .......................................................................................... 18 2.2.2 The role of social media in organizational crisis communication ...................... 19 2.2.3 The use of Twitter in organizational crisis communication ............................... 20 2.2.3.1 The use of Twitter by corporations …………………………………..20 2.2.3.2 The use of Twitter by citizens ………………………………………..21 4 2.3 AUTOMATIC CLASSIFICATION OF SENTIMENT AND EMOTION IN CRISIS RELATED MICROPOSTS .................................................................................................. 22 2.3.1 Automatic classification of sentiment ................................................................ 22 2.3.2 Automatic classification of emotion .................................................................. 23 3. RESEARCH METHODOLOGY ..................................................................................... 25 3.1 EVENT OF STUDY .................................................................................................. 25 3.2 DATA COLLECTION .............................................................................................. 26 3.3 METHOD .................................................................................................................. 27 3.3.1 Sentiment detection ............................................................................................ 27 3.3.2 Emotion detection .............................................................................................. 28 4. RESULTS AND DISCUSSION ...................................................................................... 29 4.1 MOST FREQUENTLY USED WORDS AND ANALYSIS OF THE ORGANISATION’S TWEETS ........................................................................................... 29 4.2 SENTIMENT CLASSIFICATION ........................................................................... 32 4.2.1 Quantitative discussion of English results ......................................................... 33 4.2.2 Qualitative discussion of English results ........................................................... 34 4.2.3 Quantitative discussion of Dutch results ............................................................ 36 4.2.4 Qualitative discussion of Dutch results .............................................................. 37 4.3 EMOTION CLASSIFICATION ............................................................................... 40 4.3.1 Quantitative discussion of English results ......................................................... 40 4.3.2 Quantitative discussion of Dutch results ............................................................ 42 4.3.3 Qualitative discussion ........................................................................................ 43 5. CONCLUSION ................................................................................................................ 45 6. BIBLIOGRAPHIC REFERENCES ................................................................................. 47 7. APPENDIX ...................................................................................................................... 50 7.1 TWEETS POSTED BY THE ORGANIZATION ..................................................... 50 7.1.1 Tweets posted by Germanwings ........................................................................ 50 7.1.2 Tweets posted by Lufthansa ............................................................................... 51 7.2 CORPUS OF 200 ENGLISH TWEETS .................................................................... 52 7.3 CORPUS OF 200 DUTCH TWEETS ....................................................................... 62 5 LIST OF GRAPHS AND TABLES GRAPHS Graph 1: Word Cloud English tweets March 24-25 p. 29 Graph 2: Word Cloud English tweets March 26-31 p. 30 Graph 3: Word Cloud English tweets April 1-6 p. 30 Graph 4: Word Cloud Dutch tweets March 24-25 p. 31 Graph 5: Word Cloud Dutch March 26 – April 6 p. 31 TABLES Table 1: Crisis types definitions p. 15 Table 2: SCCT crisis response strategies p. 16 Table 3: Social media categories p. 18 Table 4: Class distribution of the English corpus p. 33 Table 5: System accuracy based on 200 gold annotations p. 33 Table 6: Class distribution based on 200 gold annotations p. 34 Table 7: Tweets that received the same label both manually and automatically p. 34 Table 8: Mistake type 1 p. 35 Table 9: Mistake type 2 p. 35 Table 10: Mistake type 3 p. 35 Table 11: Mistake type 4 p. 36 Table 12: Class distribution of the Dutch corpus p. 36 Table 13: System accuracy based on 200 gold annotations p. 37 Table 14: Class distribution based on 200 gold annotations p. 37 Table 15: Tweets that received the same label both manually and automatically p. 38 Table 16: Mistake type 1 p. 38 Table 17: Mistake type 2 p. 39 6 Table 18: Mistake type 3 p. 39 Table 19: Mistake type 4 p. 39 Table 20: Occurrence emotion classes in English gold standard corpus p. 41 Table 21: Examples of English tweets expressing emotion p. 41 Table 22: Occurrence emotion classes in Dutch gold standard corpus p. 42 Table 23: Examples of Dutch tweets expressing emotion p. 43 Table 24: Elements that could help to determine emotional content p. 44 7 ABSTRACT Social media, and in particular Twitter, are increasingly being utilized during crises. It has been shown that tweets offer valuable real-time information for decision-making. However, because of the vast amount of information, this information is not directly usable. Applying sentiment and emotion analysis is one option for distinguishing important information from unimportant information. In this study, we investigate the feasibility of automatically detecting crisis-related emotions on Twitter. Therefore, two corpora of respectively English and Dutch tweets posted after the crash of Germanwings Flight 9525 were built. For each corpus, 200 tweets were manually and automatically labelled as positive, negative or neutral. Moreover, we manually tagged the emotional content of these tweets. We found that the system used for the sentiment analysis of the English tweets performed better than the system used for the sentiment analysis of the Dutch tweets. Both systems performed well with regard to the negative class label, which is important in crisis situations. Furthermore, sympathy and anger were the most recurring emotions in the English gold standard corpus, whereas sadness and contempt were the most frequently expressed emotions in the Dutch gold standard corpus. It can be concluded that automatic systems can help to enable the automatic detection of emotions: first through sentiment analysis and then through a more fine-grained detection of emotions. 1. INTRODUCTION The