Download (2073Kb)
Total Page:16
File Type:pdf, Size:1020Kb
A Thesis Submitted for the Degree of PhD at the University of Warwick Permanent WRAP URL: http://wrap.warwick.ac.uk/109949 Copyright and reuse: This thesis is made available online and is protected by original copyright. Please scroll down to view the document itself. Please refer to the repository record for this item for information to help you to cite it. Our policy information is available from the repository home page. For more information, please contact the WRAP Team at: [email protected] warwick.ac.uk/lib-publications Macro-Micro Approach for Mining Public Sociopolitical Opinion from Social Media by Bo Wang Thesis Submitted to the University of Warwick to obtain the degree of Doctor of Philosophy Department of Computer Science October 2017 This thesis is dedicated to my parents, Jianbao Wang and Xia Liu. Acknowledgments First and foremost, I would like to express my sincere gratitude to my supervisors, Dr. Maria Liakata and Prof. Rob Procter. Many thanks to their invaluable guidance and encouragement for guiding me to tackle all the research challenges during my PhD. I could not have asked for better mentors. I would like to extend my gratitude to Arkaitz Zubiaga for his advice through- out my PhD. I am also grateful to have Dr. Theo Damoulas, Prof. Mike Joy, and Prof. Alexandra I. Cristea as my annual reviewers, for their valuable suggestions on my PhD progress. I would also like to thank my friends, colleagues at the department, fellow members of WarwickNLP, and comrades at CS229 for their support and friendship. I hope part of us never grows up. Furthermore, I am thankful to all my co-authors. I would like to thank the most important people in my life, my mother Xia Liu, and my father Jianbao Wang. I can never thank them enough for their love and support. I would also like to thank my grandparents and other members of my family. Finally, I would like to thank everyone who believed in me and saw something in me when I did not see myself. ii Declarations I hereby declare that the work presented in this thesis entitled Macro-Micro Ap- proach for Mining Public Sociopolitical Opinion from Social Media is an original work and has not been submitted to any college, university or any other academic institution for the purpose of obtaining an academic degree. iii Abstract During the past decade, we have witnessed the emergence of social media, which has prominence as a means for the general public to exchange opinions towards a broad range of topics. Furthermore, its social and temporal dimensions make it a rich resource for policy makers and organisations to understand public opinion. In this thesis, we present our research in understanding public opinion on Twitter along three dimensions: sentiment, topics and summary. In the first line of our work, we study how to classify public sentiment on Twitter. We focus on the task of multi-target-specific sentiment recognition on Twitter, and propose an approach which utilises the syntactic information from parse-tree in conjunction with the left-right context of the target. We show the state-of-the-art performance on two datasets including a multi-target Twitter cor- pus on UK elections which we make public available for the research community. Additionally we also conduct two preliminary studies including cross-domain emo- tion classification on discourse around arts and cultural experiences, and social spam detection to improve the signal-to-noise ratio of our sentiment corpus. Our second line of work focuses on automatic topical clustering of tweets. Our aim is to group tweets into a number of clusters, with each cluster representing a meaningful topic, story, event or a reason behind a particular choice of senti- ment. We explore various ways of tackling this challenge and propose a two-stage hierarchical topic modelling system that is efficient and effective in achieving our goal. iv Lastly, for our third line of work, we study the task of summarising tweets on common topics, with the goal to provide informative summaries for real-world events/stories or explanation underlying the sentiment expressed towards an is- sue/entity. As most existing tweet summarisation approaches rely on extractive methods, we propose to apply state-of-the-art neural abstractive summarisation model for tweets. We also tackle the challenge of cross-medium supervised sum- marisation with no target-medium training resources. To the best of our knowledge, there is no existing work on studying neural abstractive summarisation on tweets. In addition, we present a system for providing interactive visualisation of topic-entity sentiments and the corresponding summaries in chronological order. Throughout our work presented in this thesis, we conduct experiments to evaluate and verify the effectiveness of our proposed models, comparing to relevant baseline methods. Most of our evaluations are quantitative, however, we do per- form qualitative analyses where it is appropriate. This thesis provides insights and findings that can be used for better understanding public opinion in social media. v Contents Acknowledgments ii Declarations iii Abstract iv Abbreviations xi List of Tables xiii List of Figures 1 Chapter 1 Introduction 1 1.1 Research Outline and Questions . .4 1.2 Main Contributions . .7 1.3 Publications . .8 Chapter 2 Background 11 2.1 Spam Detection on Twitter . 12 2.1.1 Social Spammer Detection . 12 2.1.2 Social Spam Detection . 13 2.2 Sentiment Analysis . 15 2.2.1 Sentiment Analysis on Social Media . 15 2.2.2 Cross-domain Sentiment Classification . 23 vi 2.2.3 Target-dependent Sentiment Recognition . 26 2.2.4 Aspect-level Sentiment Classification . 28 2.3 Tweet Clustering . 29 2.3.1 Document-Pivot Methods . 29 2.3.2 Term-Pivot Methods . 31 2.3.3 Evaluation of Topic Models . 38 2.4 Opinion Summarisation . 40 2.4.1 Extractive Summarisation . 42 2.4.2 Abstractive Summarisation . 43 2.4.3 Tweets Summarisation . 47 Chapter 3 Preliminary studies: Twitter social spam detection and cross-domain emotion analysis 51 3.1 Social Spam Detection . 52 3.1.1 Introduction . 52 3.1.2 Datasets . 54 3.1.3 Features . 55 3.1.4 Selection of Classifier . 58 3.1.5 Evaluation of Features . 59 3.1.6 Discussion and Conclusion . 60 3.2 Twitter Emotion Analysis . 63 3.2.1 Introduction . 63 3.2.2 Datasets . 64 3.2.3 Methodology . 67 3.2.4 Results and Evaluation . 72 3.2.5 Conclusion . 77 Chapter 4 Target-specific Sentiment Recognition: C lassifying senti- ment towards multiple targets in a tweet 79 vii 4.1 Single-target-specific Sentiment Recognition using Graph Kernel . 81 4.1.1 Target Relevance Through Syntactic Relations . 81 4.1.2 Generating Per-Token Annotations . 81 4.1.3 Classification Without Dependency Relations . 82 4.1.4 Using Dependency Relations . 82 4.1.5 Discussion . 83 4.2 Multi-target-specific Sentiment Classification . 84 4.3 Creating a Corpus for Multi-target-specific Sentiment in Twitter . 85 4.3.1 Data Harvesting and Entity Recognition . 85 4.3.2 Manual Annotation of Target Specific Sentiment . 87 4.4 Developing a state-of-the-art approach for target-specific sentiment . 89 4.4.1 Model development for single-target benchmarking data . 89 4.4.2 Experimental Settings . 93 4.4.3 Experimental results and comparison with other baselines . 94 4.5 Evaluation for target-specific sentiment in a multi-target setting . 96 4.5.1 State-of-the-art tweet level sentiment vs target-specific senti- ment in a multi-target setting . 100 4.6 Discussion and Conclusion . 100 Chapter 5 Topical Clustering of Tweets: A hierarchical topic mod- elling approach 103 5.1 Introduction . 103 5.2 Methodology . 105 5.3 Datasets . 107 5.4 Evaluation . 108 5.4.1 Experimental setup . 108 5.4.2 Tweet Clustering Evaluation . 111 5.4.3 Topic Coherence Evaluation . 113 viii 5.4.4 Qualitative Evaluation of Topics . 117 5.5 Conclusions and Future Work . 118 Chapter 6 Twitter Opinion Summarisation: Towards neural abstrac- tive summarisation of tweets 121 6.1 Topic-based, Temporal Sentiment Summarisation for Twitter . 122 6.1.1 System Design . 123 6.1.2 Data Visualisation . 125 6.1.3 Use Case #1 { Party Sentiment . 126 6.1.4 Use Case #2 { Grenfell Tower Fire . 127 6.1.5 Conclusion . 128 6.2 Neural Abstractive Multi-tweet Opinion Summarisation . 130 6.2.1 Problem Formulation . 130 6.2.2 Sequence-to-Sequence Attentional Model . 131 6.2.3 Extractive-Abstractive Summarisation Framework . 131 6.2.4 Pointer-Generator Network for Abstractive Summarisation . 132 6.2.5 Unsupervised Pretraining for Model Initialisation . 133 6.3 Experiments and Results . 134 6.3.1 Datasets . 134 6.3.2 Automatic Summary Evaluation Metrics . 136 6.3.3 Experimental Setup . 138 6.3.4 Results for Event Summarisation . 140 6.3.5 Results for Opinion Summarisation . 143 6.4 Conclusions and Further Work . 144 Chapter 7 Conclusions 147 7.1 Main Findings . 148 7.2 Future Directions . 151 7.2.1 Multi-target-specific Sentiment Classification . 151 ix 7.2.2 Topical Clustering of Tweets . 152 7.2.3 Abstractive Opinion Summarisation on Twitter . 152 Appendix A Seeding Keywords for Twitter Data Collection 154 A.0.1 Seeding Hashtags Using Association Rule Learning . 155 A.0.2 Use Case . 157 x Abbreviations AP Affinity Propagation AdaRNN Adaptive Recurrent Neural Network CNN Convolutional Neural Network DMM Dirichlet Multinomial Mixture ED Event Detection FSD First Story Detection G3 Three-way Gate GRNN Gated Recurrent Neural Network GSDMM Gibbs Sampling algorithm for the Dirichlet Multinomial Mixture HC Hierarchical Clustering ILP Integer Linear Programming KLD Kullback-Leibler Divergence LCTM Latent Concept Topic Model LFTM Latent Feature Topic Models LM Language Model LSTM Long Short Term Memory xi LDA Latent Dirichlet Allocation NPMI Normalised Pointwise Mutual Information OLDA Online Latent Dirichlet Allocation OOV Out of Vocabulary PMI Pointwise Mutual Information RNN Recurrent Neural Network SMS Short Message Service SSWE Sentiment-Specific Word Embedding SVM Support Vector Machine TOLDA Online LDA for Twitter Tf-idf Term frequency-inverse document frequency TCLSTM Target Connection Long Short Term Memory TDLSTM Target Dependent Long Short Term Memory WE Word Embedding WMD Word Mover's Distance xii List of Tables 3.1 Examples of spam tweets .