A Bootstrapped Approach for Abusive Intent Detection in Social Media Content by Benjemin Simons A thesis submitted to the School of Computing in conformity with the requirements for the degree of Master of Science Queen's University Kingston, Ontario, Canada September 2020 Copyright © Benjemin Simons, 2020 Abstract The proliferation of Internet connected devices continues to result in the creation of massive collections of human generated content from websites such as social media. Unfortunately, some of these sites are used by criminal or terrorist organizations for recruitment or to spread rhetoric. By analyzing this content, it is possible to gain insights into the future actions of the writers. This information can support orga- nizations in taking proactive measures to modify or stop said actions from taking place. The textual feature of interest is the expression of abusive intent, which can be thought of as a plan to carry out a malicious action. The proposed approach independently detects abuse and intent in documents, then computes a joint predic- tion for the document. Abusive language detection is a well-studied problem, which enabled a model to be trained using supervised learning. The intent detection model requires a semi-supervised technique since no labelled datasets exist. To do this, an initial collection of labels was generated using a linguistic model. These labels were then used to co-train a statistical and deep learning model. Using crowd-sourced labels, the abuse and intent models were found to have accuracies of 95% and 80%, respectively. The joint predictions were then used to prioritize documents for manual assessment. i Acknowledgments I would like to thank my supervisor, Professor David Skillicorn for all his guidance and patience over the past year and a half. His advice was always available and invaluable to the completion of this research. Thank you to all the professors and students in the School of Computing who made my time memorable and helped me work through problems and bugs. Finally, I would like to thank my family and friends for their continuous support. Their encouragement and company enabled me to finish school successfully and enjoyably. ii Contents Abstract i Acknowledgments ii Contents iii List of Tables vi List of Figures viii Chapter 1: Introduction 1 Chapter 2: Related Work 5 2.1 Linguistics . 5 2.2 Social science . 8 2.3 Computing . 9 2.3.1 Intent detection . 10 2.3.2 Abusive language detection . 11 2.4 Tools and techniques . 12 2.4.1 Data handling and manipulation . 13 2.4.2 Word embeddings . 13 2.4.3 Deep learning . 15 2.4.4 Interpretation and visualization . 19 Chapter 3: Methodology 21 3.1 Data preparation . 22 3.1.1 Source selection and data collection . 23 3.1.2 Data cleaning . 23 3.1.3 Document partitioning . 25 3.1.4 Context-sequence matrix . 25 3.1.5 Word embeddings . 26 3.2 Label generation . 29 iii 3.2.1 Linguistic template . 31 3.2.2 Rough label generation . 34 3.2.3 Label refinement . 35 3.3 Extrapolation . 41 3.3.1 Rate limiter . 42 3.3.2 Sequence learner . 43 3.3.3 Deep learner . 45 3.3.4 Consensus . 47 3.4 Abusive language . 48 3.4.1 Training dataset . 48 3.4.2 Network architecture . 49 3.4.3 Model training . 50 3.5 Abusive intent . 51 3.5.1 Prediction generation . 51 3.5.2 Distribution normalization . 52 3.5.3 Vector norm . 54 3.5.4 Product . 57 3.6 Validation . 59 3.6.1 Data generation . 59 3.6.2 Label collection . 60 3.6.3 Label validation . 62 3.6.4 Prediction validation . 63 3.7 Document aggregation . 64 3.7.1 Average . 65 3.7.2 Maximum . 65 3.7.3 Windowed maximum . 66 Chapter 4: Results 67 4.1 Data preparation . 67 4.1.1 Data cleaning . 67 4.1.2 Document partitioning . 69 4.1.3 Context-sequence matrix . 72 4.1.4 Word embeddings . 73 4.2 Label generation . 74 4.2.1 Rough label generation . 74 4.2.2 Label refinement . 76 4.3 Extrapolation . 80 4.3.1 Sequence learner . 80 4.3.2 Deep learner . 82 4.3.3 Consensus . 83 iv 4.4 Abusive language . 84 4.4.1 Model training . 85 4.4.2 Abuse predictions . 86 4.5 Abusive intent . 89 4.5.1 Distribution normalization . 89 4.5.2 Vector norm . 90 4.5.3 Product . 94 4.5.4 Predictions . 100 4.6 Validation . 103 4.6.1 Volunteers . 105 4.6.2 Collected labels . 107 4.6.3 Prediction validation . 108 4.7 Document aggregation . 111 4.7.1 Averaged . 111 4.7.2 Maximum . 113 4.7.3 Windowed . 114 Chapter 5: Discussion 116 5.1 Limitations . 117 5.1.1 Imperfect initial labels . 117 5.1.2 Lack of support to detect implicit intent . 119 5.1.3 Adversarial input . 119 5.1.4 Accepting statements as truth . 120 5.1.5 Unrepresentative validation data . 120 5.1.6 Reliance and non-uniform definitions of abuse . 120 Bibliography 122 Appendix A: Datasets 134 A.1 Storm-Front (intent) . 134 A.2 Wikipedia (intent) . 134 A.3 Iron March . 135 A.4 Manifesto . 135 A.5 Hate speech ensemble . 135 Appendix B: Computational resources 137 Appendix C: Data labelling interface 138 C.1 Architecture . 138 C.2 Ethical clearance . 140 C.3 Labelling instructions . 140 v List of Tables 3.1 Table of dependencies defining short and long form intent templates . 33 3.2 Intent model architecture . 46 3.3 Abusive language dataset composition . 49 3.4 Abusive language model architecture . 50 3.5 Qualifying contexts . 60 4.1 Dataset character lengths before and after processing . 68 4.2 Words closest to \liberal" in custom and default fastText embeddings 75 4.3 15 sequences with the highest positive rates after 1 vs. 20 epochs . 83 4.4 Examples of contexts with high abusive intent when normalizing dis- tribution . 94 4.5 Examples of contexts with high abusive intent calculated using the infinite norm . 95 4.6 Examples of contexts with high abuse or intent identified using the infinite norm . 96 4.7 Examples of contexts with high abusive intent when using the one norm 97 4.8 Examples of contexts with high abusive intent when using the two norm 98 4.9 Examples of contexts with high abusive intent when using the product calculation . 99 vi 4.10 Examples of Storm-Front contexts with low abusive intent when using the product calculation . 101 4.11 Examples of contexts with high abusive intent from abusive training dataset . 104 4.12 Examples of contexts with high abusive intent from Manifesto . 105 4.13 Examples of contexts with high abusive intent from Iron March . 106 4.14 False negative examples . 110 4.15 False positive examples . 112 4.16 Aggregated document with high abusive intent using the windowed method . 114 4.17 Aggregated document with.