DOCSLIB.ORG

Modeling Text Embedded Information Cascades

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Modeling Text Embedded Information Cascades NORTHEASTERN UNIVERSITY Modeling Text Embedded Information Cascades by Shaobin Xu A Thesis Submitted in Partial Fulfilment of the Requirements for the Degree of Doctor of Philosophy in Computer Science December, 2019 Abstract Networks mediate several aspects of society. For example, social networking services (SNS) like Twitter and Facebook have greatly helped people connect with families, friends and the outside world. Public policy diffuses over institutional and social networks that connect political actors in different areas. Inferring network structure is thus essential for understanding the transmission of ideas and information, which in turn could answer ques- tions about communities, collective actions, and influential social participants. Since many networks are not directly observed, we often rely on indirect evidence, such as the tim- ing of messages between participants, to infer latent connections. The textual content of messages, especially the reuse text originating elsewhere, is one source of such evidence. This thesis contributes techniques for detecting the evidence of text reuse and modeling underlying network structure. We propose methods to model text reuse with accidental and intentional lexical and semantic mutations. For lexical similarity detection, an n-gram shin- gling algorithm is proposed to detect “locally” reused passages, instead of near-duplicate documents, embedded within the larger text output of network nodes. For semantic sim- ilarity, we use an attention based neural network to also detect embedded reused texts. When modeling network structure, we are interested in inferring different levels of de- tails: individual links between participants, the structure of a specific information cascade, or global network properties. We propose a contrastive training objective for conditional models of edges in information cascades that has the flexibility to answer those questions ii Abstract iii and is also capable of incorporating rich node and edge features. Last but not least, net- work embedding methods prove to be a good way to learn the representations of nodes while preserving structure, node and edge properties, and side information. We propose a self-attention Transformer-based neural network trained to predict the next activated node in a given cascade to learn node embeddings. First Reader: David Smith Second Reader: Tina Eliassi-Rad Tertiary Reader: Byron Wallace External Reader: Bruce Desmarais Acknowledgment The journey to become a PhD can be daunting, frustrating, and yet wonderful. I owe so many thanks to many great people that helped me sail through the unforgettable 6 years as a PhD student. First and foremost, I would like to thank David, my advisor, for bringing me to the US, giving me the opportunity to work with him on many interesting NLP topics. I am deeply grateful for his patience and guidance throughout this time. His passion and knowledge about research have greatly educated and shaped me. This thesis could not have been done without his constant advice and support. His unique perspective of many matters also has made great impact on my life. I want to thank Tina, Byron, and Bruce for taking the time serving in my thesis commit- tee. Your comments and advice have greatly helped me make the thesis wholesome. I am so very appreciative of Byron’s detailed suggestions to revise the final draft of this thesis. I thank Professor Ryan Cordell, with whom I collaborated on part of the results related to this thesis. I am inspired by your passion on uncovering the 19-th century newspaper reprinting network and I am honored to be part of the team. I want to thank everyone from our lab – Liwen, Rui, Ansel, and Ryan. It is a privilege to have your company. Without any of you making the lab full of joy and energy, my life would have been miserable. I thank Rui for many deep discussions on work and life to keep my mind clear. I am truly grateful for all the brainstorms and discussions with Ansel iv Acknowledgment v in my final two and a half years to advance my research, as well as many aspects of the live in the US that I would never know otherwise. I thank my friends outside of my lab, Bingyu, Yupeng, Chin and Bochao. Your constant help has made my life so much easier when I already have so much weight on my shoulder for the PhD. You helped to fill a lot of void during this lonely journey. Finally, I thank my mom, Xiulin, who inspires me, encourages me and loves me uncon- ditionally and endlessly. I would not have come to the other side of the world, been able to face all the unknowns and stood on my feet, had I not had the support from her. To my family and my loving friends. Contents Abstract ii Acknowledgment iv Contents vii List of Figures xi List of Tables xiii 1 Introduction 1 1.1 Detecting Text Reuse . .2 1.2 Network Inference from Information Cascades . .5 1.3 Node Representation Learning . .7 1.4 Overview of The Thesis . .8 2 Text reuse in social networks 10 2.1 Local Text Reuse Detection . 11 2.2 Efficient N-gram Indexing . 12 2.3 Extracting and Ranking Candidate Pairs . 13 vii Contents viii 2.4 Computing Local Alignments . 14 2.5 Intrinsic Evaluation . 17 2.6 Extrinsic Evaluation . 18 2.7 Network Connections of 19c Reprints . 21 2.7.1 Dataset description . 21 2.7.2 Experiment . 24 2.8 Congressional Statements . 27 2.8.1 Dataset description . 27 2.8.2 Experiment . 28 2.9 Conclusion . 29 3 Semantic Text Reuse in Social Networks 30 3.1 Classifying text reuse as paraphrase or textual entailment . 31 3.2 Method Overview . 34 3.3 Word Representations . 35 3.4 Contextualized Sentence Representation . 37 3.5 Attention . 40 3.6 Final output . 41 3.7 Objective function . 42 3.8 Experiments . 42 3.8.1 Datasets . 42 3.8.2 Models . 44 3.8.3 Experiment settings . 46 3.8.4 Document level evaluation . 47 3.8.5 Sentence level evaluation . 49 3.8.6 Ablation Test . 50 Contents ix 3.9 Conclusion . 52 4 Modeling information cascades with rich feature sets 54 4.1 Network Structure Inference . 55 4.2 Log-linear Directed Spanning Tree Model . 56 4.3 Likelihood of a cascade . 58 4.4 Maximizing Likelihood . 59 4.5 Matrix-Tree Theorem and Laplacian Matrix . 59 4.6 Gradient . 61 4.7 ICWSM 2011 Webpost Dataset . 61 4.7.1 Dataset description . 62 4.7.2 Feature sets . 63 4.7.3 Result of unsupervised learning at cascade level . 64 4.7.4 Result of unsupervised learning at network level . 66 4.7.5 Enforcing tree structure on the data . 68 4.7.6 Result of supervised learning at cascade level . 70 4.8 State Policy Adoption Dataset . 70 4.8.1 Dataset description . 71 4.8.2 Effect of proximity of states . 71 4.9 Conclusion . 72 5 Modeling information cascades using self-attention neural networks 74 5.1 Node representation learning . 75 5.2 Information cascades as DAGs . 78 5.3 Graph self-attention network . 78 5.3.1 Analogy to language modeling . 79 5.3.2 Graph self-attention layer . 81 Contents x 5.3.3 Graph self-attention network . 83 5.3.4 Senders and receivers . 85 5.3.5 Hard attention . 86 5.3.6 Edge prediction . 90 5.4 Experiments . 90 5.4.1 Datasets . 90 5.4.2 Baselines . 93 5.4.3 Experimental settings . 95 5.4.4 Node prediction . 96 5.4.5 Edge prediction . 98 5.4.6 Effect of texts as side information . 99 5.5 Conclusion . 102 6 Conclusion 104 6.1 Future Work . 106 6.1.1 Text Reuse . 106 6.1.2 Network Structure Inference . 107 Bibliography 109 List of Figures 2.1 Average precision for aligned passages of different minimum length in characters. Vertical red lines indicate the performance of different parame- ter settings (see Table 2.1). 19 2.2 (Pseudo-)Recall for aligned passages of different minimum lengths in char- acters. 20 2.3 Newspaper issues mentioning “associated press” by year, from the Chron- icling America corpus. The black regression line fits the raw number of issues; the red line fits counts corrected for the number of times the Asso- ciated Press is mentioned in each issue. 23 2.4 Reprints of John Brown’s 1859 speech at his sentencing. Counties are shaded with historical population data, where available. Even taking pop- ulation differences into account, few newspapers in the South printed the abolitionist’s statement. 25 3.1 The overview of structure of Attention Based Convolutional Network (ABCN) 34 3.2 Unrolled Vanilla Recurrent Neural Network . 38 3.3 An illustration of ConvNet structure used in ABCN with toy example with word embeddings in R5, kernel size 2, feature map size 3. It yields a rep- resentation for the sentence in R3 ....................... 39 xi List of Figures xii 4.1 Recall, precision, and average precision of InfoPath and DST on predicting the time-varying networks generated per day. The DST model is trained un- supervisedly on separate cascades using basic and enhanced features. The upper row uses graph-structured cascades from the ICWSM 2011 dataset. The lower row uses the subset of cascades with tree structures. 69 5.1 The illustration of a cascade structure as DAG in a toy network. 79 5.2 Graph Self-Attention Network architecture, with L identical multi-headed self attention layer. 82 5.3 Graph Sender-Receiver attention network architecture, with L identical multi-headed attention layer for senders and receivers respectively. 87 5.4 Graph hard self-attention network architecture, with L − 1 identical multi- headed attention layer and the last layer replaced with a reinforcement learning agent selecting mask actions.
Load more

Recommended publications
  • Are Annotators' Word-Sense-Disambiguation
    J. Hlavácovᡠ(Ed.): ITAT 2017 Proceedings, pp. 5–14 CEUR Workshop Proceedings Vol. 1885, ISSN 1613-0073, c 2017 S. Cinková, A. Vernerová Are Annotators’ Word-Sense-Disambiguation Decisions Affected by Textual Entailment between Lexicon Glosses? Silvie Cinková, Anna Vernerová Charles University, Faculty of Mathematics and Physics Institute of Formal and Applied Linguistics Malostranské námestíˇ 25 118 00 Praha 1 Czech Republic ufal.mff.cuni.cz {cinkova,vernerova}@ufal.mff.cuni.cz Abstract: We describe an annotation experiment com- verb on the one hand, and those related to the lexicograph- bining topics from lexicography and Word Sense Disam- ical design of PDEV, such as semantic relations between biguation. It involves a lexicon (Pattern Dictionary of En- implicatures within a lemma or denotative similarity of the glish Verbs, PDEV), an existing data set (VPS-GradeUp), verb arguments, on the other hand (see Section 3 for defi- and an unpublished data set (RTE in PDEV Implicatures). nitions and examples). The aim of the experiment was twofold: a pilot annota- This paper focuses on a feature related to PDEV’s de- tion of Recognizing Textual Entailment (RTE) on PDEV sign (see Section 2.3), namely on textual entailment be- implicatures (lexicon glosses) on the one hand, and, on tween implicatures in pairs of patterns of the same lemma the other hand, an analysis of the effect of Textual Entail- entry (henceforth colempats, see Section 3.1 for definition ment between lexicon glosses on annotators’ Word-Sense- and more detail). Disambiguation decisions, compared to other predictors, We pairwise compare all colempats, examining their such as finiteness of the target verb, the explicit presence scores in the graded decision annotation with respect to of its relevant arguments, and the semantic distance be- how much they compete to become the most appropriate tween corresponding syntactic arguments in two different pattern, as well as the scores of presence of textual entail- patterns (dictionary senses).
    [Show full text]
  • Automatic Correction of Real-Word Errors in Spanish Clinical Texts
    sensors Article Automatic Correction of Real-Word Errors in Spanish Clinical Texts Daniel Bravo-Candel 1,Jésica López-Hernández 1, José Antonio García-Díaz 1 , Fernando Molina-Molina 2 and Francisco García-Sánchez 1,* 1 Department of Informatics and Systems, Faculty of Computer Science, Campus de Espinardo, University of Murcia, 30100 Murcia, Spain; [email protected] (D.B.-C.); [email protected] (J.L.-H.); [email protected] (J.A.G.-D.) 2 VÓCALI Sistemas Inteligentes S.L., 30100 Murcia, Spain; [email protected] * Correspondence: [email protected]; Tel.: +34-86888-8107 Abstract: Real-word errors are characterized by being actual terms in the dictionary. By providing context, real-word errors are detected. Traditional methods to detect and correct such errors are mostly based on counting the frequency of short word sequences in a corpus. Then, the probability of a word being a real-word error is computed. On the other hand, state-of-the-art approaches make use of deep learning models to learn context by extracting semantic features from text. In this work, a deep learning model were implemented for correcting real-word errors in clinical text. Specifically, a Seq2seq Neural Machine Translation Model mapped erroneous sentences to correct them. For that, different types of error were generated in correct sentences by using rules. Different Seq2seq models were trained and evaluated on two corpora: the Wikicorpus and a collection of three clinical datasets. The medicine corpus was much smaller than the Wikicorpus due to privacy issues when dealing Citation: Bravo-Candel, D.; López-Hernández, J.; García-Díaz, with patient information.
    [Show full text]
  • Textual Inference for Machine Comprehension Martin Gleize
    Textual Inference for Machine Comprehension Martin Gleize To cite this version: Martin Gleize. Textual Inference for Machine Comprehension. Computation and Language [cs.CL]. Université Paris Saclay (COmUE), 2016. English. NNT : 2016SACLS004. tel-01317577 HAL Id: tel-01317577 https://tel.archives-ouvertes.fr/tel-01317577 Submitted on 18 May 2016 HAL is a multi-disciplinary open access L’archive ouverte pluridisciplinaire HAL, est archive for the deposit and dissemination of sci- destinée au dépôt et à la diffusion de documents entific research documents, whether they are pub- scientifiques de niveau recherche, publiés ou non, lished or not. The documents may come from émanant des établissements d’enseignement et de teaching and research institutions in France or recherche français ou étrangers, des laboratoires abroad, or from public or private research centers. publics ou privés. Le cas échéant, logo de l’établissement co-délivrant le doctorat en cotutelle internationale de thèse , sinon mettre le logo de l’établissement de préparation de la thèse (UPSud, HEC, UVSQ, UEVE, ENS Cachan, Polytechnique, IOGS, …) NNT : 2016SACLS004 THÈSE DE DOCTORAT DE L’UNIVERSITÉ PARIS-SACLAY PRÉPARÉE À L'UNIVERSITÉ PARIS-SUD ECOLE DOCTORALE N° 580 Sciences et technologies de l'information et de la communication (STIC) Spécialité de doctorat : Informatique Par M. Martin Gleize Textual Inference for Machine Comprehension Thèse présentée et soutenue à Orsay, le 7 janvier 2016 Composition du Jury : M. Yvon François Professeur, Université Paris-Sud Président, Examinateur Mme Gardent Claire Directeur de recherche CNRS, LORIA Rapporteur M. Magnini Bernardo Senior Researcher, FBK Rapporteur M. Piwowarski Benjamin Chargé de recherche CNRS, LIP6 Examinateur Mme Grau Brigitte Professeur, ENSIIE Directeur de thèse Abstract With the ever-growing mass of published text, natural language understanding stands as one of the most sought-after goal of artificial intelligence.
    [Show full text]
  • Repurposing Entailment for Multi-Hop Question Answering Tasks
    Repurposing Entailment for Multi-Hop Question Answering Tasks Harsh Trivedi|, Heeyoung Kwon|, Tushar Khot♠, Ashish Sabharwal♠, Niranjan Balasubramanian| | Stony Brook University, Stony Brook, U.S.A. fhjtrivedi,heekwon,[email protected] ♠ Allen Institute for Artificial Intelligence, Seattle, U.S.A. ftushark,[email protected] Abstract Question Answering (QA) naturally reduces to an entailment problem, namely, verifying whether some text entails the answer to a ques- tion. However, for multi-hop QA tasks, which require reasoning with multiple sentences, it remains unclear how best to utilize entailment models pre-trained on large scale datasets such as SNLI, which are based on sentence pairs. We introduce Multee, a general architecture Figure 1: An example illustrating the challenges in us- that can effectively use entailment models for ing sentence-level entailment model for multi-sentence multi-hop QA tasks. Multee uses (i) a local reasoning needed for QA, and the high-level approach module that helps locate important sentences, used in Multee. thereby avoiding distracting information, and level, whereas question answering requires verify- (ii) a global module that aggregates informa- ing whether multiple sentences, taken together as tion by effectively incorporating importance weights. Importantly, we show that both mod- a premise, entail a hypothesis. ules can use entailment functions pre-trained There are two straightforward ways to address on a large scale NLI datasets. We evaluate per- this mismatch: (1) aggregate independent entail- formance on MultiRC and OpenBookQA, two ment decisions over each premise sentence, or (2) multihop QA datasets. When using an entail- make a single entailment decision after concate- ment function pre-trained on NLI datasets, nating all premise sentences.
    [Show full text]
  • Techniques for Recognizing Textual Entailment and Semantic Equivalence
    View metadata, citation and similar papers at core.ac.uk brought to you by CORE provided by EPrints Complutense Techniques for Recognizing Textual Entailment and Semantic Equivalence Jes´us Herrera, Anselmo Pe˜nas, Felisa Verdejo Departamento de Lenguajes y Sistemas Inform´aticos Universidad Nacional de Educaci´ona Distancia Madrid, Spain {jesus.herrera, anselmo, felisa}@lsi.uned.es Abstract. After defining what is understood by textual entailment and semantic equivalence, the present state and the desirable future of the systems aimed at recognizing them is shown. A compilation of the currently implemented techniques in the main Recognizing Textual Entailment and Semantic Equivalence systems is given. 1 Introduction The concept “textual entailment” is used to indicate the state in which the semantics of a natural language written text can be inferred from the semantics of another one. More specifically, if the truth of an enunciation entails the truth of another enunciation. For example, given the texts: 1. The three-day G8 meeting will take place in Scotland. 2. The Group of Eight summit will last three days. it is clear that the semantics of the second one can be inferred from the semantics of the first one; then, it is said that textual entailment exists between both texts. Textual entailment is a directional relationship: in the example above, the first statement entails the second one, but this entailment is not given in the opposite direction. The recognition of textual entailment requires a processing at the lexical level (for example, synonymy between meeting and summit or between G8 and Group of Eight), as well as at the syntactic level and the sentence semantic level.
    [Show full text]
  • Unified Language Model Pre-Training for Natural
    Unified Language Model Pre-training for Natural Language Understanding and Generation Li Dong∗ Nan Yang∗ Wenhui Wang∗ Furu Wei∗† Xiaodong Liu Yu Wang Jianfeng Gao Ming Zhou Hsiao-Wuen Hon Microsoft Research {lidong1,nanya,wenwan,fuwei}@microsoft.com {xiaodl,yuwan,jfgao,mingzhou,hon}@microsoft.com Abstract This paper presents a new UNIfied pre-trained Language Model (UNILM) that can be fine-tuned for both natural language understanding and generation tasks. The model is pre-trained using three types of language modeling tasks: unidirec- tional, bidirectional, and sequence-to-sequence prediction. The unified modeling is achieved by employing a shared Transformer network and utilizing specific self-attention masks to control what context the prediction conditions on. UNILM compares favorably with BERT on the GLUE benchmark, and the SQuAD 2.0 and CoQA question answering tasks. Moreover, UNILM achieves new state-of- the-art results on five natural language generation datasets, including improving the CNN/DailyMail abstractive summarization ROUGE-L to 40.51 (2.04 absolute improvement), the Gigaword abstractive summarization ROUGE-L to 35.75 (0.86 absolute improvement), the CoQA generative question answering F1 score to 82.5 (37.1 absolute improvement), the SQuAD question generation BLEU-4 to 22.12 (3.75 absolute improvement), and the DSTC7 document-grounded dialog response generation NIST-4 to 2.67 (human performance is 2.65). The code and pre-trained models are available at https://github.com/microsoft/unilm. 1 Introduction Language model (LM) pre-training has substantially advanced the state of the art across a variety of natural language processing tasks [8, 29, 19, 31, 9, 1].
    [Show full text]
  • Automatic Evaluation of Summary Using Textual Entailment
    Automatic Evaluation of Summary Using Textual Entailment Pinaki Bhaskar Partha Pakray Department of Computer Science & Department of Computer Science & Engineering, Jadavpur University, Engineering, Jadavpur University, Kolkata – 700032, India Kolkata – 700032, India [email protected] [email protected] summary is very much needed when a large Abstract number of summaries have to be evaluated, spe- cially for multi-document summaries. For sum- This paper describes about an automatic tech- mary evaluation we have developed an automat- nique of evaluating summary. The standard ed evaluation technique based on textual entail- and popular summary evaluation techniques or ment. tools are not fully automatic; they all need Recognizing Textual Entailment (RTE) is one some manual process. Using textual entail- of the recent research areas of Natural Language ment (TE) the generated summary can be evaluated automatically without any manual Processing (NLP). Textual Entailment is defined evaluation/process. The TE system is the as a directional relationship between pairs of text composition of lexical entailment module, lex- expressions, denoted by the entailing “Text” (T) ical distance module, Chunk module, Named and the entailed “Hypothesis” (H). T entails H if Entity module and syntactic text entailment the meaning of H can be inferred from the mean- (TE) module. The syntactic TE system is ing of T. Textual Entailment has many applica- based on the Support Vector Machine (SVM) tions in NLP tasks, such as Summarization, In- that uses twenty five features for lexical simi- formation Extraction, Question Answering, In- larity, the output tag from a rule based syntac- formation Retrieval. tic two-way TE system as a feature and the outputs from a rule based Chunk Module and 2 Related Work Named Entity Module as the other features.
    [Show full text]
  • Dependency-Based Textual Entailment
    Dependency-based Textual Entailment Vasile Rus Department of Computer Science Institute for Intelligent Systems The University of Memphis Memphis, TN 38120, USA [email protected] Abstract and Hypothesis are mapped into two sets of dependen- cies: the T-set and H-set, respectively. A score is then This paper studies the role of dependency information computed that measures the degree to which the depen- for the task of textual entailment. Both the Text and Hypothesis of an entailment pair are mapped into sets dencies in the H-set are present in the T-set. Based on of dependencies and a score is computed that measures the score, an entailment decision is made. Dependen- the similarity of the two sets. Based on the score an cies are labelled relations among words in a sentence. entailment decision is made. Two experiments are con- For example, there is a subj dependency between cap- ducted to measure the impact of dependencies on the ital and Besancon in the previous example which we entailment task. In one experiment we compare the represent as the triplet (subj, capital, Besancon). The dependency-based approach with a baseline approach first term in the triplet is the name or label of the re- on a standard data set. In a second experiment, we lation, the second term represents the head word, and measure the performance on a subset of the standard the last term is the dependent word. We use MINIPAR data set. The subset is so selected to minimize the (Lin 1998), an automated dependency parser, as our effect of other factors, such as word-level information, in the performance measurement process.
    [Show full text]
  • Parser Evaluation Using Textual Entailments
    Language Resources and Evaluation manuscript No. (will be inserted by the editor) Parser Evaluation Using Textual Entailments Deniz Yuret · Laura Rimell · Aydın Han Received: date / Accepted: date Abstract Parser Evaluation using Textual Entailments (PETE) is a shared task in the SemEval-2010 Evaluation Exercises on Semantic Evaluation. The task involves recognizing textual entailments based on syntactic information alone. PETE introduces a new parser evaluation scheme that is formalism independent, less prone to annotation error, and focused on semantically rele- vant distinctions. This paper describes the PETE task, gives an error analysis of the top-performing Cambridge system, and introduces a standard entail- ment module that can be used with any parser that outputs Stanford typed dependencies. Keywords Parsing · Textual Entailments 1 Introduction Parser Evaluation using Textual Entailments (PETE) is a shared task in the SemEval-2010 Evaluation Exercises on Semantic Evaluation that involves rec- ognizing textual entailments to evaluate parser performance. Given two text Deniz Yuret and Aydın Han Ko¸cUniversity Rumelifeneri Yolu 34450 Sarıyer, Istanbul,˙ Turkey Tel.: +90-212-338-1724 Fax: +90-212-338-1548 E-mail: dyuret,[email protected] Laura Rimell Computer Laboratory William Gates Building 15 JJ Thomson Avenue Cambridge CB3 0FD, UK Tel.: +44 (0)1223 334696 E-mail: [email protected] 2 Deniz Yuret et al. fragments called “text” (T) and “hypothesis” (H), recognizing textual entail- ment (RTE) is the task of determining whether the meaning of the hypothesis is entailed (can be inferred) from the text. In contrast to general RTE tasks (Dagan et al 2009) PETE is a targeted textual entailment task that focuses on syntactic entailments: Text: The man with the hat was tired.
    [Show full text]
  • Analyzing Political Polarization in News Media with Natural Language Processing
    Analyzing Political Polarization in News Media with Natural Language Processing by Brian Sharber A thesis presented to the Honors College of Middle Tennessee State University in partial fulfillment of the requirements for graduation from the University Honors College Fall 2020 Analyzing Political Polarization in News Media with Natural Language Processing by Brian Sharber APPROVED: ______________________________ Dr. Salvador E. Barbosa Project Advisor Computer Science Department Dr. Medha Sarkar Computer Science Department Chair ____________________________ Dr. John R. Vile, Dean University Honors College Copyright © 2020 Brian Sharber Department of Computer Science Middle Tennessee State University; Murfreesboro, Tennessee, USA. I hereby grant to Middle Tennessee State University (MTSU) and its agents (including an institutional repository) the non-exclusive right to archive, preserve, and make accessible my thesis in whole or in part in all forms of media now and hereafter. I warrant that the thesis and the abstract are my original work and do not infringe or violate any rights of others. I agree to indemnify and hold MTSU harmless for any damage which may result from copyright infringement or similar claims brought against MTSU by third parties. I retain all ownership rights to the copyright of my thesis. I also retain the right to use in future works (such as articles or books) all or part of this thesis. The software described in this work is free software. You can redistribute it and/or modify it under the terms of the MIT License. The software is posted on GitHub under the following repository: https://github.com/briansha/NLP-Political-Polarization iii Abstract Political discourse in the United States is becoming increasingly polarized.
    [Show full text]
  • Automatic Spelling Correction Based on N-Gram Model
    International Journal of Computer Applications (0975 – 8887) Volume 182 – No. 11, August 2018 Automatic Spelling Correction based on n-Gram Model S. M. El Atawy A. Abd ElGhany Dept. of Computer Science Dept. of Computer Faculty of Specific Education Damietta University Damietta University, Egypt Egypt ABSTRACT correction. In this paper, we are designed, implemented and A spell checker is a basic requirement for any language to be evaluated an end-to-end system that performs spellchecking digitized. It is a software that detects and corrects errors in a and auto correction. particular language. This paper proposes a model to spell error This paper is organized as follows: Section 2 illustrates types detection and auto-correction that is based on n-gram of spelling error and some of related works. Section 3 technique and it is applied in error detection and correction in explains the system description. Section 4 presents the results English as a global language. The proposed model provides and evaluation. Finally, Section 5 includes conclusion and correction suggestions by selecting the most suitable future work. suggestions from a list of corrective suggestions based on lexical resources and n-gram statistics. It depends on a 2. RELATED WORKS lexicon of Microsoft words. The evaluation of the proposed Spell checking techniques have been substantially, such as model uses English standard datasets of misspelled words. error detection & correction. Two commonly approaches for Error detection, automatic error correction, and replacement error detection are dictionary lookup and n-gram analysis. are the main features of the proposed model. The results of the Most spell checkers methods described in the literature, use experiment reached approximately 93% of accuracy and acted dictionaries as a list of correct spellings that help algorithms similarly to Microsoft Word as well as outperformed both of to find target words [16].
    [Show full text]
  • N-Gram-Based Machine Translation
    N-gram-based Machine Translation ∗ JoseB.Mari´ no˜ ∗ Rafael E. Banchs ∗ Josep M. Crego ∗ Adria` de Gispert ∗ Patrik Lambert ∗ Jose´ A. R. Fonollosa ∗ Marta R. Costa-jussa` Universitat Politecnica` de Catalunya This article describes in detail an n-gram approach to statistical machine translation. This ap- proach consists of a log-linear combination of a translation model based on n-grams of bilingual units, which are referred to as tuples, along with four specific feature functions. Translation performance, which happens to be in the state of the art, is demonstrated with Spanish-to-English and English-to-Spanish translations of the European Parliament Plenary Sessions (EPPS). 1. Introduction The beginnings of statistical machine translation (SMT) can be traced back to the early fifties, closely related to the ideas from which information theory arose (Shannon and Weaver 1949) and inspired by works on cryptography (Shannon 1949, 1951) during World War II. According to this view, machine translation was conceived as the problem of finding a sentence by decoding a given “encrypted” version of it (Weaver 1955). Although the idea seemed very feasible, enthusiasm faded shortly afterward because of the computational limitations of the time (Hutchins 1986). Finally, during the nineties, two factors made it possible for SMT to become an actual and practical technology: first, significant increment in both the computational power and storage capacity of computers, and second, the availability of large volumes of bilingual data. The first SMT systems were developed in the early nineties (Brown et al. 1990, 1993). These systems were based on the so-called noisy channel approach, which models the probability of a target language sentence T given a source language sentence S as the product of a translation-model probability p(S|T), which accounts for adequacy of trans- lation contents, times a target language probability p(T), which accounts for fluency of target constructions.
    [Show full text]