DOCSLIB.ORG

Using Semantic Folding with Textrank for Automatic Summarization

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Using Semantic Folding with Textrank for Automatic Summarization DEGREE PROJECT IN COMPUTER SCIENCE AND ENGINEERING, SECOND CYCLE, 30 CREDITS STOCKHOLM, SWEDEN 2017 Using semantic folding with TextRank for automatic summarization SIMON KARLSSON KTH ROYAL INSTITUTE OF TECHNOLOGY SCHOOL OF COMPUTER SCIENCE AND COMMUNICATION Using semantic folding with TextRank for automatic summarization SIMON KARLSSON [email protected] Master in Computer Science Date: June 27, 2017 Principal: Findwise AB Supervisor at Findwise: Henrik Laurentz Supervisor at KTH: Stefan Nilsson Examiner at KTH: Olov Engwall Swedish title: TextRank med semantisk vikning för automatisk sammanfattning School of Computer Science and Communication i Abstract This master thesis deals with automatic summarization of text and how semantic fold- ing can be used as a similarity measure between sentences in the TextRank algorithm. The method was implemented and compared with two common similarity measures. These two similarity measures were cosine similarity of tf-idf vectors and the number of overlapping terms in two sentences. The three methods were implemented and the linguistic features used in the construc- tion were stop words, part-of-speech filtering and stemming. Five different part-of- speech filters were used, with different mixtures of nouns, verbs, and adjectives. The three methods were evaluated by summarizing documents from the Document Understanding Conference and comparing them to gold-standard summarization cre- ated by human judges. Comparison between the system summaries and gold-standard summaries was made with the ROUGE-1 measure. The algorithm with semantic fold- ing performed worst of the three methods, but only 0.0096 worse in F-score than cosine similarity of tf-idf vectors that performed best. For semantic folding, the average preci- sion was 46.2% and recall 45.7% for the best-performing part-of-speech filter. ii Sammanfattning Det här examensarbetet behandlar automatisk textsammanfattning och hur semantisk vikning kan användas som likhetsmått mellan meningar i algoritmen TextRank. Meto- den implementerades och jämfördes med två vanliga likhetsmått. Dessa två likhetsmått var cosinus-likhet mellan tf-idf-vektorer samt antal överlappande termer i två mening- ar. De tre metoderna implementerades och de lingvistiska särdragen som användes vid konstruktionen var stoppord, filtrering av ordklasser samt en avstämmare. Fem olika filter för ordklasser användes, med olika blandningar av substantiv, verb och adjektiv. De tre metoderna utvärderades genom att sammanfatta dokument från DUC och jäm- föra dessa mot guldsammanfattningar skapade av mänskliga domare. Jämförelse mel- lan systemsammanfattningar och guldsammanfattningar gjordes med måttet ROUGE-1. Algoritmen med semantisk vikning presterade sämst av de tre jämförda me- toderna, dock bara 0.0096 sämre i F-score än cosinus-likhet mellan tf-idf-vektorer som presterade bäst. För semantisk vikning var den genomsnittliga precisionen 46.2% och recall 45.7% för det ordklassfiltret som presterade bäst. iii Acknowledgements I would like to thank Henrik Laurentz for invaluable guidance throughout the entire project. This thesis would simply not be what it is if it was not for you, Simon Stenström for the initial discussions that made this thesis possible, Stefan Nilsson for your feedback on the scientific approach of the project, Olov Engwall for examining this thesis, Findwise for making me feel welcome every single day, Josefine for your never ending love and support throughout my years at KTH, Family and friends for always being there for me. Contents 1 Introduction 1 1.1 Problem definition . .2 1.2 Objective . .2 1.3 Purpose . .2 1.4 Delimitations . .3 1.5 Outline of the report . .3 2 Theory 5 2.1 Automatic summarization . .5 2.1.1 Categories of automatic summarization . .6 2.1.2 TextRank: Automatic summarization using graph representations7 2.1.3 Sentence similarity . .9 2.2 Distributional semantics . 10 2.2.1 Semantic folding . 11 2.3 Linguistic features . 12 2.3.1 Tokenization . 12 2.3.2 Stop words . 12 2.3.3 PoS tagging . 12 2.3.4 Stemming . 12 2.3.5 Lemmatisation . 14 2.4 Evaluation . 14 2.4.1 Instrinsic evaluation . 14 2.4.2 Extrinsic evaluation . 15 2.5 Related work . 15 2.5.1 Distributional semantics . 15 2.5.2 Automatic summarization using context . 16 3 Implementation 18 3.1 Architecture . 18 3.2 Preprocessing . 18 3.2.1 Sentences tokenization . 19 iv CONTENTS v 3.2.2 Word tokenization . 21 3.2.3 Stop words . 21 3.2.4 PoS filtering . 21 3.2.5 Stemming . 21 3.3 Creation of the graph . 22 3.3.1 Tf-Idf . 23 3.3.2 Term overlap . 23 3.3.3 Semantic folding . 24 3.4 Scoring of the graph . 24 3.5 Picking the sentences . 24 4 Evaluation 25 4.1 Data . 25 4.2 Gold-standard comparison . 25 4.3 Preprocessing of system and reference summaries . 26 4.4 Measures . 26 4.4.1 Precision . 26 4.4.2 Recall . 27 4.4.3 F1 score . 27 4.4.4 Upper bound of the Rouge-score . 27 5 Results 28 5.1 The average score . 28 6 Discussion 30 6.1 Methodology . 30 6.2 Results . 31 6.3 Criticism . 32 6.4 Ethics and sustainability . 33 7 Conclusion 34 7.1 Future work . 34 7.1.1 Train model on specific domain . 34 7.1.2 Preprocessing . 35 7.1.3 Extrinsic evaluation . 35 7.1.4 Evaluate readability . 35 Bibliography 36 A Preprocessing 42 A.1 Steps of the porter stemming algorithm . 42 A.2 The stoplist used . 45 CONTENTS vi B Results 46 B.1 The median score . 46 B.2 The standard deviation score . 47 Abbreviations DUC Document Understanding Conference. 2, 25–27, 31, 34 idf inverse document frequency. 9, 19, 22, 23, 25, 28–32, 34, 47, 48 JUNG Java Universal Network/Graph Framework. 24 LSA Latent Semantic Analysis. 15–17 PoS Part of Speech. 3, 12, 18, 19, 21, 23, 28, 29, 32, 34, 35 ROUGE Recall-Oriented Understudy for Gisting Evaluation. 2, 25, 26, 29, 30, 46–48 tf term frequency. 9, 19, 22, 23, 25, 28–32, 34, 47, 48 vii Glossary co-occurrence When two terms occurs alongside in a text. 2 corpus A large set of text. 3, 9, 12, 16, 19–21, 23, 24, 32 n-gram A sequence of n words from a text. 2, 11, 26, 27 snippet A small piece of text. 11 vocabulary A set of words used for a specific purpose. 9, 12, 14 viii Chapter 1 Introduction This thesis presents a novel approach to automatic summarization, using the TextRank algorithm with semantic folding. This method was evaluated against two state-of-the- art methods. Automatic summarization is the concept of creating a summary of a text in such a way as to maximize the relevant information from the original text in the summary. The summary can either be extractive, which means that a subset of the sentences from the original text is chosen to the summary, or abstractive, which means that entirely new sentences are constructed and placed in the summary. One approach to extractive summarization is to create a graph from the original text in which the nodes represents sentences and edges represent some kind of similarity between the corresponding sentences. Sentences are added to the summary based on centrality in the graph, with the hypothesis that central sentences are important and hence should be in the summary. This algorithm is known as TextRank [1] 1. This thesis evaluates the performance of this algorithm using a novel method of as- sessing the similarity between sentences in TextRank, based on distributional semantics. Distributional semantics builds upon the hypothesis that words which are similar in meaning occur in similar contexts, which means that the similarity of text units can be estimated by examining the distributional similarity between text units [3]. Semantic folding is a distributional semantics method with the purpose of capturing the meaning of a text unit with the context in which it appears [4]. The novel similarity measure described in this thesis is based on semantic folding. 1Another graph-based summarization algorithm, LexRank [2], was developed at the same time as TextRank, using different similarity measures. However, TextRank will be used to denote this summariza- tion method throughout this thesis. 1 CHAPTER 1. INTRODUCTION 2 1.1 Problem definition The research question of this thesis is: How will the TextRank algorithm perform with semantic folding as similarity measure? To answer the research question, the method of using semantic folding as a similarity measure in TextRank was evaluated against two state-of-the-art methods of compar- ing the similarity between texts. These similarity measures are cosine similarity of tf-idf vectors and the number of overlapping terms of two sentences. To answer the research question, the problem was divided into four parts: 1. Implementation of the TextRank algorithm with the overlap measure for similar- ity between sentences. 2. Implementation of the TextRank algorithm with the cosine similarity of tf-idf vec- tors measure. 3. Incorporation of semantic folding in the TextRank algorithm for similarity be- tween sentences. 4. Evaluation of the three different similarity measures above. 1.2 Objective The objective of this thesis was to implement the TextRank algorithm for automatic summarization with three different similarity measures, cosine similarity of tf-idf vec- tors, term overlap and a novel approach based on semantic folding. The different methods were evaluated by comparing the constructed summaries with human cre- ated gold standard summaries from the Document Understanding Conference (DUC). The comparison was made with the Recall-Oriented Understudy for Gisting Evalua- tion (ROUGE) metric set, and specifically with the ROUGE-N measure, which is based on co-occurrences of n-grams [5]. 1.3 Purpose Information overload is a term describing the problem of making decisions due to the existence of to much information.
Load more

Recommended publications
  • Approximate Pattern Matching Using Hierarchical Graph Construction and Sparse Distributed Representation
    Portland State University PDXScholar Dissertations and Theses Dissertations and Theses 9-29-2020 Approximate Pattern Matching Using Hierarchical Graph Construction and Sparse Distributed Representation Aakanksha Mathuria Portland State University Follow this and additional works at: https://pdxscholar.library.pdx.edu/open_access_etds Part of the Electrical and Computer Engineering Commons Let us know how access to this document benefits ou.y Recommended Citation Mathuria, Aakanksha, "Approximate Pattern Matching Using Hierarchical Graph Construction and Sparse Distributed Representation" (2020). Dissertations and Theses. Paper 5581. https://doi.org/10.15760/etd.7453 This Thesis is brought to you for free and open access. It has been accepted for inclusion in Dissertations and Theses by an authorized administrator of PDXScholar. Please contact us if we can make this document more accessible: [email protected]. Approximate Pattern Matching using Hierarchical Graph Construction and Sparse Distributed Representation by Aakanksha Mathuria A thesis submitted in partial fulfillment of the requirements for the degree of Master of Science in Electrical and Computer Engineering Thesis Committee: Dan Hammerstrom, Chair Christof Teuscher Nirupama Bulusu Portland State University 2020 Abstract With recent developments in deep networks, there have been significant advances in visual object detection and recognition. However, some of these networks are still easily fooled/hacked and have shown ”bag of features” kinds of failures. Some of this is due to the fact that even deep networks make only marginal use of the complex structure that exists in real-world images. Primate visual systems appear to capture the structure in images, but how? In the research presented here, we are studying approaches for robust pattern matching using static, 2D Blocks World images based on graphical representations of the various components of an image.
    [Show full text]
  • Automatic Summarization of Student Course Feedback
    Automatic Summarization of Student Course Feedback Wencan Luo† Fei Liu‡ Zitao Liu† Diane Litman† †University of Pittsburgh, Pittsburgh, PA 15260 ‡University of Central Florida, Orlando, FL 32716 wencan, ztliu, litman @cs.pitt.edu [email protected] { } Abstract Prompt Describe what you found most interesting in today’s class Student course feedback is generated daily in Student Responses both classrooms and online course discussion S1: The main topics of this course seem interesting and forums. Traditionally, instructors manually correspond with my major (Chemical engineering) analyze these responses in a costly manner. In S2: I found the group activity most interesting this work, we propose a new approach to sum- S3: Process that make materials marizing student course feedback based on S4: I found the properties of bike elements to be most the integer linear programming (ILP) frame- interesting work. Our approach allows different student S5: How materials are manufactured S6: Finding out what we will learn in this class was responses to share co-occurrence statistics and interesting to me alleviates sparsity issues. Experimental results S7: The activity with the bicycle parts on a student feedback corpus show that our S8: “part of a bike” activity approach outperforms a range of baselines in ... (rest omitted, 53 responses in total.) terms of both ROUGE scores and human eval- uation. Reference Summary - group activity of analyzing bicycle’s parts - materials processing - the main topic of this course 1 Introduction Table 1: Example student responses and a reference summary Instructors love to solicit feedback from students. created by the teaching assistant. ‘S1’–‘S8’ are student IDs.
    [Show full text]
  • Using N-Grams to Understand the Nature of Summaries
    Using N-Grams to Understand the Nature of Summaries Michele Banko and Lucy Vanderwende One Microsoft Way Redmond, WA 98052 {mbanko, lucyv}@microsoft.com views of the event being described over different Abstract documents, or present a high-level view of an event that is not explicitly reflected in any single document. A Although single-document summarization is a useful multi-document summary may also indicate the well-studied task, the nature of multi- presence of new or distinct information contained within document summarization is only beginning to a set of documents describing the same topic (McKeown be studied in detail. While close attention has et. al., 1999, Mani and Bloedorn, 1999). To meet these been paid to what technologies are necessary expectations, a multi-document summary is required to when moving from single to multi-document generalize, condense and merge information coming summarization, the properties of human- from multiple sources. written multi-document summaries have not Although single-document summarization is a well- been quantified. In this paper, we empirically studied task (see Mani and Maybury, 1999 for an characterize human-written summaries overview), multi-document summarization is only provided in a widely used summarization recently being studied closely (Marcu & Gerber 2001). corpus by attempting to answer the questions: While close attention has been paid to multi-document Can multi-document summaries that are summarization technologies (Barzilay et al. 2002, written by humans be characterized as Goldstein et al 2000), the inherent properties of human- extractive or generative? Are multi-document written multi-document summaries have not yet been summaries less extractive than single- quantified.
    [Show full text]
  • Automatic Summarization of Medical Conversations, a Review Jessica Lopez
    Automatic summarization of medical conversations, a review Jessica Lopez To cite this version: Jessica Lopez. Automatic summarization of medical conversations, a review. TALN-RECITAL 2019- PFIA 2019, Jul 2019, Toulouse, France. pp.487-498. hal-02611210 HAL Id: hal-02611210 https://hal.archives-ouvertes.fr/hal-02611210 Submitted on 30 May 2020 HAL is a multi-disciplinary open access L’archive ouverte pluridisciplinaire HAL, est archive for the deposit and dissemination of sci- destinée au dépôt et à la diffusion de documents entific research documents, whether they are pub- scientifiques de niveau recherche, publiés ou non, lished or not. The documents may come from émanant des établissements d’enseignement et de teaching and research institutions in France or recherche français ou étrangers, des laboratoires abroad, or from public or private research centers. publics ou privés. Jessica López Espejel Automatic summarization of medical conversations, a review Jessica López Espejel 1, 2 (1) CEA, LIST, DIASI, F-91191 Gif-sur-Yvette, France. (2) Paris 13 University, LIPN, 93430 Villateneuse, France. [email protected] RÉSUMÉ L’analyse de la conversation joue un rôle important dans le développement d’appareils de simulation pour la formation des professionnels de la santé (médecins, infirmières). Notre objectif est de développer une méthode de synthèse automatique originale pour les conversations médicales entre un patient et un professionnel de la santé, basée sur les avancées récentes en matière de synthèse à l’aide de réseaux de neurones convolutionnels et récurrents. La méthode proposée doit être adaptée aux problèmes spécifiques liés à la synthèse des dialogues. Cet article présente une revue des différentes méthodes pour les résumés par extraction et par abstraction et pour l’analyse du dialogue.
    [Show full text]
  • Exploring Sentence Vector Spaces Through Automatic Summarization
    Under review as a conference paper at ICLR 2018 EXPLORING SENTENCE VECTOR SPACES THROUGH AUTOMATIC SUMMARIZATION Anonymous authors Paper under double-blind review ABSTRACT Vector semantics, especially sentence vectors, have recently been used success- fully in many areas of natural language processing. However, relatively little work has explored the internal structure and properties of spaces of sentence vectors. In this paper, we will explore the properties of sentence vectors by studying a par- ticular real-world application: Automatic Summarization. In particular, we show that cosine similarity between sentence vectors and document vectors is strongly correlated with sentence importance and that vector semantics can identify and correct gaps between the sentences chosen so far and the document. In addition, we identify specific dimensions which are linked to effective summaries. To our knowledge, this is the first time specific dimensions of sentence embeddings have been connected to sentence properties. We also compare the features of differ- ent methods of sentence embeddings. Many of these insights have applications in uses of sentence embeddings far beyond summarization. 1 INTRODUCTION Vector semantics have been growing in popularity for many other natural language processing appli- cations. Vector semantics attempt to represent words as vectors in a high-dimensional space, where vectors which are close to each other have similar meanings. Various models of vector semantics have been proposed, such as LSA (Landauer & Dumais, 1997), word2vec (Mikolov et al., 2013), and GLOVE(Pennington et al., 2014), and these models have proved to be successful in other natural language processing applications. While these models work well for individual words, producing equivalent vectors for sentences or documents has proven to be more difficult.
    [Show full text]
  • Keyphrase Based Evaluation of Automatic Text Summarization
    International Journal of Computer Applications (0975 – 8887) Volume 117 – No. 7, May 2015 Keyphrase based Evaluation of Automatic Text Summarization Fatma Elghannam Tarek El-Shishtawy Electronics Research Institute Faculty of Computers and Information Cairo, Egypt Benha University, Benha, Egypt ABSTRACT KpEval idea is to count the matches between the peer The development of methods to deal with the informative summary and reference summaries for the essential parts of contents of the text units in the matching process is a major the summary text. KpEval have three main modules, i) challenge in automatic summary evaluation systems that use lemma extractor module that breaks the text into words and fixed n-gram matching. The limitation causes inaccurate extracts their lemma forms and the associated lexical and matching between units in a peer and reference summaries. syntactic features, ii) keyphrase extractor that extracts The present study introduces a new Keyphrase based important keyphrases in their lemma forms, and iii) the Summary Evaluator (KpEval) for evaluating automatic evaluator that scoring the summary based on counting the summaries. The KpEval relies on the keyphrases since they matched keyphrases occur between the peer summary and one convey the most important concepts of a text. In the or more reference summaries. The remaining of this paper is evaluation process, the keyphrases are used in their lemma organized as follows: Section 2 reviews the previous works; form as the matching text unit. The system was applied to Section 3 the proposed keyphrase based summary evaluator; evaluate different summaries of Arabic multi-document data Section 4 discusses the performance evaluation; and section 5 set presented at TAC2011.
    [Show full text]
  • Latent Semantic Analysis and the Construction of Coherent Extracts
    Latent Semantic Analysis and the Construction of Coherent Extracts Tristan Miller German Research Center for Artificial Intelligence0 Erwin-Schrodinger-Straße¨ 57, D-67663 Kaiserslautern [email protected] Keywords: automatic summarization, latent semantic analy- many of these techniques are tied to a particular sis, LSA, coherence, extracts language or require resources such as a list of dis- Abstract course keywords and a manually marked-up cor- pus; others are constrained in the type of summary We describe a language-neutral au- they can generate (e.g., general-purpose vs. query- tomatic summarization system which focussed). aims to produce coherent extracts. It In this paper, we present a new, recursive builds an initial extract composed solely method for automatic text summarization which of topic sentences, and then recursively aims to preserve both the topic coverage and fills in the topical lacunae by provid- the coherence of the source document, yet has ing linking material between semanti- minimal reliance on language-specific NLP tools. cally dissimilar sentences. While exper- Only word- and sentence-boundary detection rou- iments with human judges did not prove tines are required. The system produces general- a statistically significant increase in tex- purpose extracts of single documents, though it tual coherence with the use of a latent should not be difficult to adapt the technique semantic analysis module, we found a to query-focussed summarization, and may also strong positive correlation between co- be of use in improving the coherence of multi- herence and overall summary quality. document summaries. 2 Latent semantic analysis 1 Introduction Our system fits within the general category of IR- A major problem with automatically-produced based systems, but rather than comparing text with summaries in general, and extracts in particular, the standard vector-space model, we employ la- is that the output text often lacks fluency and orga- tent semantic analysis (LSA) [Deerwester et al., nization.
    [Show full text]
  • Soft Similarity and Soft Cosine Measure: Similarity of Features in Vector Space Model
    Soft Similarity and Soft Cosine Measure: Similarity of Features in Vector Space Model Grigori Sidorov1, Alexander Gelbukh1, Helena Gomez-Adorno1, and David Pinto2 1 Centro de Investigacion en Computacion, Instituto Politecnico Nacional, Mexico D.F., Mexico 2 Facultad de Ciencias de la Computacion, Benemerita Universidad Autonoma de Puebla, Puebla, Mexico {sidorov,gelbukh}@cic.ipn.mx, [email protected], [email protected] Abstract. We show how to consider similarity between 1 Introduction features for calculation of similarity of objects in the Vec­ tor Space Model (VSM) for machine learning algorithms Computation of similarity of specific objects is a basic and other classes of methods that involve similarity be­ task of many methods applied in various problems in tween objects. Unlike LSA, we assume that similarity natural language processing and many other fields. In between features is known (say, from a synonym dictio­ natural language processing, text similarity plays crucial nary) and does not need to be learned from the data. role in many tasks from plagiarism detection [18] and We call the proposed similarity measure soft similarity. question answering [3] to sentiment analysis [14-16]. Similarity between features is common, for example, in The most common manner to represent objects is natural language processing: words, n-grams, or syn­ the Vector Space Model (VSM) [17]. In this model, the tactic n-grams can be somewhat different (which makes objects are represented as vectors of values of features. them different features) but still have much in common: The features characterize each object and have numeric for example, words “play” and “game” are different but values.
    [Show full text]
  • Leveraging Word Embeddings for Spoken Document Summarization
    Leveraging Word Embeddings for Spoken Document Summarization Kuan-Yu Chen*†, Shih-Hung Liu*, Hsin-Min Wang*, Berlin Chen#, Hsin-Hsi Chen† *Institute of Information Science, Academia Sinica, Taiwan #National Taiwan Normal University, Taiwan †National Taiwan University, Taiwan * # † {kychen, journey, whm}@iis.sinica.edu.tw, [email protected], [email protected] Abstract without human annotations involved. Popular methods include Owing to the rapidly growing multimedia content available on the vector space model (VSM) [9], the latent semantic analysis the Internet, extractive spoken document summarization, with (LSA) method [9], the Markov random walk (MRW) method the purpose of automatically selecting a set of representative [10], the maximum marginal relevance (MMR) method [11], sentences from a spoken document to concisely express the the sentence significant score method [12], the unigram most important theme of the document, has been an active area language model-based (ULM) method [4], the LexRank of research and experimentation. On the other hand, word method [13], the submodularity-based method [14], and the embedding has emerged as a newly favorite research subject integer linear programming (ILP) method [15]. Statistical because of its excellent performance in many natural language features may include the term (word) frequency, linguistic processing (NLP)-related tasks. However, as far as we are score, recognition confidence measure, and prosodic aware, there are relatively few studies investigating its use in information. In contrast, supervised sentence classification extractive text or speech summarization. A common thread of methods, such as the Gaussian mixture model (GMM) [9], the leveraging word embeddings in the summarization process is Bayesian classifier (BC) [16], the support vector machine to represent the document (or sentence) by averaging the word (SVM) [17], and the conditional random fields (CRFs) [18], embeddings of the words occurring in the document (or usually formulate sentence selection as a binary classification sentence).
    [Show full text]
  • Evaluating Vector-Space Models of Word Representation, Or, the Unreasonable Effectiveness of Counting Words Near Other Words
    Evaluating Vector-Space Models of Word Representation, or, The Unreasonable Effectiveness of Counting Words Near Other Words Aida Nematzadeh, Stephan C. Meylan, and Thomas L. Griffiths University of California, Berkeley fnematzadeh, smeylan, tom griffi[email protected] Abstract angle between word vectors (e.g., Mikolov et al., 2013b; Pen- nington et al., 2014). Vector-space models of semantics represent words as continuously-valued vectors and measure similarity based on In this paper, we examine whether these constraints im- the distance or angle between those vectors. Such representa- ply that Word2Vec and GloVe representations suffer from the tions have become increasingly popular due to the recent de- same difficulty as previous vector-space models in capturing velopment of methods that allow them to be efficiently esti- mated from very large amounts of data. However, the idea human similarity judgments. To this end, we evaluate these of relating similarity to distance in a spatial representation representations on a set of tasks adopted from Griffiths et al. has been criticized by cognitive scientists, as human similar- (2007) in which the authors showed that the representations ity judgments have many properties that are inconsistent with the geometric constraints that a distance metric must obey. We learned by another well-known vector-space model, Latent show that two popular vector-space models, Word2Vec and Semantic Analysis (Landauer and Dumais, 1997), were in- GloVe, are unable to capture certain critical aspects of human consistent with patterns of semantic similarity demonstrated word association data as a consequence of these constraints. However, a probabilistic topic model estimated from a rela- in human word association data.
    [Show full text]
  • Anomalous Behavior Detection Framework Using HTM-Based Semantic Folding Technique
    Hindawi Computational and Mathematical Methods in Medicine Volume 2021, Article ID 5585238, 14 pages https://doi.org/10.1155/2021/5585238 Research Article Anomalous Behavior Detection Framework Using HTM-Based Semantic Folding Technique Hamid Masood Khan ,1 Fazal Masud Khan,1 Aurangzeb Khan,2 Muhammad Zubair Asghar ,1 and Daniyal M. Alghazzawi 3 1Institute of Computing and Information Technology, Gomal University, D.I.Khan, Pakistan 2Department of Computer Science, University of Science and Technology, Bannu, Pakistan 3Information Systems Department, Faculty of Computing and Information Technology, King Abdulaziz University, Jeddah, Saudi Arabia Correspondence should be addressed to Muhammad Zubair Asghar; [email protected] Received 23 January 2021; Revised 17 February 2021; Accepted 26 February 2021; Published 16 March 2021 Academic Editor: Waqas Haider Bangyal Copyright © 2021 Hamid Masood Khan et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. Upon the working principles of the human neocortex, the Hierarchical Temporal Memory model has been developed which is a proposed theoretical framework for sequence learning. Both categorical and numerical types of data are handled by HTM. Semantic Folding Theory (SFT) is based on HTM to represent a data stream for processing in the form of sparse distributed representation (SDR). For natural language perception and production, SFT delivers a solid structural background for semantic evidence description to the fundamentals of the semantic foundation during the phase of language learning. Anomalies are the patterns from data streams that do not follow the expected behavior.
    [Show full text]
  • Approximate Pattern Matching Using Hierarchical Graph Construction and Sparse Distributed Representation
    Portland State University PDXScholar Electrical and Computer Engineering Faculty Publications and Presentations Electrical and Computer Engineering 7-23-2019 Approximate Pattern Matching using Hierarchical Graph Construction and Sparse Distributed Representation Aakanksha Mathuria Portland State University, [email protected] Dan Hammerstrom Portland State University Follow this and additional works at: https://pdxscholar.library.pdx.edu/ece_fac Part of the Engineering Commons Let us know how access to this document benefits ou.y Citation Details "Approximate Pattern Matching using Hierarchical Graph Construction and Sparse Distributed Representation," A. Mathuria, D.W. Hammerstrom, International Conference on Neuromorphic Systems, Knoxville, TN, July 23-5, 2019 This Conference Proceeding is brought to you for free and open access. It has been accepted for inclusion in Electrical and Computer Engineering Faculty Publications and Presentations by an authorized administrator of PDXScholar. Please contact us if we can make this document more accessible: [email protected]. Approximate Pattern Matching using Hierarchical Graph Construction and Sparse Distributed Representation Aakanksha Mathuria Electrical and Computer Engineering Portland State University USA [email protected] Dan W. Hammerstrom Electrical and Computer Engineering Portland State University USA [email protected] ABSTRACT training on large numbers of images. None of these techniques actually captures the spatial relationships of the low level or high- With recent developments in deep networks, there have been level features, which biological networks appear to do. significant advances in visual object detection and Efficient graph representations capture the higher order recognition. However, some of these networks are still easily information content of the objects and provide algorithmic benefits fooled/hacked and have shown “bag of features” failures.
    [Show full text]