DOCSLIB.ORG

Global-Local Word Embedding for Text Classification

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text

Load more

Global-Local Word Embedding for Text Classification by Mehran Kamkarhaghighi A Thesis Submitted to the School of Graduate and Postdoctoral Studies in Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy in Electrical and Computer Engineering Department of Electrical, Computer and Software Engineering Faculty of Engineering and Applied Science University of Ontario Institute of Technology Oshawa, Ontario, Canada April 2019 © Mehran Kamkarhaghighi, 2019 THESIS EXAMINATION INFORMATION Submitted by: Mehran Kamkarhaghighi Doctor of Philosophy in Electrical and Computer Engineering Thesis title: Global-Local Word Embedding for Text Classification An oral defense of this thesis took place on March 27, 2019 in front of the following examining committee: Examining Committee: Chair of Examining Committee Dr. Walid Morsi Ibrahim Research Supervisor Dr. Masoud Makrehchi Examining Committee Member Dr. Shahryar Rahnamayan Examining Committee Member Dr. Qusay H. Mahmoud University Examiner Dr. Jeremy S. Bradbury External Examiner Dr. Chen (Cherie) Ding, Ryerson University The above committee determined that the thesis is acceptable in form and content and that a satisfactory knowledge of the field covered by the thesis was demonstrated by the candidate during an oral examination. A signed copy of the Certificate of Approval is available from the School of Graduate and Postdoctoral Studies. I Abstract Only humans can understand and comprehend the actual meaning that underlies natural written language, whereas machines can form semantic relationships only after humans have provided the parameters that are necessary to model the meaning. To enable computer models to access the underlying meaning in written language, accurate and sufficient document representation is crucial. Recent word embedding approaches have drawn much attention to text mining research. One of the main benefits of such approaches is the use of global corpuses with the generation of pre-trained word vectors. Although very effective, these approaches have their disadvantages, namely sole reliance on pre-trained word vectors that may neglect the local context and increase word ambiguity. In this thesis, four new document representation approaches are introduced to mitigate the risk of word ambiguity and inject a local context into globally pre-trained word vectors. The proposed approaches, which are frameworks for document representation while using word embedding learning features for the task of text classification, are: Content Tree Word Embedding; Composed Maximum Spanning Content Tree; Embedding-based Word Clustering; and Autoencoder-based Word Embedding. The results show improvement in the F_score accuracy measure for a document classification task applied to IMDB Movie Reviews, Hate Speech Identification, 20 Newsgroups, Reuters-21578, and AG News as benchmark datasets in comparison to using three deep learning-based word embedding approaches, namely GloVe, Word2Vec, and fastText, as well as two other document representations: LSA and Random word embedding. Keywords: Document Representation; Word Embedding; Text Classification; Deep Learning; Neural Networks II AUTHOR’S DECLARATION I hereby declare that this thesis consists of original work of which I have authored. This is a true copy of the thesis, including any required final revisions, as accepted by my examiners. I authorize the University of Ontario Institute of Technology to lend this thesis to other institutions or individuals for the purpose of scholarly research. I further authorize the University of Ontario Institute of Technology to reproduce this thesis by photocopying or by other means, in total or in part, at the request of other institutions or individuals for the purpose of scholarly research. I understand that my thesis will be made electronically available to the public. Mehran Kamkarhaghighi III Dedicated to my wife, without whose support and encouragement none of this would have been possible. IV Acknowledgements Firstly, I would like to express my sincere gratitude to my advisor, Dr. Masoud Makrehchi, for offering me the opportunity to work with him and for continuously supporting my Ph.D. studies and related research, as well as for his patience, motivation, and sharing of his immense knowledge. I could not imagine having a better advisor and mentor for my doctoral studies. In addition to my advisor, I would like to thank the rest of my thesis committee, Dr. Shahryar Rahnamayan, Dr. Qusay H. Mahmoud, and Dr. Jeremy S. Bradbury, for not only their insightful comments and encouragement, but also for all the hard questions that encouraged me to widen my research from various perspectives. My sincere thanks also go to my friends in SciLab, Dr. Somayyeh (Bahar) Aghababaei, Mahboubeh (Tara) Ahmadalinezhad, Fateme Azimlou, Iuliia Chepurna, Dr. Eren Gultepe, Neil Seward, and Afsaneh Towhidi, without whose precious support it would not have been possible to conduct this research. I would also like to extend my thanks to my dear friends, Dr. Roozbeh Jalali and Dr. Reza Mohammad Alizadeh, for their encouragement and support throughout my research. I also wish to express my appreciation to Mrs. Catherine Lee, for editing and proofreading this thesis. Finally, I would like to express my greatest gratitude to my family: my dearest wife, Solmaz, and my father and my mother for their unconditional support during this period of my life. V STATEMENT OF CONTRIBUTIONS I hereby certify that I am the sole author of this thesis and I have used standard referencing practices to acknowledge ideas, research techniques, or other materials that belong to others. Furthermore, I hereby certify that I am the sole source of the creative works and/or inventive knowledge described in this thesis. In all cases, I (Mehran Kamkarhaghighi) was the main investigator for each of the co- authored studies. Part of the work described in Chapter 2 as the literature review was previously published as: M. Kamkarhaghighi, E. Gultepe, and M. Makrehchi, "Deep Learning for Document Representation," in Handbook of Deep Learning Applications: Springer, 2019, pp. 101- 110. The other part of the work described in Chapter 3 as the CTWE approach has been published as: M. Kamkarhaghighi and M. Makrehchi, "Content tree word embedding for document representation," Expert Systems with Applications, vol. 90, pp. 241-249, 2017. VI Table of Contents ABSTRACT .............................................................................................................................................. II ACKNOWLEDGEMENTS .................................................................................................................... V LIST OF TABLES ................................................................................................................................... X LIST OF FIGURES .............................................................................................................................. XII LIST OF ABBREVIATIONS AND SYMBOLS .............................................................................. XIII CHAPTER 1. INTRODUCTION .................................................................................................... 1 1.1 Overview .................................................................................................................................. 1 1.2 Problem Statement .................................................................................................................... 3 1.3 Research Questions ................................................................................................................... 5 1.4 Research Objectives ................................................................................................................. 5 1.5 Research Contribution .............................................................................................................. 6 1.6 Thesis Outline ........................................................................................................................... 6 CHAPTER 2. LITERATURE REVIEW ........................................................................................ 7 2.1 Introduction .............................................................................................................................. 7 2.2 Traditional Document Representations Methods ..................................................................... 7 2.3 Word2Vec ............................................................................................................................... 10 2.4 GloVe ...................................................................................................................................... 11 2.5 fastText ................................................................................................................................... 12 2.6 LSA ......................................................................................................................................... 13 2.7 Doc2Vec ................................................................................................................................. 14 2.8 Other Word Embedding Approaches ..................................................................................... 16 2.9 Taxonomy Induction ............................................................................................................... 19 2.10 Summary ................................................................................................................................
Recommended publications
  • Arxiv:2002.06235V1

    Arxiv:2002.06235V1

    Semantic Relatedness and Taxonomic Word Embeddings Magdalena Kacmajor John D. Kelleher Innovation Exchange ADAPT Research Centre IBM Ireland Technological University Dublin [email protected] [email protected] Filip Klubickaˇ Alfredo Maldonado ADAPT Research Centre Underwriters Laboratories Technological University Dublin [email protected] [email protected] Abstract This paper1 connects a series of papers dealing with taxonomic word embeddings. It begins by noting that there are different types of semantic relatedness and that different lexical representa- tions encode different forms of relatedness. A particularly important distinction within semantic relatedness is that of thematic versus taxonomic relatedness. Next, we present a number of ex- periments that analyse taxonomic embeddings that have been trained on a synthetic corpus that has been generated via a random walk over a taxonomy. These experiments demonstrate how the properties of the synthetic corpus, such as the percentage of rare words, are affected by the shape of the knowledge graph the corpus is generated from. Finally, we explore the interactions between the relative sizes of natural and synthetic corpora on the performance of embeddings when taxonomic and thematic embeddings are combined. 1 Introduction Deep learning has revolutionised natural language processing over the last decade. A key enabler of deep learning for natural language processing has been the development of word embeddings. One reason for this is that deep learning intrinsically involves the use of neural network models and these models only work with numeric inputs. Consequently, applying deep learning to natural language processing first requires developing a numeric representation for words. Word embeddings provide a way of creating numeric representations that have proven to have a number of advantages over traditional numeric rep- resentations of language.
  • Text Mining Course for KNIME Analytics Platform

    Text Mining Course for KNIME Analytics Platform

    Text Mining Course for KNIME Analytics Platform KNIME AG Copyright © 2018 KNIME AG Table of Contents 1. The Open Analytics Platform 2. The Text Processing Extension 3. Importing Text 4. Enrichment 5. Preprocessing 6. Transformation 7. Classification 8. Visualization 9. Clustering 10. Supplementary Workflows Licensed under a Creative Commons Attribution- ® Copyright © 2018 KNIME AG 2 Noncommercial-Share Alike license 1 https://creativecommons.org/licenses/by-nc-sa/4.0/ Overview KNIME Analytics Platform Licensed under a Creative Commons Attribution- ® Copyright © 2018 KNIME AG 3 Noncommercial-Share Alike license 1 https://creativecommons.org/licenses/by-nc-sa/4.0/ What is KNIME Analytics Platform? • A tool for data analysis, manipulation, visualization, and reporting • Based on the graphical programming paradigm • Provides a diverse array of extensions: • Text Mining • Network Mining • Cheminformatics • Many integrations, such as Java, R, Python, Weka, H2O, etc. Licensed under a Creative Commons Attribution- ® Copyright © 2018 KNIME AG 4 Noncommercial-Share Alike license 2 https://creativecommons.org/licenses/by-nc-sa/4.0/ Visual KNIME Workflows NODES perform tasks on data Not Configured Configured Outputs Inputs Executed Status Error Nodes are combined to create WORKFLOWS Licensed under a Creative Commons Attribution- ® Copyright © 2018 KNIME AG 5 Noncommercial-Share Alike license 3 https://creativecommons.org/licenses/by-nc-sa/4.0/ Data Access • Databases • MySQL, MS SQL Server, PostgreSQL • any JDBC (Oracle, DB2, …) • Files • CSV, txt
  • Combining Word Embeddings with Semantic Resources

    Combining Word Embeddings with Semantic Resources

    AutoExtend: Combining Word Embeddings with Semantic Resources Sascha Rothe∗ LMU Munich Hinrich Schutze¨ ∗ LMU Munich We present AutoExtend, a system that combines word embeddings with semantic resources by learning embeddings for non-word objects like synsets and entities and learning word embeddings that incorporate the semantic information from the resource. The method is based on encoding and decoding the word embeddings and is flexible in that it can take any word embeddings as input and does not need an additional training corpus. The obtained embeddings live in the same vector space as the input word embeddings. A sparse tensor formalization guar- antees efficiency and parallelizability. We use WordNet, GermaNet, and Freebase as semantic resources. AutoExtend achieves state-of-the-art performance on Word-in-Context Similarity and Word Sense Disambiguation tasks. 1. Introduction Unsupervised methods for learning word embeddings are widely used in natural lan- guage processing (NLP). The only data these methods need as input are very large corpora. However, in addition to corpora, there are many other resources that are undoubtedly useful in NLP, including lexical resources like WordNet and Wiktionary and knowledge bases like Wikipedia and Freebase. We will simply refer to these as resources. In this article, we present AutoExtend, a method for enriching these valuable resources with embeddings for non-word objects they describe; for example, Auto- Extend enriches WordNet with embeddings for synsets. The word embeddings and the new non-word embeddings live in the same vector space. Many NLP applications benefit if non-word objects described by resources—such as synsets in WordNet—are also available as embeddings.
  • Application of Machine Learning in Automatic Sentiment Recognition from Human Speech

    Application of Machine Learning in Automatic Sentiment Recognition from Human Speech

    Application of Machine Learning in Automatic Sentiment Recognition from Human Speech Zhang Liu Ng EYK Anglo-Chinese Junior College College of Engineering Singapore Nanyang Technological University (NTU) Singapore [email protected] Abstract— Opinions and sentiments are central to almost all human human activities and have a wide range of applications. As activities and have a wide range of applications. As many decision many decision makers turn to social media due to large makers turn to social media due to large volume of opinion data volume of opinion data available, efficient and accurate available, efficient and accurate sentiment analysis is necessary to sentiment analysis is necessary to extract those data. Business extract those data. Hence, text sentiment analysis has recently organisations in different sectors use social media to find out become a popular field and has attracted many researchers. However, consumer opinions to improve their products and services. extracting sentiments from audio speech remains a challenge. This project explores the possibility of applying supervised Machine Political party leaders need to know the current public Learning in recognising sentiments in English utterances on a sentiment to come up with campaign strategies. Government sentence level. In addition, the project also aims to examine the effect agencies also monitor citizens’ opinions on social media. of combining acoustic and linguistic features on classification Police agencies, for example, detect criminal intents and cyber accuracy. Six audio tracks are randomly selected to be training data threats by analysing sentiment valence in social media posts. from 40 YouTube videos (monologue) with strong presence of In addition, sentiment information can be used to make sentiments.
  • Word-Like Character N-Gram Embedding

    Word-Like Character N-Gram Embedding

    Word-like character n-gram embedding Geewook Kim and Kazuki Fukui and Hidetoshi Shimodaira Department of Systems Science, Graduate School of Informatics, Kyoto University Mathematical Statistics Team, RIKEN Center for Advanced Intelligence Project fgeewook, [email protected], [email protected] Abstract Table 1: Top-10 2-grams in Sina Weibo and 4-grams in We propose a new word embedding method Japanese Twitter (Experiment 1). Words are indicated called word-like character n-gram embed- by boldface and space characters are marked by . ding, which learns distributed representations FNE WNE (Proposed) Chinese Japanese Chinese Japanese of words by embedding word-like character n- 1 ][ wwww 自己 フォロー grams. Our method is an extension of recently 2 。␣ ！！！！ 。␣ ありがと proposed segmentation-free word embedding, 3 ！␣ ありがと ][ wwww 4 .. りがとう 一个 ！！！！ which directly embeds frequent character n- 5 ]␣ ございま 微博 めっちゃ grams from a raw corpus. However, its n-gram 6 。。 うござい 什么 んだけど vocabulary tends to contain too many non- 7 ，我 とうござ 可以 うござい 8 ！！ ざいます 没有 line word n-grams. We solved this problem by in- 9 ␣我 がとうご 吗？ 2018 troducing an idea of expected word frequency. 10 了， ください 哈哈 じゃない Compared to the previously proposed meth- ods, our method can embed more words, along tion tools are used to determine word boundaries with the words that are not included in a given in the raw corpus. However, these segmenters re- basic word dictionary. Since our method does quire rich dictionaries for accurate segmentation, not rely on word segmentation with rich word which are expensive to prepare and not always dictionaries, it is especially effective when the available.
  • Text KM & Text Mining

    Text KM & Text Mining

    Text KM & Text Mining An Introduction to Managing Knowledge in Unstructured Natural Language Documents Dekai Wu HKUST Human Language Technology Center Department of Computer Science and Engineering University of Science and Technology Hong Kong http://www.cs.ust.hk/~dekai © 2008 Dekai Wu Lecture Objectives Introduction to the concept of Text KM and Text Mining (TM) How to exploit knowledge encoded in text form How text mining is different from data mining Introduction to the various aspects of Natural Language Processing (NLP) Introduction to the different tools and methods available for TM HKUST Human Language Technology Center © 2008 Dekai Wu Textual Knowledge Management Text KM oversees the storage, capturing and sharing of knowledge encoded in unstructured natural language documents 80-90% of an organization’s explicit knowledge resides in plain English (Chinese, Japanese, Spanish, …) documents – not in structured relational databases! Case libraries are much more reasonably stored as natural language documents, than encoded into relational databases Most knowledge encoded as text will never pass through explicit KM processes (eg, email) HKUST Human Language Technology Center © 2008 Dekai Wu Text Mining Text Mining analyzes unstructured natural language documents to extract targeted types of knowledge Extracts knowledge that can then be inserted into databases, thereby facilitating structured data mining techniques Provides a more natural user interface for entering knowledge, for both employees and developers Reduces
  • Learned in Speech Recognition: Contextual Acoustic Word Embeddings

    Learned in Speech Recognition: Contextual Acoustic Word Embeddings

    LEARNED IN SPEECH RECOGNITION: CONTEXTUAL ACOUSTIC WORD EMBEDDINGS Shruti Palaskar∗, Vikas Raunak∗ and Florian Metze Carnegie Mellon University, Pittsburgh, PA, U.S.A. fspalaska j vraunak j fmetze [email protected] ABSTRACT model [10, 11, 12] trained for direct Acoustic-to-Word (A2W) speech recognition [13]. Using this model, we jointly learn to End-to-end acoustic-to-word speech recognition models have re- automatically segment and classify input speech into individual cently gained popularity because they are easy to train, scale well to words, hence getting rid of the problem of chunking or requiring large amounts of training data, and do not require a lexicon. In addi- pre-defined word boundaries. As our A2W model is trained at the tion, word models may also be easier to integrate with downstream utterance level, we show that we can not only learn acoustic word tasks such as spoken language understanding, because inference embeddings, but also learn them in the proper context of their con- (search) is much simplified compared to phoneme, character or any taining sentence. We also evaluate our contextual acoustic word other sort of sub-word units. In this paper, we describe methods embeddings on a spoken language understanding task, demonstrat- to construct contextual acoustic word embeddings directly from a ing that they can be useful in non-transcription downstream tasks. supervised sequence-to-sequence acoustic-to-word speech recog- Our main contributions in this paper are the following: nition model using the learned attention distribution. On a suite 1. We demonstrate the usability of attention not only for aligning of 16 standard sentence evaluation tasks, our embeddings show words to acoustic frames without any forced alignment but also for competitive performance against a word2vec model trained on the constructing Contextual Acoustic Word Embeddings (CAWE).
  • An Evaluation of Machine Learning Approaches to Natural Language Processing for Legal Text Classification

    An Evaluation of Machine Learning Approaches to Natural Language Processing for Legal Text Classification

    Imperial College London Department of Computing An Evaluation of Machine Learning Approaches to Natural Language Processing for Legal Text Classification Supervisors: Author: Prof Alessandra Russo Clavance Lim Nuri Cingillioglu Submitted in partial fulfillment of the requirements for the MSc degree in Computing Science of Imperial College London September 2019 Contents Abstract 1 Acknowledgements 2 1 Introduction 3 1.1 Motivation .................................. 3 1.2 Aims and objectives ............................ 4 1.3 Outline .................................... 5 2 Background 6 2.1 Overview ................................... 6 2.1.1 Text classification .......................... 6 2.1.2 Training, validation and test sets ................. 6 2.1.3 Cross validation ........................... 7 2.1.4 Hyperparameter optimization ................... 8 2.1.5 Evaluation metrics ......................... 9 2.2 Text classification pipeline ......................... 14 2.3 Feature extraction ............................. 15 2.3.1 Count vectorizer .......................... 15 2.3.2 TF-IDF vectorizer ......................... 16 2.3.3 Word embeddings .......................... 17 2.4 Classifiers .................................. 18 2.4.1 Naive Bayes classifier ........................ 18 2.4.2 Decision tree ............................ 20 2.4.3 Random forest ........................... 21 2.4.4 Logistic regression ......................... 21 2.4.5 Support vector machines ...................... 22 2.4.6 k-Nearest Neighbours .......................
  • Fasttext.Zip: Compressing Text Classification Models

    Fasttext.Zip: Compressing Text Classification Models

    Under review as a conference paper at ICLR 2017 FASTTEXT.ZIP: COMPRESSING TEXT CLASSIFICATION MODELS Armand Joulin, Edouard Grave, Piotr Bojanowski, Matthijs Douze, Herve´ Jegou´ & Tomas Mikolov Facebook AI Research fajoulin,egrave,bojanowski,matthijs,rvj,[email protected] ABSTRACT We consider the problem of producing compact architectures for text classifica- tion, such that the full model fits in a limited amount of memory. After consid- ering different solutions inspired by the hashing literature, we propose a method built upon product quantization to store word embeddings. While the original technique leads to a loss in accuracy, we adapt this method to circumvent quan- tization artefacts. Combined with simple approaches specifically adapted to text classification, our approach derived from fastText requires, at test time, only a fraction of the memory compared to the original FastText, without noticeably sacrificing quality in terms of classification accuracy. Our experiments carried out on several benchmarks show that our approach typically requires two orders of magnitude less memory than fastText while being only slightly inferior with respect to accuracy. As a result, it outperforms the state of the art by a good margin in terms of the compromise between memory usage and accuracy. 1 INTRODUCTION Text classification is an important problem in Natural Language Processing (NLP). Real world use- cases include spam filtering or e-mail categorization. It is a core component in more complex sys- tems such as search and ranking. Recently, deep learning techniques based on neural networks have achieved state of the art results in various NLP applications. One of the main successes of deep learning is due to the effectiveness of recurrent networks for language modeling and their application to speech recognition and machine translation (Mikolov, 2012).
  • Knowledge-Powered Deep Learning for Word Embedding

    Knowledge-Powered Deep Learning for Word Embedding

    Knowledge-Powered Deep Learning for Word Embedding Jiang Bian, Bin Gao, and Tie-Yan Liu Microsoft Research {jibian,bingao,tyliu}@microsoft.com Abstract. The basis of applying deep learning to solve natural language process- ing tasks is to obtain high-quality distributed representations of words, i.e., word embeddings, from large amounts of text data. However, text itself usually con- tains incomplete and ambiguous information, which makes necessity to leverage extra knowledge to understand it. Fortunately, text itself already contains well- defined morphological and syntactic knowledge; moreover, the large amount of texts on the Web enable the extraction of plenty of semantic knowledge. There- fore, it makes sense to design novel deep learning algorithms and systems in order to leverage the above knowledge to compute more effective word embed- dings. In this paper, we conduct an empirical study on the capacity of leveraging morphological, syntactic, and semantic knowledge to achieve high-quality word embeddings. Our study explores these types of knowledge to define new basis for word representation, provide additional input information, and serve as auxiliary supervision in deep learning, respectively. Experiments on an analogical reason- ing task, a word similarity task, and a word completion task have all demonstrated that knowledge-powered deep learning can enhance the effectiveness of word em- bedding. 1 Introduction With rapid development of deep learning techniques in recent years, it has drawn in- creasing attention to train complex and deep models on large amounts of data, in order to solve a wide range of text mining and natural language processing (NLP) tasks [4, 1, 8, 13, 19, 20].
  • Supervised and Unsupervised Word Sense Disambiguation on Word Embedding Vectors of Unambiguous Synonyms

    Supervised and Unsupervised Word Sense Disambiguation on Word Embedding Vectors of Unambiguous Synonyms

    Supervised and Unsupervised Word Sense Disambiguation on Word Embedding Vectors of Unambiguous Synonyms Aleksander Wawer Agnieszka Mykowiecka Institute of Computer Science Institute of Computer Science PAS PAS Jana Kazimierza 5 Jana Kazimierza 5 01-248 Warsaw, Poland 01-248 Warsaw, Poland [email protected] [email protected] Abstract Unsupervised WSD algorithms aim at resolv- ing word ambiguity without the use of annotated This paper compares two approaches to corpora. There are two popular categories of word sense disambiguation using word knowledge-based algorithms. The first one orig- embeddings trained on unambiguous syn- inates from the Lesk (1986) algorithm, and ex- onyms. The first one is an unsupervised ploit the number of common words in two sense method based on computing log proba- definitions (glosses) to select the proper meaning bility from sequences of word embedding in a context. Lesk algorithm relies on the set of vectors, taking into account ambiguous dictionary entries and the information about the word senses and guessing correct sense context in which the word occurs. In (Basile et from context. The second method is super- al., 2014) the concept of overlap is replaced by vised. We use a multilayer neural network similarity represented by a DSM model. The au- model to learn a context-sensitive transfor- thors compute the overlap between the gloss of mation that maps an input vector of am- the meaning and the context as a similarity mea- biguous word into an output vector repre- sure between their corresponding vector represen- senting its sense. We evaluate both meth- tations in a semantic space.
  • Local Homology of Word Embeddings

    Local Homology of Word Embeddings

    Local Homology of Word Embeddings Tadas Temcinasˇ 1 Abstract Intuitively, stratification is a decomposition of a topologi- Topological data analysis (TDA) has been widely cal space into manifold-like pieces. When thinking about used to make progress on a number of problems. stratification learning and word embeddings, it seems intu- However, it seems that TDA application in natural itive that vectors of words corresponding to the same broad language processing (NLP) is at its infancy. In topic would constitute a structure, which we might hope this paper we try to bridge the gap by arguing why to be a manifold. Hence, for example, by looking at the TDA tools are a natural choice when it comes to intersections between those manifolds or singularities on analysing word embedding data. We describe the manifolds (both of which can be recovered using local a parallelisable unsupervised learning algorithm homology-based algorithms (Nanda, 2017)) one might hope based on local homology of datapoints and show to find vectors of homonyms like ‘bank’ (which can mean some experimental results on word embedding either a river bank, or a financial institution). This, in turn, data. We see that local homology of datapoints in has potential to help solve the word sense disambiguation word embedding data contains some information (WSD) problem in NLP, which is pinning down a particular that can potentially be used to solve the word meaning of a word used in a sentence when the word has sense disambiguation problem. multiple meanings. In this work we present a clustering algorithm based on lo- cal homology, which is more relaxed1 that the stratification- 1.