DOCSLIB.ORG

Analysis of Word Embeddings Using Fuzzy Clustering

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Analysis of Word Embeddings Using Fuzzy Clustering Analysis of Word Embeddings using Fuzzy Clustering Shahin Atakishiyev Marek Z. Reformat Department of Electrical and Computer Engineering Department of Electrical and Computer Engineering University of Alberta University of Alberta Edmonton, AB, Canada Edmonton, AB, Canada [email protected] [email protected] Abstract – In data dominated systems and applications, a concept techniques. High-dimensional spaces often have a devastating of representing words in a numerical format has gained a lot of effect on data clustering in terms of performance and quality; attention. There are a few approaches used to generate such a this issue is regarded as the curse of the dimensionality. representation. An interesting issue that should be considered is the ability of such representations – called embeddings – to imitate Our study sheds some light on the comparative analysis of the human-based semantic similarity between words. In this study, above fuzzy clustering methods to observe if and how the we perform a fuzzy-based analysis of vector representations of results of fuzzy clustering change with the dimensionality of words, i.e., word embeddings. We use two popular fuzzy clustering word embeddings. Additionally, we illustrate ‘usefulness’ of algorithms on count-based word embeddings, known as GloVe, of fuzzy clustering via analysis of degrees of belongings of words different dimensionality. Words from WordSim-353, called the gold standard, are represented as vectors and clustered. The to different clusters. results indicate that fuzzy clustering algorithms are very sensitive The paper is structured as follows: Section II contains the to high-dimensional data, and parameter tuning can dramatically description of fuzzy C-Means (FCM) and fuzzy Gustafson- change their performance. We show that by adjusting the value of Kessel (FGK) algorithms. In Section III, we provide an the fuzzifier parameter, fuzzy clustering can be successfully applied to vectors of high – up to one hundred – dimensions. overview of the validity indices applied for fuzzy clustering Additionally, we illustrate that fuzzy clustering allows to provide processes. The theoretical background behind the construction interesting results regarding membership of words to different process of GloVe embeddings is covered in Section IV. Section clusters. V outlines the methodology, while Section VI shows our experimental results. Finally, the obtained conclusions are Keywords -- fuzzy clustering, fuzzy C-means, fuzzy Gustafson- presented in Section VII. Kessel, cluster validity, word embeddings, word vectors II. FUZZY CLUSTERING Zadeh’s fuzzy sets theory [5] has triggered a number of studies I. INTRODUCTION focused on the application of theoretical and empirical concepts of fuzzy logic to data clustering. In contrast to hard clustering Word embeddings become an essential element of methods that techniques, where one point is assigned exactly to only one focus on analysis and comparison of texts. One of the most cluster, fuzzy clustering allows data points to pertain to several popular embeddings is GloVe [10]. The embedding is obtained clusters with different grades of membership. via analysis of word-word co-occurrences in a text corpus. A natural question is related to the ability of the embeddings to We have analyzed the behavior of FCM and FGK in clustering represent a human-based semantic similarity of words. of vector representations of words in different dimensions. The details of these clustering methods have been described in the Data clustering is the process of grouping objects in a way that following sections. similarity between data points that belong to the same group (cluster) becomes as high as possible, while similarity between A. Fuzzy C-means clustering points from different groups gets as small as possible. It is an important task in analysis processes and has been successfully Fuzzy C-means algorithm was introduced by Bezdek [6] in applied to pattern recognition [1] [2], image segmentation [3], 1981. It allows an observation to belong to multiple clusters fault diagnosis and search engines [4]. Fuzzy clustering which with varying grades of membership. Having D as the number of allows data points to belong to several numbers of clusters with data points, N as the number of clusters, m as the fuzzifier different membership grades has been proved to have useful parameter, 푥푖 as the i-th data point, 푐푖 as the center of the j-th applications in many areas. Specifically, fuzzy C-Means (FCM) cluster, 휇푖푗 as the membership degree of 푥푖 for the j-th cluster, clustering and its augmented version – fuzzy Gustafson-Kessel FCM aims to minimize (FGK) clustering are the most popular fuzzy clustering 퐷 푁 푚 2 푐 푐 퐽 = ∑ ∑ 휇푖푗 ||푥푖 − 푐푗|| (1) 푁 푁 푚 2 푚 푇 푖=1 푗=1 퐽 = ∑ ∑ 휇푖푘 퐷퐺퐾 = ∑ ∑ 휇푖푘(푥푘 −∨푖) 퐴푖(푥푘 −∨푖) (8) 푘=1 푖=1 푖=1 푖=1 The FCM clustering proceeds in the following way: Like FCM, this optimization is also subject to the following constraints: 1. Cluster membership values 휇푖푗 and initial cluster centers are initialized randomly. 푐 휇푖푘 ∈ [0,1] , ∀푖, 푘 and ∑푖=1 휇푖푘 =1, ∀푘 (9) 2. Cluster centers are computed according to the formula: We see that the computation of the FGK algorithm is more ∑퐷 휇푚푥 푖=1 푖푗 푖 convoluted than FCM clustering. 푐푗 = 퐷 푚 (2) ∑푖=1 휇푖푗 III VALIDITY INDICES 3. Membership grades ( 휇푖푗) are updated in the following way: There are several validity indices to analyze the performance of the fuzzy clustering algorithms. One of them was proposed by 1 푐 Bezdek [6]. It is called fuzzy partition coefficient (FPC). This 휇푖푗 = 2 , 휇푖푗 ∈ [0,1] 푎푛푑 ∑푖=1 휇푖푗 = 1 (3) ||푥 −푐 || index is calculated as follows: 푁 푖 푗 푚−1 ∑1 ( ) 푁 푐 ||푥푖−푐푘|| 1 퐹푃퐶 = ∑ ∑ 휇 2 (10) 푁 푖푘 4. The objective function 퐽 is calculated 푘=1 푖=1 5. The steps 2,3,4 are repeated until the value of the objective function gets less than a specified threshold. FPC changes between [0,1] range and the maximum value indicates the best clustering quality. Another popular index to Fuzzy C-means (FCM) has many useful applications in medical measure fuzzy clustering quality was proposed by Xie and Beni image analysis, pattern recognition, and software quality (XB) in 1991 [9]. It focuses on two properties: cluster prediction [6,7], to name just a few. The most important factors compactness and separation: affecting the performance of this algorithm are the fuzzifier 푐 parameter m, the size and the dimensionality of data. The 푁 푚 2 ∑푘=1(휇푖푘) ||푥푘 −∨푖|| performance analysis of the algorithm for high-dimensional 푋퐵 = ∑ 2 (11) 푁푚푖푛 푖푘||푥푘 −∨푖|| clustering will be discussed in Section VII in detail. 푖=1 B. Fuzzy Gustafson-Kessel clustering The numerator part shows the strength of the compactness of fuzzy clustering, and the denominator shows the strength of Fuzzy Gustafson-Kessel (FGK) extends FCM by introducing an separation between those fuzzy clusters. If a range of clusters adaptive distance norm that allows the algorithm to identify { 푘1, 푘2… 푘푖} is taken, the 푘푖 minimizing this index will be the clusters with different geometrical shapes [8]. The distance optimal number of clusters for the dataset. metric is defined in the following way: IV. GloVe VECTORS 2 푇 퐷퐺퐾 = (푥푘 −∨푖) 퐴푖(푥푘 −∨푖) (4) One of the most known unsupervised learning algorithm to where 퐴푖 itself is computed from a fuzzy covariance matrix of produce vector representations of words is GloVe. It is based on each cluster: a word-word co-occurrence in text corpora. The term stands for Global Vectors as the representation is able to capture global 1/푑 −1 퐴푖 = (휌푖|퐶푖) 퐶푖 , (5) corpus statistics. ∑푁 휇푚(푥 −∨ )푇(푥 −∨ ) A. Overview of GloVe 퐶 = 푘=1 푖푘 푘 푖 푘 푖 (6) 푖 ∑푁 휇푚 푘=1 푖푘 Let us start with defining a word-word co-occurrence counts matrix as X, where 푋 is the number of times the word 푗 exists Here the parameter 휌푖 is the constrained form of the determinant 푖푗 of 퐴푖, and ∨푖 is the center of a specified cluster i. in the context of the word i. Let use denote 푋푖 = ∑푘 푋푖푘 the |퐴푖| = 휌푖 , 휌푖 > 0, ∀푖 (7) number of times any word appears in the context of the word i. Lastly, let 푃푖푗 = 푃(푗|푖) = 푋푖푗/푋푖 become the probability in Enabling the matrix 퐴푖 to change with fixed determinant serves which word j exists in the context of the word i. Using a simple to optimize the shape of clusters by keeping the cluster’s example, we demonstrate how aspects of meanings can be volume constant [8]. Gustafson-Kessel clustering minimizes extracted from the word-word co-occurrences statistics. the following criterion: Pennington et al. show this with good examples. Assuming we have text corpora related to thermodynamics and we may take to handle with these two vectors is to sum and assign the sum words 푖 = 푖푐푒 and 푗 = 푠푡푒푎푚. We investigate the relationship vector as a unique representation of the word: of these words by learning from the co-occurrence probabilities with other sample words. For instance, taking the word 푖푐푒 and 푊푓푖푛푎푙 = 푊 + 푊 ˜ (16) word t=solid, we can expect that 푃푖푡/푃푗푡 will be large. Likewise, if we select words t that are related to steam but not to ice such Summing two sets of vectors into one effectively reflects words that t=gas, then we expect that the value of ratio should be in the embeddings space. The authors have built the vocabulary small. Global vectors try to leverage a series of functions called of most frequent 400,000 words, and made them publicly F that represents those ratios [10] [11].
Load more

Recommended publications
  • Deep Learning Models for Spatio-Temporal Forecasting and Analysis
    UNIVERSITY OF CALIFORNIA, IRVINE Deep Learning Models for Spatio-Temporal Forecasting and Analysis DISSERTATION submitted in partial satisfaction of the requirements for the degree of DOCTOR OF PHILOSOPHY in Computer Science by Reza Asadi Dissertation Committee: Professor Amelia Regan, Chair Professor Michael Dillencourt Professor R. Jayakrishnan 2020 c 2020 Reza Asadi DEDICATION To my family for their love and support ii TABLE OF CONTENTS Page LIST OF FIGURES vi LIST OF TABLES ix ACKNOWLEDGMENTS x VITA xi ABSTRACT OF THE DISSERTATION xiii 1 Introduction 1 1.1 Spatio-temporal traffic data . .1 1.2 Machine learning problems in traffic data . .6 1.2.1 Traffic flow prediction . .7 1.2.2 Imputing incomplete traffic data . .7 1.2.3 Clustering of traffic flow data . .8 1.3 Outline and contributions . .9 2 Distributed network flow problem 12 2.1 Introduction . 13 2.2 Preliminaries . 15 2.2.1 Notation . 15 2.2.2 Graph theory . 16 2.2.3 Eliminating affine equality constraints in optimization problems . 19 2.3 Problem definition . 20 2.4 A cycle-basis distributed ADMM solution . 23 2.4.1 Minimum search variable . 23 iii 2.4.2 Constructing cyber layer to solve the optimal power flow in a dis- tributed manner . 24 2.5 Numerical example . 28 2.6 Conclusion and future research . 30 3 Spatio-temporal clustering of traffic data 33 3.1 Introduction . 34 3.2 Technical background . 37 3.2.1 Problem definition . 37 3.2.2 Autoencoders . 38 3.2.3 Deep embedded clustering . 39 3.3 Spatio-temporal clustering . 40 3.4 Experimental results .
    [Show full text]
  • Hierarchical Fuzzy Support Vector Machine (SVM) for Rail Data Classification
    Preprints of the 19th World Congress The International Federation of Automatic Control Cape Town, South Africa. August 24-29, 2014 Hierarchical Fuzzy Support Vector Machine (SVM) for Rail Data Classification R. Muscat*, M. Mahfouf*, A. Zughrat*, Y.Y. Yang*, S. Thornton**, A.V. Khondabi* and S. Sortanos* *Department of Automatic Control and Systems Engineering, The University of Sheffield, Mappin St., Sheffield, S1 3JD, UK (Tel: +44 114 2225607) ([email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]) **Teesside Technology Centre, Tata Steel Europe, Eston Rd., Middlesbrough, TS6 6US, UK ([email protected]) Abstract: This study aims at designing a modelling architecture to deal with the imbalanced data relating to the production of rails. The modelling techniques are based on Support Vector Machines (SVMs) which are sensitive to class imbalance. An internal (Biased Fuzzy SVM) and external (data under- sampling) class imbalance learning methods were applied to the data. The performance of the techniques when implemented on the latter was better, while in both cases the inclusion of a fuzzy membership improved the performance of the SVM. Fuzzy C-Means (FCM) Clustering was analysed for reducing the number of support vectors of the Fuzzy SVM model, concluding it is effective in reducing model complexity without any significant performance deterioration. Keywords: Classification; railway steel manufacture; class imbalance; support vector machines; fuzzy support vector machines; fuzzy c-means clustering. The paper is organised as follows. Section 2 provides an 1 INTRODUCTION overview of the modelling data, obtained from the rail Cost management has become a very important factor in manufacturing process, and input variable selection.
    [Show full text]
  • Multi-View Fuzzy Clustering with the Alternative Learning Between Shared Hidden Space and Partition
    1 Multi-View Fuzzy Clustering with The Alternative Learning between Shared Hidden Space and Partition Zhaohong Deng, Senior Member, IEEE, Chen Cui, Peng Xu, Ling Liang, Haoran Chen, Te Zhang, Shitong Wang 1Abstract—As the multi-view data grows in the real world, posed in relevant literature [1-6] and most of them are classi- multi-view clustering has become a prominent technique in data fied into two categories: mining, pattern recognition, and machine learning. How to ex- i. Multi-view clustering methods based on single view clus- ploit the relationship between different views effectively using the tering methods, such as K-means [7, 8], Fuzzy C-means (FCM) characteristic of multi-view data has become a crucial challenge. [9,10], Maximum Entropy Clustering (MEC) [11, 12] and Aiming at this, a hidden space sharing multi-view fuzzy cluster- Possibilistic C-means (PCM) [13, 14]. Such approach usually ing (HSS-MVFC) method is proposed in the present study. This considers each view independently and treats each of them as method is based on the classical fuzzy c-means clustering model, an independent clustering task, followed by ensemble methods and obtains associated information between different views by [15, 16] to achieve the final clustering result. However, such introducing shared hidden space. Especially, the shared hidden space and the fuzzy partition can be learned alternatively and strategy is likely to cause poor results or unsteadiness of the contribute to each other. Meanwhile, the proposed method uses algorithm because of potential high deviation of a certain view maximum entropy strategy to control the weights of different or significant discrepancies between the results of each view.
    [Show full text]
  • Dynamic Topology Model of Q-Learning LEACH Using Disposable Sensors in Autonomous Things Environment
    applied sciences Article Dynamic Topology Model of Q-Learning LEACH Using Disposable Sensors in Autonomous Things Environment Jae Hyuk Cho * and Hayoun Lee School of Electronic Engineering, Soongsil University, Seoul 06978, Korea; [email protected] * Correspondence: [email protected]; Tel.: +82-2-811-0969 Received: 10 November 2020; Accepted: 15 December 2020; Published: 17 December 2020 Abstract: Low-Energy Adaptive Clustering Hierarchy (LEACH) is a typical routing protocol that effectively reduces transmission energy consumption by forming a hierarchical structure between nodes. LEACH on Wireless Sensor Network (WSN) has been widely studied in the recent decade as one key technique for the Internet of Things (IoT). The main aims of the autonomous things, and one of advanced of IoT, is that it creates a flexible environment that enables movement and communication between objects anytime, anywhere, by saving computing power and utilizing efficient wireless communication capability. However, the existing LEACH method is only based on the model with a static topology, but a case for a disposable sensor is included in an autonomous thing’s environment. With the increase of interest in disposable sensors which constantly change their locations during the operation, dynamic topology changes should be considered in LEACH. This study suggests the probing model for randomly moving nodes, implementing a change in the position of a node depending on the environment, such as strong winds. In addition, as a method to quickly adapt to the change in node location and construct a new topology, we propose Q-learning LEACH based on Q-table reinforcement learning and Fuzzy-LEACH based on Fuzzifier method.
    [Show full text]
  • Parameter Specification for Fuzzy Clustering by Q-Learning
    Parameter Specification for Fuzzy Clustering by Q-Learning Chi-Hyon Oh, Eriko Ikeda, Katsuhiro Honda and Hidetomo Ichihashi Department of Industrial Engineering, College of Engineering, Osaka Prefecture University, 1–1, Gakuencho, Sakai, Osaka, 599-8531, JAPAN E-mail: [email protected] Abstract In this paper, we propose a new method to specify the sequence of parameter values for a fuzzy clustering algorithm by using Q-learning. In the clustering algorithm, we employ similarities between two data points and distances from data to cluster centers as the fuzzy clustering criteria. The fuzzy clustering is achieved by optimizing an objective function which is solved by the Picard iteration. The fuzzy clustering algorithm might be useful but its result depends on the parameter specifications. To conquer the dependency on the parameter values, we use Q-learning to learn the sequential update for the parameters during the iterative optimization procedure of the fuzzy clustering. In the numerical example, we show how the clustering validity improves by the obtained parameter update sequences. Keywords Parameter Specification, Fuzzy Clustering, Reinforcement Learning. I. Introduction Many fuzzy clustering algorithms have been proposed since Ruspini developed the first one [1]. Fuzzy ISODATA [2] and its extension, Fuzzy c-means [3], are the popular fuzzy clustering algorithm using distance- based objective function methods. Other approaches are driven by several fuzzy clustering criteria. First, we propose a new fuzzy clustering algorithm, which adopts the similarities between two data point and the distances from data to cluster centers as the fuzzy clustering criteria to be optimized. We can expect that our algorithm might ably partition the data set into some clusters.
    [Show full text]
  • Fuzzy Clustering Based Image Denoising and Improved Support Vector Machine (ISVM) Based Nearest Target for Retina Images
    International Journal of Recent Technology and Engineering (IJRTE) ISSN: 2277-3878, Volume-8 Issue-5, January 2020 Fuzzy Clustering Based Image Denoising and Improved Support Vector Machine (ISVM) Based Nearest Target for Retina Images B. Sivaranjani, C. Kalaiselvi Abstract: A developing automated retinal disease diagnostic Particularly in areas in which retinal specialists may not be system based on image analysis has now demonstrated the ability easily available for diabetic retinopathy and premature in clinical research. Though, the accuracy of these systems has retinopathy (ROP) screening. In this section, the difficult been negotiated repeatedly, generally due to the basic effort in issue of matching and recording retinal images was perceiving the abnormal structures as well as due to deficits in discussed to allow for new applications for the image gaining that affects image quality. Use the fuzzy clustering; the noises contained in the samples are omitted from teleophthalmology. A new technique for locating optic discs the above. Unless the noises will be taken away from the samples in retinal images was suggested in [1]. instead dimension reduction initializes the optimization of Mutual Information (MI) as just a coarse localization process The first phase of certain vessel segmentation, disease that narrows the domain of optimization and tries to avoid local diagnosis, and retinal recognition algorithms would be to optimization. Furthermore, the suggested work closer to the locate the optic disc and its core. In [2] the latest research on retina picture being done using the Improved Support Vector the essential characteristics and features of DR eye Machine (ISVM) system used in the area-based registration, telehealth services was evaluated in the categories listed: offering a reliable approach.
    [Show full text]
  • An Energy-Efficient Spectrum-Aware Reinforcement Learning-Based Clustering Algorithm for Cognitive Radio Sensor Networks
    Sensors 2015, 15, 19783-19818; doi:10.3390/s150819783 OPEN ACCESS sensors ISSN 1424-8220 www.mdpi.com/journal/sensors Article An Energy-Efficient Spectrum-Aware Reinforcement Learning-Based Clustering Algorithm for Cognitive Radio Sensor Networks Ibrahim Mustapha 1,2,†, Borhanuddin Mohd Ali 1,*, Mohd Fadlee A. Rasid 1,†, Aduwati Sali 1,† and Hafizal Mohamad 3,† 1 Department of Computer and Communications Systems Engineering and Wireless and Photonics Research Centre, Faculty of Engineering, Universiti Putra Malaysia, 43400 Serdang Selangor, Malaysia; E-Mails: [email protected] (I.M.); [email protected] (M.F.A.R.); [email protected] (A.S.) 2 Department of Electrical and Electronics Engineering, Faculty of Engineering, University of Maiduguri, P. M. B. 1069, Maiduguri, Nigeria 3 Wireless Networks and Protocol Research Lab, MIMOS Berhad, Technology Park Malaysia, 57000 Kuala Lumpur, Malaysia; E-Mail: [email protected] † These authors contributed equally to this work. * Author to whom correspondence should be addressed; E-Mail: [email protected]; Tel.: +60-3-8946-6443; Fax: +60-3-8656-7127. Academic Editor: Davide Brunelli Received: 25 May 2015 / Accepted: 31 July 2015 / Published: 13 August 2015 Abstract: It is well-known that clustering partitions network into logical groups of nodes in order to achieve energy efficiency and to enhance dynamic channel access in cognitive radio through cooperative sensing. While the topic of energy efficiency has been well investigated in conventional wireless sensor networks, the latter has not been extensively explored. In this paper, we propose a reinforcement learning-based spectrum-aware clustering algorithm that allows a member node to learn the energy and cooperative sensing costs for neighboring clusters to achieve an optimal solution.
    [Show full text]
  • A Hybrid Technique Based on Fuzzy-C-Means and SVM for Detection of Brain Tumor in MRI Images Dipali B.Birnale¹, Prof .S
    ISSN XXXX XXXX © 2016 IJESC Research Article Volume 6 Issue No. 8 A Hybrid Technique based on Fuzzy-C-Means and SVM for Detection of Brain Tumor in MRI Images Dipali B.Birnale¹, Prof .S. N. Patil² Associate Professor² Department of Electronics Engineering PVPIT Budhgaon, Maharashtra, India Abstract: MRI is the most important technique, in detecting the brain tumor. In this paper data mining methods are used for classificat ion of MRI images. A new hybrid technique based on the support vector machine (SVM) and fuzzy c -means for brain tumor classification is proposed. The proposed algorithm is a combination of support vector machine (SVM) and fuzzy –c- means, a hybrid technique for prediction of brain tumor. In this algorithm the image is De-noised first and then morphological operations are used for skull striping. Fuzzy c-means (FCM) clustering is used for the segmentation of the image to detect the suspicious region in brain MRI image & feature are extracted from the brain image, after which SVM technique is applied to classify the brain MRI images, which provide accurate and more effective result for classification of brain MRI images. Keywords: Data Mining, MRI, De-noising, Fuzzy C-means clustering, Support Vector Machine (SVM) I. INTRODUCTION Flow Chart In medical image segmentation, brain tumor detection is one of the most challenging tasks, since brain images are complicated and tumors can be analyzed only by expert physicians. Brain tumor is a group of abnormal cells that grows inside of the brain or around the brain. The tumor may be primary or secondary.
    [Show full text]
  • Deep Adaptive Fuzzy Clustering for Evolutionary Unsupervised Representation Learning Dayu Tan, Zheng Huang, Xin Peng, Member, IEEE, Weimin Zhong, and Vladimir Mahalec
    1 Deep Adaptive Fuzzy Clustering for Evolutionary Unsupervised Representation Learning Dayu Tan, Zheng Huang, Xin Peng, Member, IEEE, Weimin Zhong, and Vladimir Mahalec Abstract—Cluster assignment of large and complex images is a extensively studied and applied to different fields [1]. This crucial but challenging task in pattern recognition and computer type of method iteratively groups the features with a standard vision. In this study, we explore the possibility of employing fuzzy division method, for example, k-means [2] and fuzzy means clustering in a deep neural network framework. Thus, we present a novel evolutionary unsupervised learning representation model [3], which use the subsequent assignment as supervision to with iterative optimization. It implements the deep adaptive fuzzy update the clusters. Numerous different clustering methods are clustering (DAFC) strategy that learns a convolutional neural calculated by distance functions, and their combinations with network classifier from given only unlabeled data samples. DAFC embedding algorithms have been explored in literature [4, 5]. consists of a deep feature quality-verifying model and a fuzzy The conventional method always employs Euclidean distance clustering model, where deep feature representation learning loss function and embedded fuzzy clustering with the weighted to express this similarity, while stochastic neighbor embedding adaptive entropy is implemented. We joint fuzzy clustering to (SNE) converts this distance relationship into a conditional the deep reconstruction model, in which fuzzy membership is probability to express similarity. In this way, the distance utilized to represent a clear structure of deep cluster assignments between similar sample points in high-dimensional space is and jointly optimize for the deep representation learning and also similar to that in low-dimensional space.
    [Show full text]
  • Intrusion Detection Using Fuzzy Clustering and Artificial Neural
    Advances in Neural Networks, Fuzzy Systems and Artificial Intelligence Intrusion Detection using Fuzzy Clustering and Artificial Neural Network Shraddha Surana Research Scholar, Department of Computer Engineering, Vishwakarma Institute of Technology, Pune India [email protected] host based IDS makes use of log files from individual computers, whereas a network based Abstract IDS captures packets from the network and analyses its contents. An online IDS is able to flag This paper presents the outline of a hybrid an intrusion while it’s happening whereas an Artificial Neural Network (ANN) based on fuzzy offline IDS analyses records after the event has clustering and neural networks for an Intrusion occurred and raises a flag indicating that a security Detection System (IDS). While neural networks breach had occurred since the last intrusion are effective in capturing the non-linearity in data detection check was performed. An Anomaly provided, it also has certain limitations including based IDS detects deviation from normal the requirement of high computational resources. behaviour while misuse based IDS compare By clustering the data, each ANN is trained on a activities on the system with known behaviours of particular subset of data, reducing time required attackers. and achieving a higher detection rate. The outline of the implemented algorithm is as follows: first This paper outlines a hybrid approach using the data is divided into smaller homogeneous Artificial Neural Networks and Fuzzy clustering to groups/ subsets using a fuzzy clustering technique. detect intrusions in a network. The method Subsequently, a separate ANN is trained on each outlined is network based which extracts features subset.
    [Show full text]
  • Developing Support Vector Machine with New Fuzzy Selection for the Infringement of a Patent Rights Problem
    mathematics Article Developing Support Vector Machine with New Fuzzy Selection for the Infringement of a Patent Rights Problem Chih-Yao Chang 1,2 and Kuo-Ping Lin 3,4,* 1 Graduate Institute of Technology, Innovation & Intellectual Property Management, National Cheng Chi University, Taipei 116302, Taiwan; [email protected] 2 Taiwan Development & Research Academia of Economic & Technology, Taipei 104, Taiwan 3 Department of Industrial Engineering and Enterprise Information, Tunghai University, Taichung 40704, Taiwan 4 Faculty of Finance and Banking, Ton Duc Thang University, Ho Chi Minh City 758307, Vietnam * Correspondence: [email protected]; Tel.: +886-4-2359-4319 (ext. 111) Received: 16 June 2020; Accepted: 30 July 2020; Published: 1 August 2020 Abstract: Classification problems are very important issues in real enterprises. In the patent infringement issue, accurate classification could help enterprises to understand court decisions to avoid patent infringement. However, the general classification method does not perform well in the patent infringement problem because there are too many complex variables. Therefore, this study attempts to develop a classification method, the support vector machine with new fuzzy selection (SVMFS), to judge the infringement of patent rights. The raw data are divided into training and testing sets. However, the data quality of the training set is not easy to evaluate. Effective data quality management requires a structural core that can support data operations. This study adopts new fuzzy selection based on membership values, which are generated from fuzzy c-means clustering, to select appropriate data to enhance the classification performance of the support vector machine (SVM). An empirical example based on the SVMFS shows that the proposed SVMFS can obtain a superior accuracy rate.
    [Show full text]
  • A Unified Deep Learning Framework for Text Data Mining Using Deep Adaptive Fuzzy Clustering
    European Journal of Molecular & Clinical Medicine ISSN 2515-8260 Volume 07, Issue 09, 2020 A UNIFIED DEEP LEARNING FRAMEWORK FOR TEXT DATA MINING USING DEEP ADAPTIVE FUZZY CLUSTERING S. Praveen1, Dr. R. Priya, MCA, M.phil, Ph.D2, Research Scholar1, Associate Professor and Head, Department of Computer Science 1,2 Sree Narayana Guru College, Coimbatore, Abstract Text clustering is an important method for effectively organising, summarising, and navigating text information. The purpose of the clustering is to distinguish and classify the similarity among the text instance as label. However, in the absence of labels, the text data to be clustered cannot be used to train the text representation model based on deep learning as it contains high dimensional data with complex latent distributions. To address this problem, a new unified deep learning framework for text clustering based on deep representation learning is proposed using the deep adaptive fuzzy clustering in this paper to provide soft partition of data. Initially reconstruction of original data into feature space carried out using the word embedding process of deep learning. Word embedding process is a learnt representation of the text or sentence towards clustering into vector containing words, characters and N-grams of words. Further clustering of feature vector is carried out with max pooling layer to determine the inter-cluster seperability and intra-cluster compactness. Moreover learning of the feature space is processed with gradient descent. Moreover tuning of feature vector is fine tuned on basis of Discriminant information using hyper parameter optimization with fewer epochs. Finally representation learning and soft clustering has been achieved using deep adaptive fuzzy clustering and quantum annealing based optimization has been employed .The results demonstrate that the clustering approach more stable and accurate than the traditional FCM clustering algorithm on employing k fold validation for evaluation.
    [Show full text]