DOCSLIB.ORG

Multi-View Clustering Via Canonical Correlation Analysis

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Multi-View Clustering Via Canonical Correlation Analysis Multi-View Clustering via Canonical Correlation Analysis Kamalika Chaudhuri [email protected] UC San Diego, 9500 Gilman Drive, La Jolla, CA Sham M. Kakade [email protected] Karen Livescu [email protected] Karthik Sridharan [email protected] Toyota Technological Institute at Chicago, 6045 S. Kenwood Ave., Chicago, IL Keywords: multi-view learning, clustering, canonical correlation analysis Abstract ing problem of interest [BM98, KF07, AZ07]. In this work, we explore how having two ‘views’ makes the Clustering data in high dimensions is be- clustering problem significantly more tractable. lieved to be a hard problem in general. A number of efficient clustering algorithms de- Much recent work has been done on understanding veloped in recent years address this prob- under what conditions we can learn a mixture model. lem by projecting the data into a lower- The basic problem is as follows: We are given indepen- dimensional subspace, e.g. via Principal dent samples from a mixture of k distributions, and Components Analysis (PCA) or random pro- our task is to either: 1) infer properties of the under- jections, before clustering. Here, we consider lying mixture model (e.g. the mixing weights, means, constructing such projections using multiple etc.) or 2) classify a random sample according to which views of the data, via Canonical Correlation distribution in the mixture it was generated from. Analysis (CCA). Under no restrictions on the underlying mixture, this Under the assumption that the views are un- problem is considered to be hard. However, in many correlated given the cluster label, we show applications, we are only interested in clustering the that the separation conditions required for data when the component distributions are “well sep- the algorithm to be successful are signifi- arated”. In fact, the focus of recent clustering algo- cantly weaker than prior results in the lit- rithms [Das99, VW02, AM05, BV08] is on efficiently erature. We provide results for mixtures learning with as little separation as possible. Typ- of Gaussians and mixtures of log concave ically, the separation conditions are such that when distributions. We also provide empirical given a random sample from the mixture model, the support from audio-visual speaker clustering Bayes optimal classifier is able to reliably recover (where we desire the clusters to correspond to which cluster generated that point. speaker ID) and from hierarchical Wikipedia document clustering (where one view is the This work makes a natural multi-view assumption: words in the document and the other is the that the views are (conditionally) uncorrelated, con- link structure). ditioned on which mixture component generated the views. There are many natural applications for which 1. Introduction this assumption applies. For example, we can con- sider multi-modal views, with one view being a video The multi-view approach to learning is one in which stream and the other an audio stream, of a speaker — we have ‘views’ of the data (sometimes in a rather ab- here, conditioned on the speaker identity and maybe stract sense) and the goal is to use the relationship be- the phoneme (both of which could label the generat- tween these views to alleviate the difficulty of a learn- ing cluster), the views may be uncorrelated. A second th example is the words and link structure in a document Appearing in Proceedings of the 26 International Confer- from a corpus such as Wikipedia – here, conditioned ence on Machine Learning, Montreal, Canada, 2009. Copy- right 2009 by the author(s)/owner(s). on the category of each document, the words in it and Multi-View Clustering via Canonical Correlation Analysis its link structure may be uncorrelated. In this paper, None of these algorithms are affine invariant. we provide experiments for both settings. Finally, [BV08] provide an affine-invariant algorithm Under this multi-view assumption, we provide a sim- for learning mixtures of general Gaussians, so long as ple and efficient subspace learning method, based the mixture has a suitably low Fisher coefficient when on Canonical Correlation Analysis (CCA). This algo- in isotropic position. However, their separation in- rithm is affine invariant and is able to learn with some volves a large polynomial dependence on 1 . wmin of the weakest separation conditions to date. The in- The two results most closely related to ours are the tuitive reason for this is that under our multi-view work of [VW02] and [CR08]. [VW02] show that it is assumption, we are able to (approximately) find the sufficient to find the subspace spanned by the means low-dimensional subspace spanned by the means of of the distributions in the mixture for effective cluster- the component distributions. This subspace is impor- ing. Like our algorithm, [CR08] use a projection onto tant, because, when projected onto this subspace, the the top k −1 singular value decomposition subspace of means of the distributions are well-separated, yet the the canonical correlations matrix. They also require typical distance between points from the same distri- a spreading condition, which is related to our require- bution is smaller than in the original space. The num- ment on the rank. We borrow techniques from both of ber of samples we require to cluster correctly scales these papers. as O(d), where d is the ambient dimension. Finally, we show through experiments that CCA-based algo- [BL08] propose a similar algorithm for multi-view clus- rithms consistently provide better performance than tering, in which data is projected onto the top direc- standard PCA-based clustering methods when applied tions obtained by kernel CCA across the views. They to datasets in the two quite different domains of audio- show empirically that for clustering images using the visual speaker clustering and hierarchical Wikipedia associated text as a second view (where the target clus- document clustering by category. tering is a human-defined category), CCA-based clus- tering methods out-perform PCA-based algorithms. Our work adds to the growing body of results which show how the multi-view framework can alleviate the This Work. Our input is data on a fixed set of ob- difficulty of learning problems. jects from two views, where View j is assumed to be j j generated by a mixture of k Gaussians (D1,...,Dk), Related Work. Most provably efficient clustering for j = 1, 2. To generate a sample, a source i is picked (1) (2) algorithms first project the data down to some low- with probability wi, and x and x in Views 1 and 1 2 dimensional space and then cluster the data in this 2 are drawn from distributions Di and Di . Following lower dimensional space (an algorithm such as sin- prior theoretical work, our goal is to show that our al- gle linkage usually suffices here). Typically, these al- gorithm recovers the correct clustering, provided the gorithms also work under a separation requirement, input mixture obeys certain conditons. which is measured by the minimum distance between We impose two requirements on these mixtures. First, the means of any two mixture components. we require that conditioned on the source, the two One of the first provably efficient algorithms for learn- views are uncorrelated. Notice that this is a weaker ing mixture models is due to [Das99], who learns restriction than the condition that given source i, the 1 2 a mixture of spherical Gaussians by randomly pro- samples from Di and Di are drawn independently. jecting the mixture onto a low-dimensional subspace. Moreover, this condition allows the distributions in the [VW02] provide an algorithm with an improved sepa- mixture within each view to be completely general, so ration requirement that learns a mixture of k spheri- long as they are uncorrelated across views. Although cal Gaussians, by projecting the mixture down to the we do not prove this, our algorithm seems robust to k-dimensional subspace of highest variance. [KSV05, small deviations from this assumption. AM05] extend this result to mixtures of general Gaus- Second, we require the rank of the CCA matrix across sians; however, they require a separation propor- the views to be at least k − 1, when each view is in tional to the maximum directional standard deviation isotropic position, and the k − 1-th singular value of of any mixture component. [CR08] use a canonical this matrix to be at least λ . This condition ensures correlations-based algorithm to learn mixtures of axis- min that there is sufficient correlation between the views. aligned Gaussians with a separation proportional to If the first two conditions hold, then we can recover σ∗, the maximum directional standard deviation in the the subspace containing the means in both views. subspace containing the means of the distributions. Their algorithm requires a coordinate-independence In addition, for mixtures of Gaussians, if in at least property, and an additional “spreading” condition. one view, say View 1, we have that for every pair of Multi-View Clustering via Canonical Correlation Analysis distributions i and j in the mixture, of dimension d1 and V2 of dimension d2, respectively. j We let d = d1+d2. Let µi , for i = 1, . , k and j = 1, 2, 1 1 ∗ 1/4p ||µi − µj || > Cσ k log(n/δ) be the mean of distribution i in view j, and let wi be the mixing weight for distribution i. for some constant C, then our algorithm can also de- termine which component each sample came from. For simplicity, we assume that the data have mean 0. 1 We denote the covariance matrix of the data as: Here µi is the mean of the i-th component in View ∗ 1 and σ is the maximum directional standard devi- > (1) (1) > ation in the subspace containing the means in View Σ = E[xx ], Σ11 = E[x (x ) ] (2) (2) > (1) (2) > 1.
Load more

Recommended publications
  • Canonical Correlation a Tutorial
    Canonical Correlation a Tutorial Magnus Borga January 12, 2001 Contents 1 About this tutorial 1 2 Introduction 2 3 Definition 2 4 Calculating canonical correlations 3 5 Relating topics 3 5.1 The difference between CCA and ordinary correlation analysis . 3 5.2Relationtomutualinformation................... 4 5.3 Relation to other linear subspace methods ............. 4 5.4RelationtoSNR........................... 5 5.4.1 Equalnoiseenergies.................... 5 5.4.2 Correlation between a signal and the corrupted signal . 6 A Explanations 6 A.1Anoteoncorrelationandcovariancematrices........... 6 A.2Affinetransformations....................... 6 A.3Apieceofinformationtheory.................... 7 A.4 Principal component analysis . ................... 9 A.5 Partial least squares ......................... 9 A.6 Multivariate linear regression . ................... 9 A.7Signaltonoiseratio......................... 10 1 About this tutorial This is a printable version of a tutorial in HTML format. The tutorial may be modified at any time as will this version. The latest version of this tutorial is available at http://people.imt.liu.se/˜magnus/cca/. 1 2 Introduction Canonical correlation analysis (CCA) is a way of measuring the linear relationship between two multidimensional variables. It finds two bases, one for each variable, that are optimal with respect to correlations and, at the same time, it finds the corresponding correlations. In other words, it finds the two bases in which the correlation matrix between the variables is diagonal and the correlations on the diagonal are maximized. The dimensionality of these new bases is equal to or less than the smallest dimensionality of the two variables. An important property of canonical correlations is that they are invariant with respect to affine transformations of the variables. This is the most important differ- ence between CCA and ordinary correlation analysis which highly depend on the basis in which the variables are described.
    [Show full text]
  • A Tutorial on Canonical Correlation Analysis
    Finding the needle in high-dimensional haystack: A tutorial on canonical correlation analysis Hao-Ting Wang1,*, Jonathan Smallwood1, Janaina Mourao-Miranda2, Cedric Huchuan Xia3, Theodore D. Satterthwaite3, Danielle S. Bassett4,5,6,7, Danilo Bzdok,8,9,10,* 1Department of Psychology, University of York, Heslington, York, YO10 5DD, United Kingdome 2Centre for Medical Image Computing, Department of Computer Science, University College London, London, United Kingdom; MaX Planck University College London Centre for Computational Psychiatry and Ageing Research, University College London, London, United Kingdom 3Department of Psychiatry, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, 19104, USA 4Department of Bioengineering, University of Pennsylvania, Philadelphia, Pennsylvania 19104, USA 5Department of Electrical and Systems Engineering, University of Pennsylvania, Philadelphia, Pennsylvania 19104, USA 6Department of Neurology, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA 19104 USA 7Department of Physics & Astronomy, School of Arts & Sciences, University of Pennsylvania, Philadelphia, PA 19104 USA 8 Department of Psychiatry, Psychotherapy and Psychosomatics, RWTH Aachen University, Germany 9JARA-BRAIN, Jülich-Aachen Research Alliance, Germany 10Parietal Team, INRIA, Neurospin, Bat 145, CEA Saclay, 91191, Gif-sur-Yvette, France * Correspondence should be addressed to: Hao-Ting Wang [email protected] Department of Psychology, University of York, Heslington, York, YO10 5DD, United Kingdome Danilo Bzdok [email protected] Department of Psychiatry, Psychotherapy and Psychosomatics, RWTH Aachen University, Germany 1 1 ABSTRACT Since the beginning of the 21st century, the size, breadth, and granularity of data in biology and medicine has grown rapidly. In the eXample of neuroscience, studies with thousands of subjects are becoming more common, which provide eXtensive phenotyping on the behavioral, neural, and genomic level with hundreds of variables.
    [Show full text]
  • Text Mining Course for KNIME Analytics Platform
    Text Mining Course for KNIME Analytics Platform KNIME AG Copyright © 2018 KNIME AG Table of Contents 1. The Open Analytics Platform 2. The Text Processing Extension 3. Importing Text 4. Enrichment 5. Preprocessing 6. Transformation 7. Classification 8. Visualization 9. Clustering 10. Supplementary Workflows Licensed under a Creative Commons Attribution- ® Copyright © 2018 KNIME AG 2 Noncommercial-Share Alike license 1 https://creativecommons.org/licenses/by-nc-sa/4.0/ Overview KNIME Analytics Platform Licensed under a Creative Commons Attribution- ® Copyright © 2018 KNIME AG 3 Noncommercial-Share Alike license 1 https://creativecommons.org/licenses/by-nc-sa/4.0/ What is KNIME Analytics Platform? • A tool for data analysis, manipulation, visualization, and reporting • Based on the graphical programming paradigm • Provides a diverse array of extensions: • Text Mining • Network Mining • Cheminformatics • Many integrations, such as Java, R, Python, Weka, H2O, etc. Licensed under a Creative Commons Attribution- ® Copyright © 2018 KNIME AG 4 Noncommercial-Share Alike license 2 https://creativecommons.org/licenses/by-nc-sa/4.0/ Visual KNIME Workflows NODES perform tasks on data Not Configured Configured Outputs Inputs Executed Status Error Nodes are combined to create WORKFLOWS Licensed under a Creative Commons Attribution- ® Copyright © 2018 KNIME AG 5 Noncommercial-Share Alike license 3 https://creativecommons.org/licenses/by-nc-sa/4.0/ Data Access • Databases • MySQL, MS SQL Server, PostgreSQL • any JDBC (Oracle, DB2, …) • Files • CSV, txt
    [Show full text]
  • A Survey of Clustering with Deep Learning: from the Perspective of Network Architecture
    Received May 12, 2018, accepted July 2, 2018, date of publication July 17, 2018, date of current version August 7, 2018. Digital Object Identifier 10.1109/ACCESS.2018.2855437 A Survey of Clustering With Deep Learning: From the Perspective of Network Architecture ERXUE MIN , XIFENG GUO, QIANG LIU , (Member, IEEE), GEN ZHANG , JIANJING CUI, AND JUN LONG College of Computer, National University of Defense Technology, Changsha 410073, China Corresponding authors: Erxue Min ([email protected]) and Qiang Liu ([email protected]) This work was supported by the National Natural Science Foundation of China under Grant 60970034, Grant 61105050, Grant 61702539, and Grant 6097003. ABSTRACT Clustering is a fundamental problem in many data-driven application domains, and clustering performance highly depends on the quality of data representation. Hence, linear or non-linear feature transformations have been extensively used to learn a better data representation for clustering. In recent years, a lot of works focused on using deep neural networks to learn a clustering-friendly representation, resulting in a significant increase of clustering performance. In this paper, we give a systematic survey of clustering with deep learning in views of architecture. Specifically, we first introduce the preliminary knowledge for better understanding of this field. Then, a taxonomy of clustering with deep learning is proposed and some representative methods are introduced. Finally, we propose some interesting future opportunities of clustering with deep learning and give some conclusion remarks. INDEX TERMS Clustering, deep learning, data representation, network architecture. I. INTRODUCTION property of highly non-linear transformation. For the sim- Data clustering is a basic problem in many areas, such as plicity of description, we call clustering methods with deep machine learning, pattern recognition, computer vision, data learning as deep clustering1 in this paper.
    [Show full text]
  • A Tree-Based Dictionary Learning Framework
    A Tree-based Dictionary Learning Framework Renato Budinich∗ & Gerlind Plonka† June 11, 2020 We propose a new outline for adaptive dictionary learning methods for sparse encoding based on a hierarchical clustering of the training data. Through recursive application of a clustering method the data is organized into a binary partition tree representing a multiscale structure. The dictionary atoms are defined adaptively based on the data clusters in the partition tree. This approach can be interpreted as a generalization of a discrete Haar wavelet transform. Furthermore, any prior knowledge on the wanted structure of the dictionary elements can be simply incorporated. The computational complex- ity of our proposed algorithm depends on the employed clustering method and on the chosen similarity measure between data points. Thanks to the multi- scale properties of the partition tree, our dictionary is structured: when using Orthogonal Matching Pursuit to reconstruct patches from a natural image, dic- tionary atoms corresponding to nodes being closer to the root node in the tree have a tendency to be used with greater coefficients. Keywords. Multiscale dictionary learning, hierarchical clustering, binary partition tree, gen- eralized adaptive Haar wavelet transform, K-means, orthogonal matching pursuit 1 Introduction arXiv:1909.03267v2 [cs.LG] 9 Jun 2020 In many applications one is interested in sparsely approximating a set of N n-dimensional data points Y , columns of an n N real matrix Y = (Y1;:::;Y ). Assuming that the data j × N ∗R. Budinich is with the Fraunhofer SCS, Nordostpark 93, 90411 Nürnberg, Germany, e-mail: re- [email protected] †G. Plonka is with the Institute for Numerical and Applied Mathematics, University of Göttingen, Lotzestr.
    [Show full text]
  • Comparison of Dimensionality Reduction Techniques on Audio Signals
    Comparison of Dimensionality Reduction Techniques on Audio Signals Tamás Pál, Dániel T. Várkonyi Eötvös Loránd University, Faculty of Informatics, Department of Data Science and Engineering, Telekom Innovation Laboratories, Budapest, Hungary {evwolcheim, varkonyid}@inf.elte.hu WWW home page: http://t-labs.elte.hu Abstract: Analysis of audio signals is widely used and this work: car horn, dog bark, engine idling, gun shot, and very effective technique in several domains like health- street music [5]. care, transportation, and agriculture. In a general process Related work is presented in Section 2, basic mathe- the output of the feature extraction method results in huge matical notation used is described in Section 3, while the number of relevant features which may be difficult to pro- different methods of the pipeline are briefly presented in cess. The number of features heavily correlates with the Section 4. Section 5 contains data about the evaluation complexity of the following machine learning method. Di- methodology, Section 6 presents the results and conclu- mensionality reduction methods have been used success- sions are formulated in Section 7. fully in recent times in machine learning to reduce com- The goal of this paper is to find a combination of feature plexity and memory usage and improve speed of following extraction and dimensionality reduction methods which ML algorithms. This paper attempts to compare the state can be most efficiently applied to audio data visualization of the art dimensionality reduction techniques as a build- in 2D and preserve inter-class relations the most. ing block of the general process and analyze the usability of these methods in visualizing large audio datasets.
    [Show full text]
  • Robust Hierarchical Clustering∗
    Journal of Machine Learning Research 15 (2014) 4011-4051 Submitted 12/13; Published 12/14 Robust Hierarchical Clustering∗ Maria-Florina Balcan [email protected] School of Computer Science Carnegie Mellon University Pittsburgh, PA 15213, USA Yingyu Liang [email protected] Department of Computer Science Princeton University Princeton, NJ 08540, USA Pramod Gupta [email protected] Google, Inc. 1600 Amphitheatre Parkway Mountain View, CA 94043, USA Editor: Sanjoy Dasgupta Abstract One of the most widely used techniques for data clustering is agglomerative clustering. Such algorithms have been long used across many different fields ranging from computational biology to social sciences to computer vision in part because their output is easy to interpret. Unfortunately, it is well known, however, that many of the classic agglomerative clustering algorithms are not robust to noise. In this paper we propose and analyze a new robust algorithm for bottom-up agglomerative clustering. We show that our algorithm can be used to cluster accurately in cases where the data satisfies a number of natural properties and where the traditional agglomerative algorithms fail. We also show how to adapt our algorithm to the inductive setting where our given data is only a small random sample of the entire data set. Experimental evaluations on synthetic and real world data sets show that our algorithm achieves better performance than other hierarchical algorithms in the presence of noise. Keywords: unsupervised learning, clustering, agglomerative algorithms, robustness 1. Introduction Many data mining and machine learning applications ranging from computer vision to biology problems have recently faced an explosion of data. As a consequence it has become increasingly important to develop effective, accurate, robust to noise, fast, and general clustering algorithms, accessible to developers and researchers in a diverse range of areas.
    [Show full text]
  • Space-Time Hierarchical Clustering for Identifying Clusters in Spatiotemporal Point Data
    International Journal of Geo-Information Article Space-Time Hierarchical Clustering for Identifying Clusters in Spatiotemporal Point Data David S. Lamb 1,* , Joni Downs 2 and Steven Reader 2 1 Measurement and Research, Department of Educational and Psychological Studies, College of Education, University of South Florida, 4202 E Fowler Ave, Tampa, FL 33620, USA 2 School of Geosciences, University of South Florida, 4202 E Fowler Ave, Tampa, FL 33620, USA; [email protected] (J.D.); [email protected] (S.R.) * Correspondence: [email protected] Received: 2 December 2019; Accepted: 27 January 2020; Published: 1 February 2020 Abstract: Finding clusters of events is an important task in many spatial analyses. Both confirmatory and exploratory methods exist to accomplish this. Traditional statistical techniques are viewed as confirmatory, or observational, in that researchers are confirming an a priori hypothesis. These methods often fail when applied to newer types of data like moving object data and big data. Moving object data incorporates at least three parts: location, time, and attributes. This paper proposes an improved space-time clustering approach that relies on agglomerative hierarchical clustering to identify groupings in movement data. The approach, i.e., space–time hierarchical clustering, incorporates location, time, and attribute information to identify the groups across a nested structure reflective of a hierarchical interpretation of scale. Simulations are used to understand the effects of different parameters, and to compare against existing clustering methodologies. The approach successfully improves on traditional approaches by allowing flexibility to understand both the spatial and temporal components when applied to data. The method is applied to animal tracking data to identify clusters, or hotspots, of activity within the animal’s home range.
    [Show full text]
  • Chapter G03 – Multivariate Methods
    g03 – Multivariate Methods Introduction – g03 Chapter g03 – Multivariate Methods 1. Scope of the Chapter This chapter is concerned with methods for studying multivariate data. A multivariate data set consists of several variables recorded on a number of objects or individuals. Multivariate methods can be classified as those that seek to examine the relationships between the variables (e.g., principal components), known as variable-directed methods, and those that seek to examine the relationships between the objects (e.g., cluster analysis), known as individual-directed methods. Multiple regression is not included in this chapter as it involves the relationship of a single variable, known as the response variable, to the other variables in the data set, the explanatory variables. Routines for multiple regression are provided in Chapter g02. 2. Background 2.1. Variable-directed Methods Let the n by p data matrix consist of p variables, x1,x2,...,xp,observedonn objects or individuals. Variable-directed methods seek to examine the linear relationships between the p variables with the aim of reducing the dimensionality of the problem. There are different methods depending on the structure of the problem. Principal component analysis and factor analysis examine the relationships between all the variables. If the individuals are classified into groups then canonical variate analysis examines the between group structure. If the variables can be considered as coming from two sets then canonical correlation analysis examines the relationships between the two sets of variables. All four methods are based on an eigenvalue decomposition or a singular value decomposition (SVD)of an appropriate matrix. The above methods may reduce the dimensionality of the data from the original p variables to a smaller number, k, of derived variables that adequately represent the data.
    [Show full text]
  • A Call for Conducting Multivariate Mixed Analyses
    Journal of Educational Issues ISSN 2377-2263 2016, Vol. 2, No. 2 A Call for Conducting Multivariate Mixed Analyses Anthony J. Onwuegbuzie (Corresponding author) Department of Educational Leadership, Box 2119 Sam Houston State University, Huntsville, Texas 77341-2119, USA E-mail: [email protected] Received: April 13, 2016 Accepted: May 25, 2016 Published: June 9, 2016 doi:10.5296/jei.v2i2.9316 URL: http://dx.doi.org/10.5296/jei.v2i2.9316 Abstract Several authors have written methodological works that provide an introductory- and/or intermediate-level guide to conducting mixed analyses. Although these works have been useful for beginning and emergent mixed researchers, with very few exceptions, works are lacking that describe and illustrate advanced-level mixed analysis approaches. Thus, the purpose of this article is to introduce the concept of multivariate mixed analyses, which represents a complex form of advanced mixed analyses. These analyses characterize a class of mixed analyses wherein at least one of the quantitative analyses and at least one of the qualitative analyses both involve the simultaneous analysis of multiple variables. The notion of multivariate mixed analyses previously has not been discussed in the literature, illustrating the significance and innovation of the article. Keywords: Mixed methods data analysis, Mixed analysis, Mixed analyses, Advanced mixed analyses, Multivariate mixed analyses 1. Crossover Nature of Mixed Analyses At least 13 decision criteria are available to researchers during the data analysis stage of mixed research studies (Onwuegbuzie & Combs, 2010). Of these criteria, the criterion that is the most underdeveloped is the crossover nature of mixed analyses. Yet, this form of mixed analysis represents a pivotal decision because it determines the level of integration and complexity of quantitative and qualitative analyses in mixed research studies.
    [Show full text]
  • A Study of Hierarchical Clustering Algorithm
    International Journal of Information and Computation Technology. ISSN 0974-2239 Volume 3, Number 11 (2013), pp. 1225-1232 © International Research Publications House http://www. irphouse.com /ijict.htm A Study of Hierarchical Clustering Algorithm Yogita Rani¹ and Dr. Harish Rohil2 1Reseach Scholar, Department of Computer Science & Application, CDLU, Sirsa-125055, India. 2Assistant Professor, Department of Computer Science & Application, CDLU, Sirsa-125055, India. Abstract Clustering is the process of grouping the data into classes or clusters, so that objects within a cluster have high similarity in comparison to one another but these objects are very dissimilar to the objects that are in other clusters. Clustering methods are mainly divided into two groups: hierarchical and partitioning methods. Hierarchical clustering combine data objects into clusters, those clusters into larger clusters, and so forth, creating a hierarchy of clusters. In partitioning clustering methods various partitions are constructed and then evaluations of these partitions are performed by some criterion. This paper presents detailed discussion on some improved hierarchical clustering algorithms. In addition to this, author have given some criteria on the basis of which one can also determine the best among these mentioned algorithms. Keywords: Hierarchical clustering; BIRCH; CURE; clusters ;data mining. 1. Introduction Data mining allows us to extract knowledge from our historical data and predict outcomes of our future situations. Clustering is an important data mining task. It can be described as the process of organizing objects into groups whose members are similar in some way. Clustering can also be define as the process of grouping the data into classes or clusters, so that objects within a cluster have high similarity in comparison to one another but are very dissimilar to objects in other clusters.
    [Show full text]
  • CNN Features Are Also Great at Unsupervised Classification
    CNN features are also great at unsupervised classification Joris Gu´erin,Olivier Gibaru, St´ephaneThiery, and Eric Nyiri Laboratoire des Sciences de l'Information et des Syst`emes (CNRS UMR 7296) Arts et M´etiersParisTech, Lille, France [email protected] ABSTRACT This paper aims at providing insight on the transferability of deep CNN features to unsupervised problems. We study the impact of different pretrained CNN feature extractors on the problem of image set clustering for object classification as well as fine-grained classification. We propose a rather straightforward pipeline combining deep-feature extraction using a CNN pretrained on ImageNet and a classic clustering algorithm to classify sets of images. This approach is compared to state-of-the-art algorithms in image-clustering and provides better results. These results strengthen the belief that supervised training of deep CNN on large datasets, with a large variability of classes, extracts better features than most carefully designed engineering approaches, even for unsupervised tasks. We also validate our approach on a robotic application, consisting in sorting and storing objects smartly based on clustering. K EYWORDS Transfer learning, Image clustering, Robotics application 1. Introduction In a close future, it is likely to see industrial robots performing tasks requiring to make complex decisions. In this perspective, we have developed an automatic sorting and storing application (see section 1.1.2) consisting in clustering images based on semantic content and storing the objects in boxes accordingly using a serial robot (https://youtu.be/ NpZIwY3H-gE). This application can have various uses in shopfloors (workspaces can be organized before the workday, unsorted goods can be sorted before packaging, ...), which motivated this study of image-set clustering.
    [Show full text]