IJRECE VOL. 7 ISSUE 4 OCT.-DEC 2019 ISSN: 2393-9028 (PRINT) | ISSN: 2348-2281 (ONLINE) Comparative study of Unsupervised learning algorithms on Multispectral SatelliteImages PrasadKaviti1, Valli Kumari Vatsavayi2 1Computer Science and Systems Engineering, Andhra University, AP, India 2Computer Science and Systems Engineering, Andhra University, AP, India ([email protected], [email protected]) Unlabeled image data is handled by image clustering approach Abstract—In computer vision, one of the evolving fields with since it is an unsupervised learning method.Specific image lot of applications is image clustering. The challenging vector should be derived for different use cases. There are problem is to choose appropriate clustering algorithm for a different clustering algorithms introduced [7]. Each clustering given image data set and the clustering algorithm have a algorithm follows an assumption of grouping data points with strong impact on clustering accuracy. There arevarious state of similar qualities and features in a feature space. Thus, all art clustering algorithms available with a dependency on nearby points are clustered with some sort of similarities. We parameter tuning. K-Means algorithm is one of such standard cannot make sure of this assumption because high volumes of algorithms. K-Means has the requirement of prior image data are being collected in real world scenarios. So, it is specification of number of clusters for centroids initialization. challenging to handle huge amounted, unstructured and But number is unknown in most of the cases.Another popular unlabeled image collection. technique is Agglomerative clustering which depends on The clustering algorithms family differ by approach [22] and connectivity matrix and requires the number of clusters to be differ using various metrics for measuring the distance between specified in prior. The connectivity matrix consumes more data objects (Euclidean distance, Manhattan distance, etc.). space hence more complexity. Another popular clustering The criteria to join two clusters must be chosen by the user and technique is Mean-Shift clustering which is nonparametric in this selection may differs from user to user. The main nature. Mean-Shift clustering algorithm required bandwidth objectives of clustering algorithms are scalability, dealing with parameter ‘h’ to be tuned even though it is nonparametric. different types of attributes, discovering clusters with arbitrary Spectral Clustering is another standard clustering shape, minimal requirements of domain knowledge to algorithmdepends on similarity matrix which leads to a determine input parameters, ability to deal with noise and memory constraint. DBSCAN is another standard algorithm outliers, insensitivity to order of input records, high which requires two parameters eps,minpts to be tuned and it is dimensionality and usability. Also, there are wide ranges of density based algorithm. In this paper, a comparative study on issues that are faced during clustering like: dealing with large all above mentioned techniques is presented. Experimentation number of dimensions and large number of data items which is done on a sentinel image for each of these methods. can become an overhead because of time complexity, defining distance measure should be made appropriately which is a Keywords—Clustering; K-Means; DBSCAN; Agglomerative tedious task when it comes to multidimensional spaces, etc. clustering; Mean-Shift; Spectral clustering Hence care should be taken while applying the clustering algorithm to images. In computer vision, one of the evolving fields with lot of applications is image clustering. The challenging problem is to I. INTRODUCTION choose appropriate clustering algorithm for a given image data One of the interesting fields of Machine Learning is set and the clustering algorithm have a strong impact on Unsupervised learning or Clustering where similar groups are clustering accuracy. There are various state of art clustering classifiedfrom the datasets. Image clustering partitions image algorithms available with a dependency on parameter data into clusters based on similarities. Similarity might be tuning.Some of them are K-Means [1][10], DBSCAN [9], images looking similar or similar size or similar pixel Agglomerative clustering [8], Spectral clustering [15][16], distribution, similar background etc. Similarity definition Affinity propagation and Mean-Shift [2][3]. But these changes with different set of rules for every methodology. INTERNATIONAL JOURNAL OF RESEARCH IN ELECTRONICS AND COMPUTER ENGINEERING A UNIT OF I2OR 63 | P a g e IJRECE VOL. 7 ISSUE 4 OCT.-DEC 2019 ISSN: 2393-9028 (PRINT) | ISSN: 2348-2281 (ONLINE) algorithms require tuning of some parameters. K-Means algorithm is one of such standard algorithms. K-Means has the Algorithm requirement of prior specification of number of clusters for centroids initialization. But number is unknown in most of the Input: Set of data points 푃 = {푝1, 푝2, … , 푝푛}, K – Number of cases. Another popular technique is Agglomerative clustering clusters desired which depends on connectivity matrix and requires the number Output: K number of clusters of clusters to be specified in prior. The connectivity matrix 1. Choose K points randomly from P, say 푀 = consumes more space hence more complexity. Another {푚1, 푚2, … , 푚푘} popular clustering technique is Mean-Shift clustering [3] which 2. Do is nonparametric in nature [2]. Mean-Shift clustering algorithm i) Select a point 푝푖 ∈ 푃 required bandwidth parameter ‘h’ to be tuned [5] even though ii) Calculate distance between 푝 and all points of M it is nonparametric. Spectral Clustering [17][18] is another 푖 iii) Assign 푝 to the nearest point in M standard clustering algorithm depends on similarity matrix 푖 which leads to a memory constraint. DBSCAN [6][11] is For all ‘n’ points another standard algorithm which requires two parameters 3. Calculate mean of all newly assigned points of M and eps,minpts to be tuned and it is density based update {푚 , 푚 , … , 푚 } algorithm.Affinity propagation [19] is another clustering 1 2 푘 4. Repeat steps 2,3 until no change in cluster centres. algorithm depends on similarity matrix. In this paper, a survey is presented on all the mentioned clustering algorithms. For results comparison, a sentinel image B. Agglomerative clustering: is considered. The agglomerative clustering [8] is the most common type of hierarchical clustering used to group objects in clusters based II. EXISTING CLUSTERING ALGORITHMS on their similarity. Agglomerative clustering works in a “bottom-up” manner. That is, each object is initially considered as a single-element A. K-Means cluster (leaf). At each step of the algorithm, the two clusters that are the most similar are combined into a new bigger K-means clustering[6][10] is an unsupervised learning cluster (nodes). This procedure is iterated until all points are algorithmto cluster unlabeled data which is non categorized member of just one single big cluster (root). The result is a and not grouped. The goal of this algorithm is to find groups in tree-based representation of the objects, named dendrogram. the data, with the number of groups represented by the variable K. The algorithm works iteratively to assign each data Advantages: point to one of K groups based on the features that are No specification of number of clusters. provided. Data points are clustered based on feature similarity. Easy to implement and performs well in some cases. First is the data assignment step in which each centroid defines Easy to decidethe number of clusters by observing the one of the clusters. In this step, each data point is assigned to dendrogram its nearest centroid, based on the squared Euclidean distance. The next step is Centroid update. In this step, the centroids are Disadvantages: recomputed. This is done by taking the mean of all data points Once the instances are assigned, we cannot undo the assigned to that centroid's cluster. assignment. So the instances cannot be moved around The algorithm iterates between steps one and two until a after the previous step. stopping criterion is met (i.e., no data points change clusters, the sum of the distances is minimized, or some maximum Time complexity is more for large datasets number of iterations is reached). Final resultsare impacted by the initial seeds Advantages: Sensitive to outliers Easy to implement K-Meanscomputationally faster because of its number Algorithm of variables Disadvantages: Input: Set of data points 푃 = {푝1, 푝2, … , 푝푛}, A- Adjacency or Difficult to predict the number of clusters Connectivity matrix to show distance between points. Output: A dendogram of ordered points. Final results could be strongly impacted bye initial seeds. INTERNATIONAL JOURNAL OF RESEARCH IN ELECTRONICS AND COMPUTER ENGINEERING A UNIT OF I2OR 64 | P a g e IJRECE VOL. 7 ISSUE 4 OCT.-DEC 2019 ISSN: 2393-9028 (PRINT) | ISSN: 2348-2281 (ONLINE) 1. Assume each point 푝푖 ∈ 푃 as an individual cluster, i.e, the cluster set C be 퐶 = {{푝1}, {푝2}, … , {푝푛}} . Algorithm 2. Do i) From A, take two closest clusters in C and Input: Set of data points 푃 = {푝1, 푝2, … , 푝푛}, eps – Maximum combine them as a single cluster. radius of neighborhood, minpts – Minimum number of points to form a cluster. ii) Update A, C with new clusters. Output: Set of clusters formed Until a single cluster remains in C 1. Randomly choose a point 푝푖 2. Form a cluster C with other points 푝푗 ∈ 푃 | 푑푖푠푡푎푛푐푒(푝푖, 푝푗) < 푒푝푠 , i.e, density reachable C. DBSCAN from 푝푖. 3. If (number of points in C >minpts) then form a The DBSCAN [9] algorithm should be used to find cluster. associations and structures in data that are hard to find 4. Else 푖푓(푝 == 푏표푟푑푒푟푝표푖푛푡|| 푛표푠푖푒푝표푖푛푡) then manually but that can be relevant and useful to find patterns 푖 choose another point from P. and predict trends. Based on a set of points, DBSCAN groups together points that 5. Repeat from step 1 to 4 until all the points in P are are close to each other based on a distance measurement processed.