View of to Begin With, AHC Treats Each Data Object As a Separate the Data to the User, Frequently Achieved Using Clustering
Total Page：16
File Type：pdf, Size：1020Kb
BioData Mining BioMed Central Research Open Access Fast approximate hierarchical clustering using similarity heuristics Meelis Kull*1,2 and Jaak Vilo*1,2 Address: 1Institute of Computer Science, University of Tartu, Liivi 2, 50409 Tartu, Estonia and 2Quretec Ltd. Ülikooli 6a, 51003 Tartu, Estonia Email: Meelis Kull* - [email protected]; Jaak Vilo* - [email protected] * Corresponding authors Published: 22 September 2008 Received: 8 April 2008 Accepted: 22 September 2008 BioData Mining 2008, 1:9 doi:10.1186/1756-0381-1-9 This article is available from: http://www.biodatamining.org/content/1/1/9 © 2008 Kull and Vilo; licensee BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. Abstract Background: Agglomerative hierarchical clustering (AHC) is a common unsupervised data analysis technique used in several biological applications. Standard AHC methods require that all pairwise distances between data objects must be known. With ever-increasing data sizes this quadratic complexity poses problems that cannot be overcome by simply waiting for faster computers. Results: We propose an approximate AHC algorithm HappieClust which can output a biologically meaningful clustering of a large dataset more than an order of magnitude faster than full AHC algorithms. The key to the algorithm is to limit the number of calculated pairwise distances to a carefully chosen subset of all possible distances. We choose distances using a similarity heuristic based on a small set of pivot objects. The heuristic efficiently finds pairs of similar objects and these help to mimic the greedy choices of full AHC. Quality of approximate AHC as compared to full AHC is studied with three measures. The first measure evaluates the global quality of the achieved clustering, while the second compares biological relevance using enrichment of biological functions in every subtree of the clusterings. The third measure studies how well the contents of subtrees are conserved between the clusterings. Conclusion: The HappieClust algorithm is well suited for large-scale gene expression visualization and analysis both on personal computers as well as public online web applications. The software is available from the URL http://www.quretec.com/HappieClust Background we concentrate on hierarchical methods that model the Various types of biological data resulting from high- data in a tree structure and thus leave more freedom to the throughput experiments require analysis, often consisting user. of many steps. The first steps tend to be unsupervised and require little or no input from the user, while the further Probably the most well-known hierarchical clustering steps need more human-computer interaction. One possi- method is agglomerative hierarchical clustering (AHC). ble starting point of interaction is showing an overview of To begin with, AHC treats each data object as a separate the data to the user, frequently achieved using clustering. cluster. The following agglomeration steps iteratively merge the two nearest clusters. Simultaneously, the clus- Partitioning-based clustering methods like K-means split tering tree (dendrogram) is built from leaves towards root, the data into non-overlapping clusters [1]. In this article, where merging of clusters is depicted as a common parent Page 1 of 14 (page number not for citation purposes) BioData Mining 2008, 1:9 http://www.biodatamining.org/content/1/1/9 for two subtrees. Finally, the user is shown the dendro- The speed problem is caused by the fact that all pairwise gram, possibly together with data objects visualized at comparisons have to be performed between the data each leaf. There are different versions of AHC, depending objects. Each comparison involves a distance calculation on how the distance between clusters is measured. It can using the distance measure specified by the user, where be defined as the distance between the closest or furthest short distances correspond to more similar objects. Not a objects, resulting in single or complete linkage AHC, single pair can be omitted, because already in the first iter- respectively. Other known strategies include UPGMA ation AHC uses each of the pairwise distances to find the (unweighted pair-group method using arithmetic aver- closest pair. In the following we take the only route to a ages) and WPGMA (weighted pair-group method using significant speedup and drop the requirement of calculat- arithmetic averages) clustering. For a more detailed expo- ing all pairwise distances. This certainly affects the den- sition refer to Legendre & Legendre [2]. drogram and the conclusions that can be drawn from it. In this article, we present a fast approximate hierarchical AHC is often used to visualize microarray gene expression clustering algorithm which uses similarity heuristics to data. Originally suggested by Eisen et al. [3], there now retain high quality of clustering. exist many tools for performing AHC of gene expression data, e.g. Cluster 3.0 [4] coupled with Java Treeview [5], Our approximate hierarchical clustering algorithm first MultiExperiment Viewer (MeV 4.0) [6], and EP:NG [7] calculates a subset of pairwise distances and then per- with clustering tools from EPCLUST 1.0 [8]. An important forms agglomerative clustering based on these distances. issue of using this simple and intuitive procedure is its The logic of the agglomeration part is straightforward – speed. Some of the best implementations can achieve make merging decisions based on the known distances quadratic speed in the number of data objects [9]. At least only. The quality of the resulting clustering depends heav- quadratic running time is required for general distance ily on the subset of distances chosen in the first step. measures, as exemplified by the need of first finding a sin- Therefore, we use heuristics to rapidly find pairs of similar gle closest pair of objects from all possible candidate data objects. The heuristics are based on the observation pairs. that if two objects are close enough to each other then the distance to any third object from both of these is approx- As will be shown in the Results section, full AHC of genes imately the same. We turn this observation upside down in medium-sized expression datasets can be performed in and look for pairs of objects which are approximately at a couple of minutes on a workstation computer. Cluster- the same distance from several other objects (which we ing large datasets with 20000+ genes and several thou- refer to as pivots). These pairs are more probably similar sands of conditions can already take several hours. One and form the subset of pairs for which the distance is cal- possible workaround involves decreasing the size of the culated. Experiments show that adding a random set of dataset by filtering out the genes that change too little over pairs to the pool further raises the quality of clustering. the conditions. However, dropping a large set of data in a Pivots have earlier been used in the database community very early stage of study might be undesirable. Secondly, for several similarity search algorithms [14]. when the number of experimental conditions is large, almost all genes show some differential expression. The running time of the HappieClust algorithm can be Another possibility is to first run K-means and then apply easily controlled by choosing the number of distances that AHC on the centers of obtained clusters. The downside is are to be calculated and used in the clustering. Such a fea- that K-means is inherently limited in the choice of the dis- ture is very useful in web-based applications where users tance measure. K-medoids is free from this constraint, but expect fast response time. has quadratic complexity [10]. The experiments to evaluate speed and quality of Happie- To our knowledge, significant speedups to AHC have only Clust have been carried out using two datasets: DNA been achieved in specialized computational environ- microarray survey of gene expression in normal human ments, such as parallel computing [11] and graphics hard- tissues [15] and human gene expression atlas of public ware acceleration [12]. Another direction of research has microarray data [16] (accession E-TABM-185 of ArrayEx- involved the development of hierarchical clustering meth- press [17]). ods in a different framework, such as DIANA [10] or SOTA [13]. Both of these are divisive hierarchical clustering Methods algorithms as opposed to AHC. There is a clear need for Approximate hierarchical clustering fast AHC for interactive tools, web services and other Suppose we have a dataset X with n data objects, X = (x1, applications, running on personal and workstation com- x2,...,xn) and a user-defined distance measure d(xi, xj) to puters or servers with many simultaneous users. state how similar the two objects xi and xj are, for any i and j. We assume the distance measure to be a semi-metric, i.e. Page 2 of 14 (page number not for citation purposes) BioData Mining 2008, 1:9 http://www.biodatamining.org/content/1/1/9 it must be non-negative and symmetric and the distance Our approximate hierarchical clustering works exactly the from an object to itself must be 0: same way except that the graph is not required to be com- plete, i.e. the algorithm works even if only a subset of pair- ≥=… dx(,ij x )01 ij , , ,; n wise distances is known. This is a clear advantage in case ==… of large datasets and/or complex distance measures. dx(,ij x ) dx (,) ji x ij ,1 , ,; n Therefore, we redefine the distance between clusters to be dx(,) x= 0 i==…1, ,n . ii the minimum (or maximum or average) of only the The dataset can have duplicate objects and we do not known distances measured between the objects of the two require the triangle inequality d(xi, xj) ≤ d(xi, xk) + d(xk, xj).