DOCSLIB.ORG

Communication-Efficient Jaccard Similarity for High-Performance Distributed Genome Comparisons

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Communication-Efficient Jaccard Similarity for High-Performance Distributed Genome Comparisons Communication-Efficient Jaccard similarity for High-Performance Distributed Genome Comparisons Maciej Besta1y, Raghavendra Kanakagiri5y, Harun Mustafa1;3;4, Mikhail Karasikov1;3;4, Gunnar Ratsch¨ 1;3;4, Torsten Hoefler1, Edgar Solomonik2 1Department of Computer Science, ETH Zurich 2Department of Computer Science, University of Illinois at Urbana-Champaign 3University Hospital Zurich, Biomedical Informatics Research 4SIB Swiss Institute of Bioinformatics 5Department of Computer Science, Indian Institute of Technology Tirupati yBoth authors contributed equally to this work. Abstract—The Jaccard similarity index is an important mea- sequencing data sets), machine learning (clustering, object sure of the overlap of two sets, widely used in machine learning, recognition) [8], information retrieval (plagiarism detection), computational genomics, information retrieval, and many other and many others [82], [33], [65], [68], [70], [92], [80]. areas. We design and implement SimilarityAtScale, the first communication-efficient distributed algorithm for computing the We focus on the application of the Jaccard similarity index Jaccard similarity among pairs of large datasets. Our algo- and distance to genetic distances between high-throughput rithm provides an efficient encoding of this problem into a DNA sequencing samples, a problem of high importance multiplication of sparse matrices. Both the encoding and sparse for different areas of computational biology [71], [92], [29]. matrix product are performed in a way that minimizes data Genetic distances are frequently used to approximate the movement in terms of communication and synchronization costs. We apply our algorithm to obtain similarity among all pairs of evolutionary distances between the species or populations a set of large samples of genomes. This task is a key part of represented in sequence sets in a computationally intractable modern metagenomics analysis and an evergrowing need due to manner [71], [63], [92], [29]. This enables or facilitates analyz- the increasing availability of high-throughput DNA sequencing ing the evolutionary history of populations and species [63], data. The resulting scheme is the first to enable accurate the investigation of the biodiversity of a sample, as well as Jaccard distance derivations for massive datasets, using large- scale distributed-memory systems. We package our routines in other applications [92], [64]. However, the enormous sizes a tool, called GenomeAtScale, that combines the proposed algo- of today’s high-throughput sequencing datasets often make rithm with tools for processing input sequences. Our evaluation it infeasible to compute the exact values of J or dJ [79]. on real data illustrates that one can use GenomeAtScale to Recent works (such as Mash [63]) propose approximations, effectively employ tens of thousands of processors to reach new for example using the MinHash representation of J(A; B), frontiers in large-scale genomic and metagenomic analysis. While GenomeAtScale can be used to foster DNA research, the more which is the primary locality-sensitive hashing (LSH) scheme general underlying SimilarityAtScale algorithm may be used for used for genetic comparisons [57]. Yet, these approximations high-performance distributed similarity computations in other often lead to inaccurate approximations of dJ for highly data analytics application domains. similar pairs of sequence sets, and tend to be ineffective Index Terms—Distributed Jaccard Distance, Distributed Jac- for computation of a distance between highly dissimilar sets card similarity, Genome Sequence Distance, Metagenome Se- quence Distance, High-Performance Genome Processing, k-Mers, unless very large sketch sizes are used [63], as noted by Matrix-Matrix Multiplication, Cyclops Tensor Framework the Mash authors [63]. Thus, developing a scalable, high- Code and data: https://github.com/cyclops-community/jaccard-ctf performance, and accurate scheme for computing the Jaccard arXiv:1911.04200v3 [cs.CE] 11 Nov 2020 similarity index is an open problem of considerable relevance I. INTRODUCTION for genomics computations and numerous other applications The concept of similarity has gained much attention in in general data analytics. different areas of data analytics [81]. A popular method of Yet, there is little research on high-performance distributed assessing a similarity of two entities is based on computing derivations of either J or dJ . Existing works target very small their Jaccard similarity index J(A; B) [49], [59]. J(A; B) is datasets (e.g., 16MB of raw input text [31]), only provide a statistic that assesses the overlap of two sets A and B. simulated evaluation for experimental hardware [56], focus It is defined as the ratio between the size of the intersec- on inefficient MapReduce solutions [26], [6], [86] that need tion of A and B and the size of the union of A and B: asymptotically more communication due to using the allreduce J(A; B) = jA \ Bj=jA [ Bj. A closely related notion, the collective communication pattern over reducers [47], are lim- Jaccard distance dJ = 1 − J, measures the dissimilarity of ited to a single server [36], [66], [69], use an approach based sets. The general nature and simplicity of the Jaccard similarity on deriving the Cartesian product, which may require quadratic and distance has allowed for their wide use in numerous do- space (infeasible for the large input datasets considered in mains, for example computational genomics (comparing DNA this work) [48], or do not target parallelism or distribution at all [7]. Most works target novel use cases of J and dJ [20], are publicly available in order to enable interpretability and [22], [83], [53], or mathematical foundations of these mea- reproducibility, and to facilitate its integration into current and sures [54], [62]. We attribute this to two factors. First, many future genomics analysis pipelines. domains (for example genomics) only recently discovered the II. JACCARD MEASURES:DEFINITIONS AND IMPORTANCE usefulness of Jaccard measures [92]. Second, the rapid growth of the availability of high-throughput sequencing data and We start with defining the Jaccard measures and with its increasing relevance to genetic analysis have meant that discussing their applications in different domains. The most previous complex genetic analysis methods are falling out of important symbols used in this work are listed in Table I. favor due to their poor scaling properties [92]. To the best of While we focus on high-performance computations of dis- our knowledge, no work provides a scalable high-performance tances between genome sequences, our design and implemen- solution for computing J or dJ . tation are generic and applicable to any other use case. In this work, we design and implement SimilarityAtScale: the first algorithm for distributed, fast, and scalable derivation General Jaccard measures: of the Jaccard Similarity index and Jaccard distance. We fol- J(X; Y ) The Jaccard similarity index of sets X and Y . dJ (X; Y ) The Jaccard distance between sets X and Y ; dJ = 1 − J. low [44], devising an algorithm based on an algebraic formula- Algebraic Jaccard measures (SimilarityAtScale), details in III-A: tion of Jaccard similarity matrix computation as sparse matrix The number of possible data values (attributes) and data samples, m; n multiplication. Our main contribution is a communication- one sample contains zero, one, or more values (attributes). avoiding algorithm for both preprocessing the sparse matrices ⊗; Matrix-vector (MV) and vector dot products. The indicator matrix (it determines the presence of data values as well as computing Jaccard similarity in parallel via sparse A m×n in data samples), A 2 B ; B = f0; 1g. matrix multiplication. We use our algorithm as a core element The similarity and distance matrices with the values of Jaccard S; D n×n of GenomeAtScale, a tool that we develop to facilitate high- measures between all pairs of data samples; S; D 2 R . B; C Intermediate matrices used to compute S, B; C 2 n×n. performance genetic distance computation. GenomeAtScale N Genomics and metagenomics computations (GenomeAtScale): enables the first massively-parallel and provably accurate cal- k The number of single nucleotides in a genome subsequence. culations of Jaccard distances between genomes. By maintain- m; n The number of analyzed k-mers and genome data samples. ing compatibility with standard bioinformatics data formats, TABLE I: The most important symbols used in the paper. we allow for GenomeAtScale to be seamlessly integrated into existing analysis pipelines. Our performance analysis illus- A. Fundamental Definitions trates the scalability of our solutions. In addition, we bench- The Jaccard similarity index [59] is a statistic used to assess mark GenomeAtScale on both large (2,580 human RNASeq the similarity of sets. It is defined as the ratio of the set experiments) and massive (almost all public bacterial and intersection cardinality and the set union cardinality, viral whole-genome sequencing experiments) scales, and we make this data publicly available to foster high-performance jX \ Y j jX \ Y j distributed genomics research. Despite our focus on genomics J(X; Y ) = = ; jX [ Y j jXj + jY j − jX \ Y j data, the algebraic formulation and the implementation of our routines for deriving Jaccard measures are generic and can be where X and Y are sample sets. If both X and Y are directly used in other settings. empty, the index is defined as J(X; Y ) = 1. One can then Specifically, our work makes the following contributions. use J(X; Y ) to define the Jaccard distance dJ (X; Y ) = 1−J(X; Y ), which assesses the dissimilarity of sets and which • We design SimilarityAtScale, the first communication- is a proper metric (on the collection of all finite sets). efficient distributed algorithm to compute the Jaccard sim- We are interested in the computation of similarity between ilarity index and distance. all pairs of a collection of sets (in the context of genomics, this • We use our algorithm as a backend to GenomeAtScale, the collection contains samples from one or multiple genomes). first tool that enables fast, scalable, accurate, and large-scale Let X = fX1;:::;Xng be the set of data samples.
Load more

Recommended publications
  • A Generalized Framework for Analyzing Taxonomic, Phylogenetic, and Functional Community Structure Based on Presence–Absence Data
    Article A Generalized Framework for Analyzing Taxonomic, Phylogenetic, and Functional Community Structure Based on Presence–Absence Data János Podani 1,2,*, Sandrine Pavoine 3 and Carlo Ricotta 4 1 Department of Plant Systematics, Ecology and Theoretical Biology, Institute of Biology, Eötvös University, H-1117 Budapest, Hungary 2 MTA-ELTE-MTM Ecology Research Group, Eötvös University, H-1117 Budapest, Hungary 3 Centre d'Ecologie et des Sciences de la Conservation (CESCO), Muséum national d’Histoire naturelle, CNRS, Sorbonne Université, 75005 Paris, France ; [email protected] 4 Department of Environmental Biology, University of Rome ‘La Sapienza’, 00185 Rome, Italy; [email protected] * Correspondence: [email protected] Received: 17 October 2018; Accepted: 6 November 2018; Published: 12 November 2018 Abstract: Community structure as summarized by presence–absence data is often evaluated via diversity measures by incorporating taxonomic, phylogenetic and functional information on the constituting species. Most commonly, various dissimilarity coefficients are used to express these aspects simultaneously such that the results are not comparable due to the lack of common conceptual basis behind index definitions. A new framework is needed which allows such comparisons, thus facilitating evaluation of the importance of the three sources of extra information in relation to conventional species-based representations. We define taxonomic, phylogenetic and functional beta diversity of species assemblages based on the generalized Jaccard dissimilarity index. This coefficient does not give equal weight to species, because traditional site dissimilarities are lowered by taking into account the taxonomic, phylogenetic or functional similarity of differential species in one site to the species in the other. These, together with the traditional, taxon- (species-) based beta diversity are decomposed into two additive fractions, one due to taxonomic, phylogenetic or functional excess and the other to replacement.
    [Show full text]
  • A Similarity-Based Community Detection Method with Multiple Prototype Representation Kuang Zhou, Arnaud Martin, Quan Pan
    A similarity-based community detection method with multiple prototype representation Kuang Zhou, Arnaud Martin, Quan Pan To cite this version: Kuang Zhou, Arnaud Martin, Quan Pan. A similarity-based community detection method with multiple prototype representation. Physica A: Statistical Mechanics and its Applications, Elsevier, 2015, pp.519-531. 10.1016/j.physa.2015.07.016. hal-01185866 HAL Id: hal-01185866 https://hal.archives-ouvertes.fr/hal-01185866 Submitted on 21 Aug 2015 HAL is a multi-disciplinary open access L’archive ouverte pluridisciplinaire HAL, est archive for the deposit and dissemination of sci- destinée au dépôt et à la diffusion de documents entific research documents, whether they are pub- scientifiques de niveau recherche, publiés ou non, lished or not. The documents may come from émanant des établissements d’enseignement et de teaching and research institutions in France or recherche français ou étrangers, des laboratoires abroad, or from public or private research centers. publics ou privés. A similarity-based community detection method with multiple prototype representation Kuang Zhoua,b,∗, Arnaud Martinb, Quan Pana aSchool of Automation, Northwestern Polytechnical University, Xi'an, Shaanxi 710072, PR China bDRUID, IRISA, University of Rennes 1, Rue E. Branly, 22300 Lannion, France Abstract Communities are of great importance for understanding graph structures in social networks. Some existing community detection algorithms use a single prototype to represent each group. In real applications, this may not adequately model the different types of communities and hence limits the clustering performance on social networks. To address this problem, a Similarity-based Multi-Prototype (SMP) community detection approach is proposed in this paper.
    [Show full text]
  • Similarity Indices I: What Do They Measure?
    NRC-1 Addendum Similarity Indices I: What Do They Measure? by J. W. Johnston November 1976 Prepared for The Nuclear Regulatory Commission Pacific Northwest Laboratories NOTICE This report was prepared as an account of work sponsored by the United States Government. Neither the United States nor the Energy Research and Development Administration, nor any of their employees, nor any of their contractors, subcontractors, or their employees, makes any warranty, express or implied, or assumes any legal liability or responsibility for the accuracy, completeness or usefulness of any information, apparatus, product or process disclosed, or represents that its use would not infringe privately owned rights. PACIFIC NORTHWEST LABORATORY operated by BATTELLE for the ENERGY RESEARCH AND DEVELOPMENT ADMIFulSTRATION Under Contract E Y-76-C-06-7830 Printed in the Ucited States of America Avai!able from hcat~onalTechntcal lnformat~onService U.S. Department of Commerce 5285 Port Royal Road Spr~ngf~eld,Virginta 22151 Price: Pr~ntedCODV $*; Microfiche $3.00 NTI5 'Pages Selling Price ERDA-RL RICHL&\D M 4 BNWL-2152 NRC-1 Addendum Similarity Indices I: What Do They Measure? by J.W. Johnston November 1976 Prepared for The Nuclear Regulatory Commission Battelle Pacific Northwest Laboratories Richland, Washington 99352 SUMMARY The characteristics of 25 similarity indices used in studies of eco- logical communities were investigated. The type of data structure, to which these indices are frequently applied, was described as consisting of vectors of measurements on attributes (species) observed in a set of sam- ples. A genera1 similarity index was characterized as the result of a two step process defined on a pair of vectors.
    [Show full text]
  • A New Similarity Measure Based on Simple Matching Coefficient for Improving the Accuracy of Collaborative Recommendations
    I.J. Information Technology and Computer Science, 2019, 6, 37-49 Published Online June 2019 in MECS (http://www.mecs-press.org/) DOI: 10.5815/ijitcs.2019.06.05 A New Similarity Measure Based on Simple Matching Coefficient for Improving the Accuracy of Collaborative Recommendations Vijay Verma Computer Engineering Department, National Institute of Technology, Kurukshetra, Haryana, India-136119 E-mail: [email protected] Rajesh Kumar Aggarwal Computer Engineering Department, National Institute of Technology, Kurukshetra, Haryana, India-136119 E-mail: [email protected] Received: 13 February 2019; Accepted: 22 March 2019; Published: 08 June 2019 Abstract—Recommender Systems (RSs) are essential tools of an e-commerce portal in making intelligent I. INTRODUCTION decisions for an individual to obtain product Nowadays, the world wide web (or simply web) serves recommendations. Neighborhood-based approaches are as an important means for e-commerce businesses. The traditional techniques for collaborative recommendations rapid growth of e-commerce businesses presents a and are very popular due to their simplicity and efficiency. potentially overwhelming number of alternatives to their Neighborhood-based recommender systems use users, this frequently results in the information overload numerous kinds of similarity measures for finding similar problem. Recommender Systems (RSs) are the critical users or items. However, the existing similarity measures tools for avoiding information overload problem and function only on common ratings between a pair of users provide useful suggestions that may be relevant to the (i.e. ignore the uncommon ratings) thus do not utilize all users [1]. RSs help individuals by providing personalized ratings made by a pair of users.
    [Show full text]
  • The Duality of Similarity and Metric Spaces
    applied sciences Article The Duality of Similarity and Metric Spaces OndˇrejRozinek 1,2,* and Jan Mareš 1,3,* 1 Department of Process Control, Faculty of Electrical Engineering and Informatics, University of Pardubice, 530 02 Pardubice, Czech Republic 2 CEO at Rozinet s.r.o., 533 52 Srch, Czech Republic 3 Department of Computing and Control Engineering, Faculty of Chemical Engineering, University of Chemistry and Technology, 166 28 Prague, Czech Republic * Correspondence: [email protected] (O.R.); [email protected] (J.M.) Abstract: We introduce a new mathematical basis for similarity space. For the first time, we describe the relationship between distance and similarity from set theory. Then, we derive generally valid relations for the conversion between similarity and a metric and vice versa. We present a general solution for the normalization of a given similarity space or metric space. The derived solutions lead to many already used similarity and distance functions, and combine them into a unified theory. The Jaccard coefficient, Tanimoto coefficient, Steinhaus distance, Ruzicka similarity, Gaussian similarity, edit distance and edit similarity satisfy this relationship, which verifies our fundamental theory. Keywords: similarity metric; similarity space; distance metric; metric space; normalized similarity metric; normalized distance metric; edit distance; edit similarity; Jaccard coefficient; Gaussian similarity 1. Introduction Mathematical spaces have been studied for centuries and belong to the basic math- ematical theories, which are used in various real-world applications [1]. In general, a mathematical space is a set of mathematical objects with an associated structure. This Citation: Rozinek, O.; Mareš, J. The structure can be specified by a number of operations on the objects of the set.
    [Show full text]
  • Distributed Clustering Algorithm for Large Scale Clustering Problems
    IT 13 079 Examensarbete 30 hp November 2013 Distributed clustering algorithm for large scale clustering problems Daniele Bacarella Institutionen för informationsteknologi Department of Information Technology Abstract Distributed clustering algorithm for large scale clustering problems Daniele Bacarella Teknisk- naturvetenskaplig fakultet UTH-enheten Clustering is a task which has got much attention in data mining. The task of finding subsets of objects sharing some sort of common attributes is applied in various fields Besöksadress: such as biology, medicine, business and computer science. A document search engine Ångströmlaboratoriet Lägerhyddsvägen 1 for instance, takes advantage of the information obtained clustering the document Hus 4, Plan 0 database to return a result with relevant information to the query. Two main factors that make clustering a challenging task are the size of the dataset and the Postadress: dimensionality of the objects to cluster. Sometimes the character of the object makes Box 536 751 21 Uppsala it difficult identify its attributes. This is the case of the image clustering. A common approach is comparing two images using their visual features like the colors or shapes Telefon: they contain. However, sometimes they come along with textual information claiming 018 – 471 30 03 to be sufficiently descriptive of the content (e.g. tags on web images). Telefax: The purpose of this thesis work is to propose a text-based image clustering algorithm 018 – 471 30 00 through the combined application of two techniques namely Minhash Locality Sensitive Hashing (MinHash LSH) and Frequent itemset Mining. Hemsida: http://www.teknat.uu.se/student Handledare: Björn Lyttkens Lindén Ämnesgranskare: Kjell Orsborn Examinator: Ivan Christoff IT 13 079 Tryckt av: Reprocentralen ITC ”L’arte rinnova i popoli e ne rivela la vita.
    [Show full text]
  • Near-Duplication Document Detection Using Weight One Permutation Hashing
    Regular Paper Journal of Computing Science and Engineering, Vol. 13, No. 2, June 2019, pp. 78-88 Near-Duplication Document Detection Using Weight One Permutation Hashing Xinpan Yuan, Songlin Wang, and Xiaojun Deng* School of Computer Science, Hunan University of Technology, Zhuzhou, China [email protected], [email protected], [email protected] Abstract As a standard algorithm for efficiently calculating set similarity, Minwise hashing is widely used to detect text similarity. The major drawback associated with Minwise hashing is expensive preprocessing. One permutation hashing (OPH) is proposed in order to reduce the number of random permutations. OPH divides the space Ω evenly into k bins, and selects the smallest nonzero value in each bin to re-index the selected elements. We propose a weight one permutation hashing (WOPH) by dividing the entire space Ω into k1 and k2 bins and sampling k1 and k2 in proportion to form a weighted kw. WOPH has a wider range of precision by expanding the proportion of w1 and w2 to different accuracy levels of the user. The variance of WOPH can be rapidly decreased first and then slowly decreased, although the final variance is the same as OPH with the same k. We combined the dynamic double filter with WOPH to reduce the calculation time by eliminat- ing unnecessary comparison in advance. For example, for a large number of real data with low similarity accompanied by high threshold queries, the filter reduces the comparison of WOPH by 85%. Category: Databases / Data Mining Keywords: One Permutation Hashing; WOPH; Bin; Non-uniform partition; Weighting; Dynamic double filter I.
    [Show full text]
  • Comparing Clusterings - an Overview ∗
    Comparing Clusterings - An Overview ∗ Silke Wagner Dorothea Wagner January 12, 2007 1 Introduction As the amount of data we nowadays have to deal with becomes larger and larger, the methods that help us to detect structures in the data and to identify interesting subsets in the data become more and more important. One of these methods is clustering, i.e. segmenting a set of elements into subsets such that the elements in each subset are somehow ”similiar” to each other and elements of different subsets are ”unsimilar”. In the literature we can find a large variety of clustering algorithms, each having certain advantages but also certain drawbacks. Typical questions that arise in this context comprise: • Is the algorithm sensitive to small perturbations, i.e. can small changes in the data (so-called ”noise”) entail large changes in the clustering? • Is the algorithm sensitive to the order of the data, i.e. can another order of the data result in a very different clustering? • How similar are the solutions of two different algorithms? • If an optimal solution is available: How close is the clustering solution to the optimal one? For examining these aspects, it would be desirable to have a ”measure” for the similarity between two clusterings or for their distance1. In a more gen- eral context, it can be necessary to combine different clusterings to a single one, i.e. calculating a ”mean value” of the clusterings. Possible applications are: ∗The authors gratefully acknowledge financial support from the European Commission within DELIS (contract no. 001907). 1Every similarity measure can be transformed into a distance measure and vice versa.
    [Show full text]
  • A Minhash Approach for Fast Scanpath Classification
    A MinHash approach for fast scanpath classification David Geisler Nora Castner University of Tuebingen University of Tuebingen Germany Germany [email protected] [email protected] Gjergji Kasneci Enkelejda Kasneci University of Tuebingen University of Tuebingen Germany Germany [email protected] [email protected] ABSTRACT Quantization 1 Word Frequency 2 The visual scanpath describes the shift of visual attention over time. R J F A Characteristic patterns in the attention shifts allow inferences about Q T S cognitive processes, performed tasks, intention, or expertise. To anal- S 0 U B S yse such patterns, the scanpath is often represented as a sequence K 2 D V of symbols that can be used to calculate a similarity score to other Y P O C Z scanpaths. However, as the length of the scanpath or the number of L possible symbols increases, established methods for scanpath sim- ilarity become inefficient, both in terms of runtime and memory T T T T T T T T T T T kNN Classification Q Q Q Q Q Q Q Q Q Q Q consumption. We present a MinHash approach for efficient scanpath E A A A A A A A A A A A similarity calculation. Our approach shows competitive results in I K L M N O P Q R S T U X W S H clustering and classification of scanpaths compared to established Classification 1 4 MinHash 3 methods such as Needleman-Wunsch, but at a fraction of the required S0 S2 J(S0,S2)= runtime.
    [Show full text]
  • Efficient Pairwise Neuroimage Analysis Using the Soft Jaccard Index and 3D Keypoint Sets 3
    IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. XX, NO. XX, XXXX 2021 1 Efficient Pairwise Neuroimage Analysis using the Soft Jaccard Index and 3D Keypoint Sets Laurent Chauvin, Kuldeep Kumar, Christian Desrosiers, William Wells III and Matthew Toews Abstract— We propose a novel pairwise distance mea- relationships, and also characteristics shared across the popu- sure between image keypoint sets, for the purpose of large- lation including racial ancestry, sex, hereditary disease status, scale medical image indexing. Our measure generalizes the etc. [2], [3]. The brain is the center of cognition and a Jaccard index to account for soft set equivalence (SSE) be- tween keypoint elements, via an adaptive kernel framework complex organ, tightly coupled to the genetic evolution of modeling uncertainty in keypoint appearance and geom- animals and in particular that of the human species. To what etry. A new kernel is proposed to quantify the variability degree is the human brain phenotype coupled to the underlying of keypoint geometry in location and scale. Our distance genotype? How does the brain image manifold vary locally 2 measure may be estimated between O(N ) image pairs in with genotype, i.e. immediate relatives sharing 25-100% of O(N log N) operations via keypoint indexing. Experiments report the first results for the task of predicting family their polymorphic genes, or between broad groups defined by relationships from medical images, using 1010 T1-weighted subtle genetic factors such as racial ancestry or sex? MRI brain volumes of 434 families including monozygotic Large, publicly available databases allow us to investigate and dizygotic twins, siblings and half-siblings sharing these questions from aggregated MRI and genetic data, e.g.
    [Show full text]
  • An Experiment with Intelligent Scissors Interactive Segmentation Technique on Specific Images
    International Journal of Scientific and Research Publications, Volume 6, Issue 7, July 2016 60 ISSN 2250-3153 An experiment with Intelligent Scissors interactive segmentation technique on specific images Anuja Deshpande*, Pradeep Dahikar*, Pankaj Agrawal** * [email protected], Department of Electronics, Kamla Nehru Mahavidyalaya, Sakkardara, Nagpur, Maharashtra, India 440007 **Dept. of Electronics and Communication Engg, G. H. Raisoni Academy of Engineering and Technology, Nagpur. Maharashtra, India 440016 Abstract- The paper discusses an experiment on Intelligent using different techniques for different types of images or Scissors interactive segmentation technique on selected images segmentation needs is the most suitable approach to solving the with the aim to assess suitability for specific types of images. In segmentation problem [10]. The purpose of segmentation also this experiment accuracy of the segmentation has been assessed needs to be considered while evaluating segmentation technique, by computing Jaccard Index, Dice Coefficient and Hausdorrf since the segmentation results should also be extensible to other Distance on segmented and ground truth images. In this applications [11]. experiment Intelligent Scissors was not found to be effective on certain images, particularly when images have similarity in As stated in [9], using Intelligent Scissors, Contour is created by foreground and background, at least in few areas, if not in finding an edge along the path using the algorithm. To create the entirety, along the object boundary. Contour, initial seeds are placed manually near the object boundary and traced manually using mouse gestures. When the Index Terms- Intelligent Scissors, suitability, Jaccard Index, mouse position comes close to the object edge, a live-wire Dice Coefficient, Hausdorrf Distance boundary “snaps” to the edge, and wraps around.
    [Show full text]
  • Distributed Link Prediction in Large Scale Graphs Using Apache Spark
    Aristotle University of Thessaloniki Postgraduate program on «Internet and World Wide Web» Department of Informatics Distributed Link Prediction in Large Scale Graphs using Apache Spark Master’s thesis of Anastasios Theodosiou Supervisor: Associate Prof. Apostolos N. Papadopoulos Thesis submitted in fulfillment of the requirements for the degree of Master on Internet and Web Computing at the School of Informatics of the Aristotle University of Thessaloniki. Thessaloniki, February 2019 Abstract Online social networks (OSNs) such as Facebook, Instagram, Twitter, and LinkedIn are aware of high acceptance since they enable users to share digital content (links, pictures, videos), express or share opinions and expand their social circle, by making new friends. All these kinds of interactions that users participate in the lead to the evolution and expansion of social networks over time. OSNs support users, providing them with friend recommendations based on the existing explicit friendship network but also to their preferences resulting from their interaction with the net which they gradually build. Link prediction methods try to predict the likelihood of a future connection between two nodes in a given network. This can be used in biological networks to infer protein-protein interactions or to suggest possible friends to a user in an OSN. In e-commerce it can help to build recommendation systems such as the "people who bought this may also buy" feature, e.g., on Amazon and in the security domain link prediction can help to identify hidden groups of people who use potential violence and criminals. Due to the massive amounts of data that is collected today, the need for scalable approaches arises to this problem.
    [Show full text]