DOCSLIB.ORG

Maximally Consistent Sampling and the Jaccard Index of Probability Distributions

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Maximally Consistent Sampling and the Jaccard Index of Probability Distributions Maximally Consistent Sampling and the Jaccard Index of Probability Distributions Anonymous Anonymous Abstract—We introduce simple, efficient algorithms for MinHashes of vectors of positive integers, yielding a computing a MinHash of a probability distribution, suitable collision probability of for both sparse and dense data, with equivalent running P times to the state of the art for both cases. The collision i min (xi; yi) JW (x; y) = P probability of these algorithms is a new measure of the max (x ; y ) similarity of positive vectors which we investigate in detail. i i i We describe the sense in which this collision probability is Subsequent work[7][8][9] has improved the efficiency optimal for any Locality Sensitive Hash based on sampling. of the second algorithm and extended it to arbitrary We argue that this similarity measure is more useful for probability distributions than the similarity pursued by positive weights, while still achieving JW as the collision other algorithms for weighted MinHash, and is the natural probability. generalization of the Jaccard index. JW is one of several generalizations of the Jaccard Index Terms—Locality Sensitive Hashing, Retrieval, index to non-negative vectors. It is useful because it is MinHash, Jaccard index, Jensen-Shannon divergence monotonic with respect to the L1 distance between x and y when they are both L1 normalized, but it is unnatural I. INTRODUCTION in many ways for probability distributions. 2 MinHashing[1] is a popular Locality Sensitive Hash- If we convert sets to binary vectors x; y, with xi; yi ing algorithm for clustering and retrieval on large f0; 1g, then JW (x; y) = J(X; Y ). But if we convert these datasets. Its extreme simplicity and efficiency, as well vectors to probabilityn o distributionsn byo normalizing them 2 1 2 1 6 as its natural pairing with MapReduce and key-value so that xi 0; jXj ; yi 0; jY j , then JW (x; y) = datastores, have made it a basic building block in J(X; Y ) when jXj= 6 jY j. The correspondence breaks many domains, particularly document clustering[1][2] and it no longer generalizes the Jaccard index. Instead, and graph clustering[3][4]. for jXj > jY j, Given a finite set U, and a uniformly random per- 7! jX \ Y j mutation π, the map X arg mini2X π(i) provides a JW (x; y) = < J(X; Y ): (1) representation of any subset X of U that is stable under jX n Y j + jXj small changes to X. If X, Y are both subsets of U the As a consequence, switching a system from an un- well-known Jaccard index[5] is defined as weighted MinHash to a MinHash based on JW will jX \ Y j generally decrease the collision probabilities. J(X; Y ) = jX [ Y j Furthermore, JW is insensitive to important differ- ences between probability distributions. It counts all Stability of arg mini2X π(i) with respect to X is quan- differences on an element in the same linear scale tified by regardless of the mass the distributions share on that [ ] element. For instance, JW ((a; b; c; 0); (a; b; 0; c)) = Pr arg min π(i) = arg min π(i) = J(X; Y ) JW ((a + c; b); (a; b + c)). This makes it a poor choice i2X i2Y when the ultimate goal is to measure similarity using Practically, this random permutation is generated by an expression based in information-theory or likelihood applying some hash function to each i with a fixed where having differing support typically results in the random seed, hence “MinHashing.” worst possible score. In order to hash objects other than sets, Chum et al.[6] For a drop-in replacement for the Jaccard index that introduced two algorithms for incorporating weights treats its input as a probability distribution, we’d like it in the computation of MinHashes. The first algorithm to have the following properties. associates constant global weights with the set members, 1) Scale invariant. suitable forP idf weighting. The collision probability that 2) Not lower than the Jaccard Index when applied to w results is Pi2X\Y i . The second algorithm computes discrete uniform distributions. i2X[Y wi 1 Algorithm 1: P-MinHash. We require only a single for retrieving documents that are similar under JW exponentially distributed hash per nonzero element. than JW itself (and consequently, sometimes better for input : Vector x, Seed s retrieving based on L1-distance.) output: Stable sample from x II. P-MINHASH AND ITS MATCH PROBABILITY foreach i where xi > 0 do − log(UniformNonZeroFloat(i;s)) Let h :[n] ! (0; 1] be a pseudo-random hash mapping ei xi ≤ ≤ end every element 1 i n to an independent uniform random value in (0; 1]. Over a non-negative vector x, return arg mini ei define − log h(i) H(x) := arg min 3) Sensitive to changes in support, in a similar way i xi to information-based measures. For brevity, we will be using the extended real number 4) Easily achievable as a collision probability. system in which 1=1 := 0. Each term is an exponen- JW fails all but the last. tially distributed random variable with rate xi, 1) It isn’t scale invariant, JW (αx; y) =6 JW (x; y). [ ] − log h(i) − 2) If the vectors are normalized to make it scale Pr < y = 1 − e xiy; invariant, the values drop below the corresponding xi Jaccard index (equation 1.) so it follows that 3) It is insensitive to changes in support. Pxi 4) Good algorithms exist, but they are non-trivial. Pr [H(x) = i] = : i xi Existing work has thoroughly explored improvements to Chum et al.’s second algorithm, while leaving their This well known and beautiful property of expo- first untouched. In this work we instead take their first nential random variables derives from the fact that algorithm as a starting point. We extend it to arbitrary Pr [min(x; y) > α] = Pr [x > α] Pr [y > α]. positive vectors (rather than sets with constant global Theorem II.1. For any two nonnegative vectors x; y 2 weights) and analyze the result. In doing so, we find Rn +, that the collision probability is a new generalization of Pr [H(x) = H(y)] = JP (x; y): the Jaccard Index to positive vectors, which we here call − log h(i) JP . Proof. Any monotonic transform on will not X xi 1 change the arg min, so multiplying each xi by a positive JP (x; y) = P ( ): (2) xj yj α won’t either. H(αx) = H(x). Thus for xi; yi > 0 fi : xi;yi>0g j max x ; y [ ( ) ( )] i i x y The names used here, JW and JP , are chosen to reflect Pr [i = H(x) = H(y)] = Pr i = H = H xi yi how each function interprets x and y, and the conditions ( ) ( ) − under which they match the original Jaccard index. JW By definition, H x = H y = i means log h(i) ≤ xi yi xi=xi treats a difference in magnitude the same as any other − − − log h(j) and log h(i) ≤ log h(j) , for j =6 i, or difference, so treats vectors as “weighted sets.” JP is xj =xi yi=yi yj =yi equivalently, scale invariant, so any input is treated the same as a ( ) corresponding probability distribution. − − − ≤ log h(j) log h(j) The primary contribution of this work is to derive and log h(i) min ; xj=xi yj=yi analyze JP , and to show that in many situations where − log h(j) the objects being hashed are probability distributions, JP ≤ ( ): max xj ; yj is a more useful collision probability than JW . xi yi We will describe the sense in which JP is an optimal i i collision probability for any LSH based on sampling. We Now we desire a new vector z such that H(z ) = i will prove that if the collision probability of a sampling if and only if H(x=xi) = H(y=yi) = i. This requires − log h(j) − log( h(j) ) i P that i i = xj yj . Thus for fixed i, zj = algorithm exceeds J on one pair, it must sacrifice zj =zi max ; ( ) xi yi P xj yj collisions on a pair that has higher J . max ; x(i yi ) We will motivate JP ’s utility by showing experimen- P . Consequently, max xk ; yk tally that it has a tighter relationship to the Jensen- k xi yi Shannon divergence than JW , and is more closely cen- [ ] 1 Pr H(zi) = i = P ( ) tered around the Jaccard index than JW . We will even x y max j ; j show empirically that in some circumstances, it is better j xi yi 2 (a) Pr [H(x) = i] (b) Pr [H(y) = i] (c) Pr [H(x);H(y) = i; j] (d) Pr [H(x) = H(y) = i] Fig. 1: The P-MinHash process can be interpreted geometrically as dividing the unit simplex into smaller simplexes proportional to the mass of each term, then selecting the same point in each simplex as its representative. Pictured here are x = (0:5; 0:4; 0:1); y = (0:2; 0:4; 0:4). Colors are assigned to unique values of i, or (i; j) in the case of the joint distributions. The sum of the remaining colored areas in (d) is proportional to JP (x; y). Repeating this process for all i in the intersection yields exterior face. As a result, a uniformly chosen point on X 1 the unit simplex will land in one of the sub-simplices Pr [H(x) = H(y)] = P ( ) with probability given by x.
Load more

Recommended publications
  • A Generalized Framework for Analyzing Taxonomic, Phylogenetic, and Functional Community Structure Based on Presence–Absence Data
    Article A Generalized Framework for Analyzing Taxonomic, Phylogenetic, and Functional Community Structure Based on Presence–Absence Data János Podani 1,2,*, Sandrine Pavoine 3 and Carlo Ricotta 4 1 Department of Plant Systematics, Ecology and Theoretical Biology, Institute of Biology, Eötvös University, H-1117 Budapest, Hungary 2 MTA-ELTE-MTM Ecology Research Group, Eötvös University, H-1117 Budapest, Hungary 3 Centre d'Ecologie et des Sciences de la Conservation (CESCO), Muséum national d’Histoire naturelle, CNRS, Sorbonne Université, 75005 Paris, France ; [email protected] 4 Department of Environmental Biology, University of Rome ‘La Sapienza’, 00185 Rome, Italy; [email protected] * Correspondence: [email protected] Received: 17 October 2018; Accepted: 6 November 2018; Published: 12 November 2018 Abstract: Community structure as summarized by presence–absence data is often evaluated via diversity measures by incorporating taxonomic, phylogenetic and functional information on the constituting species. Most commonly, various dissimilarity coefficients are used to express these aspects simultaneously such that the results are not comparable due to the lack of common conceptual basis behind index definitions. A new framework is needed which allows such comparisons, thus facilitating evaluation of the importance of the three sources of extra information in relation to conventional species-based representations. We define taxonomic, phylogenetic and functional beta diversity of species assemblages based on the generalized Jaccard dissimilarity index. This coefficient does not give equal weight to species, because traditional site dissimilarities are lowered by taking into account the taxonomic, phylogenetic or functional similarity of differential species in one site to the species in the other. These, together with the traditional, taxon- (species-) based beta diversity are decomposed into two additive fractions, one due to taxonomic, phylogenetic or functional excess and the other to replacement.
    [Show full text]
  • A Similarity-Based Community Detection Method with Multiple Prototype Representation Kuang Zhou, Arnaud Martin, Quan Pan
    A similarity-based community detection method with multiple prototype representation Kuang Zhou, Arnaud Martin, Quan Pan To cite this version: Kuang Zhou, Arnaud Martin, Quan Pan. A similarity-based community detection method with multiple prototype representation. Physica A: Statistical Mechanics and its Applications, Elsevier, 2015, pp.519-531. 10.1016/j.physa.2015.07.016. hal-01185866 HAL Id: hal-01185866 https://hal.archives-ouvertes.fr/hal-01185866 Submitted on 21 Aug 2015 HAL is a multi-disciplinary open access L’archive ouverte pluridisciplinaire HAL, est archive for the deposit and dissemination of sci- destinée au dépôt et à la diffusion de documents entific research documents, whether they are pub- scientifiques de niveau recherche, publiés ou non, lished or not. The documents may come from émanant des établissements d’enseignement et de teaching and research institutions in France or recherche français ou étrangers, des laboratoires abroad, or from public or private research centers. publics ou privés. A similarity-based community detection method with multiple prototype representation Kuang Zhoua,b,∗, Arnaud Martinb, Quan Pana aSchool of Automation, Northwestern Polytechnical University, Xi'an, Shaanxi 710072, PR China bDRUID, IRISA, University of Rennes 1, Rue E. Branly, 22300 Lannion, France Abstract Communities are of great importance for understanding graph structures in social networks. Some existing community detection algorithms use a single prototype to represent each group. In real applications, this may not adequately model the different types of communities and hence limits the clustering performance on social networks. To address this problem, a Similarity-based Multi-Prototype (SMP) community detection approach is proposed in this paper.
    [Show full text]
  • Similarity Indices I: What Do They Measure?
    NRC-1 Addendum Similarity Indices I: What Do They Measure? by J. W. Johnston November 1976 Prepared for The Nuclear Regulatory Commission Pacific Northwest Laboratories NOTICE This report was prepared as an account of work sponsored by the United States Government. Neither the United States nor the Energy Research and Development Administration, nor any of their employees, nor any of their contractors, subcontractors, or their employees, makes any warranty, express or implied, or assumes any legal liability or responsibility for the accuracy, completeness or usefulness of any information, apparatus, product or process disclosed, or represents that its use would not infringe privately owned rights. PACIFIC NORTHWEST LABORATORY operated by BATTELLE for the ENERGY RESEARCH AND DEVELOPMENT ADMIFulSTRATION Under Contract E Y-76-C-06-7830 Printed in the Ucited States of America Avai!able from hcat~onalTechntcal lnformat~onService U.S. Department of Commerce 5285 Port Royal Road Spr~ngf~eld,Virginta 22151 Price: Pr~ntedCODV $*; Microfiche $3.00 NTI5 'Pages Selling Price ERDA-RL RICHL&\D M 4 BNWL-2152 NRC-1 Addendum Similarity Indices I: What Do They Measure? by J.W. Johnston November 1976 Prepared for The Nuclear Regulatory Commission Battelle Pacific Northwest Laboratories Richland, Washington 99352 SUMMARY The characteristics of 25 similarity indices used in studies of eco- logical communities were investigated. The type of data structure, to which these indices are frequently applied, was described as consisting of vectors of measurements on attributes (species) observed in a set of sam- ples. A genera1 similarity index was characterized as the result of a two step process defined on a pair of vectors.
    [Show full text]
  • A New Similarity Measure Based on Simple Matching Coefficient for Improving the Accuracy of Collaborative Recommendations
    I.J. Information Technology and Computer Science, 2019, 6, 37-49 Published Online June 2019 in MECS (http://www.mecs-press.org/) DOI: 10.5815/ijitcs.2019.06.05 A New Similarity Measure Based on Simple Matching Coefficient for Improving the Accuracy of Collaborative Recommendations Vijay Verma Computer Engineering Department, National Institute of Technology, Kurukshetra, Haryana, India-136119 E-mail: [email protected] Rajesh Kumar Aggarwal Computer Engineering Department, National Institute of Technology, Kurukshetra, Haryana, India-136119 E-mail: [email protected] Received: 13 February 2019; Accepted: 22 March 2019; Published: 08 June 2019 Abstract—Recommender Systems (RSs) are essential tools of an e-commerce portal in making intelligent I. INTRODUCTION decisions for an individual to obtain product Nowadays, the world wide web (or simply web) serves recommendations. Neighborhood-based approaches are as an important means for e-commerce businesses. The traditional techniques for collaborative recommendations rapid growth of e-commerce businesses presents a and are very popular due to their simplicity and efficiency. potentially overwhelming number of alternatives to their Neighborhood-based recommender systems use users, this frequently results in the information overload numerous kinds of similarity measures for finding similar problem. Recommender Systems (RSs) are the critical users or items. However, the existing similarity measures tools for avoiding information overload problem and function only on common ratings between a pair of users provide useful suggestions that may be relevant to the (i.e. ignore the uncommon ratings) thus do not utilize all users [1]. RSs help individuals by providing personalized ratings made by a pair of users.
    [Show full text]
  • The Duality of Similarity and Metric Spaces
    applied sciences Article The Duality of Similarity and Metric Spaces OndˇrejRozinek 1,2,* and Jan Mareš 1,3,* 1 Department of Process Control, Faculty of Electrical Engineering and Informatics, University of Pardubice, 530 02 Pardubice, Czech Republic 2 CEO at Rozinet s.r.o., 533 52 Srch, Czech Republic 3 Department of Computing and Control Engineering, Faculty of Chemical Engineering, University of Chemistry and Technology, 166 28 Prague, Czech Republic * Correspondence: [email protected] (O.R.); [email protected] (J.M.) Abstract: We introduce a new mathematical basis for similarity space. For the first time, we describe the relationship between distance and similarity from set theory. Then, we derive generally valid relations for the conversion between similarity and a metric and vice versa. We present a general solution for the normalization of a given similarity space or metric space. The derived solutions lead to many already used similarity and distance functions, and combine them into a unified theory. The Jaccard coefficient, Tanimoto coefficient, Steinhaus distance, Ruzicka similarity, Gaussian similarity, edit distance and edit similarity satisfy this relationship, which verifies our fundamental theory. Keywords: similarity metric; similarity space; distance metric; metric space; normalized similarity metric; normalized distance metric; edit distance; edit similarity; Jaccard coefficient; Gaussian similarity 1. Introduction Mathematical spaces have been studied for centuries and belong to the basic math- ematical theories, which are used in various real-world applications [1]. In general, a mathematical space is a set of mathematical objects with an associated structure. This Citation: Rozinek, O.; Mareš, J. The structure can be specified by a number of operations on the objects of the set.
    [Show full text]
  • Distributed Clustering Algorithm for Large Scale Clustering Problems
    IT 13 079 Examensarbete 30 hp November 2013 Distributed clustering algorithm for large scale clustering problems Daniele Bacarella Institutionen för informationsteknologi Department of Information Technology Abstract Distributed clustering algorithm for large scale clustering problems Daniele Bacarella Teknisk- naturvetenskaplig fakultet UTH-enheten Clustering is a task which has got much attention in data mining. The task of finding subsets of objects sharing some sort of common attributes is applied in various fields Besöksadress: such as biology, medicine, business and computer science. A document search engine Ångströmlaboratoriet Lägerhyddsvägen 1 for instance, takes advantage of the information obtained clustering the document Hus 4, Plan 0 database to return a result with relevant information to the query. Two main factors that make clustering a challenging task are the size of the dataset and the Postadress: dimensionality of the objects to cluster. Sometimes the character of the object makes Box 536 751 21 Uppsala it difficult identify its attributes. This is the case of the image clustering. A common approach is comparing two images using their visual features like the colors or shapes Telefon: they contain. However, sometimes they come along with textual information claiming 018 – 471 30 03 to be sufficiently descriptive of the content (e.g. tags on web images). Telefax: The purpose of this thesis work is to propose a text-based image clustering algorithm 018 – 471 30 00 through the combined application of two techniques namely Minhash Locality Sensitive Hashing (MinHash LSH) and Frequent itemset Mining. Hemsida: http://www.teknat.uu.se/student Handledare: Björn Lyttkens Lindén Ämnesgranskare: Kjell Orsborn Examinator: Ivan Christoff IT 13 079 Tryckt av: Reprocentralen ITC ”L’arte rinnova i popoli e ne rivela la vita.
    [Show full text]
  • Communication-Efficient Jaccard Similarity for High-Performance Distributed Genome Comparisons
    Communication-Efficient Jaccard similarity for High-Performance Distributed Genome Comparisons Maciej Besta1y, Raghavendra Kanakagiri5y, Harun Mustafa1;3;4, Mikhail Karasikov1;3;4, Gunnar Ratsch¨ 1;3;4, Torsten Hoefler1, Edgar Solomonik2 1Department of Computer Science, ETH Zurich 2Department of Computer Science, University of Illinois at Urbana-Champaign 3University Hospital Zurich, Biomedical Informatics Research 4SIB Swiss Institute of Bioinformatics 5Department of Computer Science, Indian Institute of Technology Tirupati yBoth authors contributed equally to this work. Abstract—The Jaccard similarity index is an important mea- sequencing data sets), machine learning (clustering, object sure of the overlap of two sets, widely used in machine learning, recognition) [8], information retrieval (plagiarism detection), computational genomics, information retrieval, and many other and many others [82], [33], [65], [68], [70], [92], [80]. areas. We design and implement SimilarityAtScale, the first communication-efficient distributed algorithm for computing the We focus on the application of the Jaccard similarity index Jaccard similarity among pairs of large datasets. Our algo- and distance to genetic distances between high-throughput rithm provides an efficient encoding of this problem into a DNA sequencing samples, a problem of high importance multiplication of sparse matrices. Both the encoding and sparse for different areas of computational biology [71], [92], [29]. matrix product are performed in a way that minimizes data Genetic distances are frequently used to approximate the movement in terms of communication and synchronization costs. We apply our algorithm to obtain similarity among all pairs of evolutionary distances between the species or populations a set of large samples of genomes. This task is a key part of represented in sequence sets in a computationally intractable modern metagenomics analysis and an evergrowing need due to manner [71], [63], [92], [29].
    [Show full text]
  • Near-Duplication Document Detection Using Weight One Permutation Hashing
    Regular Paper Journal of Computing Science and Engineering, Vol. 13, No. 2, June 2019, pp. 78-88 Near-Duplication Document Detection Using Weight One Permutation Hashing Xinpan Yuan, Songlin Wang, and Xiaojun Deng* School of Computer Science, Hunan University of Technology, Zhuzhou, China [email protected], [email protected], [email protected] Abstract As a standard algorithm for efficiently calculating set similarity, Minwise hashing is widely used to detect text similarity. The major drawback associated with Minwise hashing is expensive preprocessing. One permutation hashing (OPH) is proposed in order to reduce the number of random permutations. OPH divides the space Ω evenly into k bins, and selects the smallest nonzero value in each bin to re-index the selected elements. We propose a weight one permutation hashing (WOPH) by dividing the entire space Ω into k1 and k2 bins and sampling k1 and k2 in proportion to form a weighted kw. WOPH has a wider range of precision by expanding the proportion of w1 and w2 to different accuracy levels of the user. The variance of WOPH can be rapidly decreased first and then slowly decreased, although the final variance is the same as OPH with the same k. We combined the dynamic double filter with WOPH to reduce the calculation time by eliminat- ing unnecessary comparison in advance. For example, for a large number of real data with low similarity accompanied by high threshold queries, the filter reduces the comparison of WOPH by 85%. Category: Databases / Data Mining Keywords: One Permutation Hashing; WOPH; Bin; Non-uniform partition; Weighting; Dynamic double filter I.
    [Show full text]
  • Comparing Clusterings - an Overview ∗
    Comparing Clusterings - An Overview ∗ Silke Wagner Dorothea Wagner January 12, 2007 1 Introduction As the amount of data we nowadays have to deal with becomes larger and larger, the methods that help us to detect structures in the data and to identify interesting subsets in the data become more and more important. One of these methods is clustering, i.e. segmenting a set of elements into subsets such that the elements in each subset are somehow ”similiar” to each other and elements of different subsets are ”unsimilar”. In the literature we can find a large variety of clustering algorithms, each having certain advantages but also certain drawbacks. Typical questions that arise in this context comprise: • Is the algorithm sensitive to small perturbations, i.e. can small changes in the data (so-called ”noise”) entail large changes in the clustering? • Is the algorithm sensitive to the order of the data, i.e. can another order of the data result in a very different clustering? • How similar are the solutions of two different algorithms? • If an optimal solution is available: How close is the clustering solution to the optimal one? For examining these aspects, it would be desirable to have a ”measure” for the similarity between two clusterings or for their distance1. In a more gen- eral context, it can be necessary to combine different clusterings to a single one, i.e. calculating a ”mean value” of the clusterings. Possible applications are: ∗The authors gratefully acknowledge financial support from the European Commission within DELIS (contract no. 001907). 1Every similarity measure can be transformed into a distance measure and vice versa.
    [Show full text]
  • A Minhash Approach for Fast Scanpath Classification
    A MinHash approach for fast scanpath classification David Geisler Nora Castner University of Tuebingen University of Tuebingen Germany Germany [email protected] [email protected] Gjergji Kasneci Enkelejda Kasneci University of Tuebingen University of Tuebingen Germany Germany [email protected] [email protected] ABSTRACT Quantization 1 Word Frequency 2 The visual scanpath describes the shift of visual attention over time. R J F A Characteristic patterns in the attention shifts allow inferences about Q T S cognitive processes, performed tasks, intention, or expertise. To anal- S 0 U B S yse such patterns, the scanpath is often represented as a sequence K 2 D V of symbols that can be used to calculate a similarity score to other Y P O C Z scanpaths. However, as the length of the scanpath or the number of L possible symbols increases, established methods for scanpath sim- ilarity become inefficient, both in terms of runtime and memory T T T T T T T T T T T kNN Classification Q Q Q Q Q Q Q Q Q Q Q consumption. We present a MinHash approach for efficient scanpath E A A A A A A A A A A A similarity calculation. Our approach shows competitive results in I K L M N O P Q R S T U X W S H clustering and classification of scanpaths compared to established Classification 1 4 MinHash 3 methods such as Needleman-Wunsch, but at a fraction of the required S0 S2 J(S0,S2)= runtime.
    [Show full text]
  • Efficient Pairwise Neuroimage Analysis Using the Soft Jaccard Index and 3D Keypoint Sets 3
    IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. XX, NO. XX, XXXX 2021 1 Efficient Pairwise Neuroimage Analysis using the Soft Jaccard Index and 3D Keypoint Sets Laurent Chauvin, Kuldeep Kumar, Christian Desrosiers, William Wells III and Matthew Toews Abstract— We propose a novel pairwise distance mea- relationships, and also characteristics shared across the popu- sure between image keypoint sets, for the purpose of large- lation including racial ancestry, sex, hereditary disease status, scale medical image indexing. Our measure generalizes the etc. [2], [3]. The brain is the center of cognition and a Jaccard index to account for soft set equivalence (SSE) be- tween keypoint elements, via an adaptive kernel framework complex organ, tightly coupled to the genetic evolution of modeling uncertainty in keypoint appearance and geom- animals and in particular that of the human species. To what etry. A new kernel is proposed to quantify the variability degree is the human brain phenotype coupled to the underlying of keypoint geometry in location and scale. Our distance genotype? How does the brain image manifold vary locally 2 measure may be estimated between O(N ) image pairs in with genotype, i.e. immediate relatives sharing 25-100% of O(N log N) operations via keypoint indexing. Experiments report the first results for the task of predicting family their polymorphic genes, or between broad groups defined by relationships from medical images, using 1010 T1-weighted subtle genetic factors such as racial ancestry or sex? MRI brain volumes of 434 families including monozygotic Large, publicly available databases allow us to investigate and dizygotic twins, siblings and half-siblings sharing these questions from aggregated MRI and genetic data, e.g.
    [Show full text]
  • An Experiment with Intelligent Scissors Interactive Segmentation Technique on Specific Images
    International Journal of Scientific and Research Publications, Volume 6, Issue 7, July 2016 60 ISSN 2250-3153 An experiment with Intelligent Scissors interactive segmentation technique on specific images Anuja Deshpande*, Pradeep Dahikar*, Pankaj Agrawal** * [email protected], Department of Electronics, Kamla Nehru Mahavidyalaya, Sakkardara, Nagpur, Maharashtra, India 440007 **Dept. of Electronics and Communication Engg, G. H. Raisoni Academy of Engineering and Technology, Nagpur. Maharashtra, India 440016 Abstract- The paper discusses an experiment on Intelligent using different techniques for different types of images or Scissors interactive segmentation technique on selected images segmentation needs is the most suitable approach to solving the with the aim to assess suitability for specific types of images. In segmentation problem [10]. The purpose of segmentation also this experiment accuracy of the segmentation has been assessed needs to be considered while evaluating segmentation technique, by computing Jaccard Index, Dice Coefficient and Hausdorrf since the segmentation results should also be extensible to other Distance on segmented and ground truth images. In this applications [11]. experiment Intelligent Scissors was not found to be effective on certain images, particularly when images have similarity in As stated in [9], using Intelligent Scissors, Contour is created by foreground and background, at least in few areas, if not in finding an edge along the path using the algorithm. To create the entirety, along the object boundary. Contour, initial seeds are placed manually near the object boundary and traced manually using mouse gestures. When the Index Terms- Intelligent Scissors, suitability, Jaccard Index, mouse position comes close to the object edge, a live-wire Dice Coefficient, Hausdorrf Distance boundary “snaps” to the edge, and wraps around.
    [Show full text]