String & Text Mining in Bioinformatics

Total Page:16

File Type:pdf, Size:1020Kb

String & Text Mining in Bioinformatics Data Mining in Bioinformatics Day 9: String & Text Mining in Bioinformatics Karsten Borgwardt February 21 to March 4, 2011 Machine Learning & Computational Biology Research Group MPIs Tübingen Karsten Borgwardt: Data Mining in Bioinformatics, Page 1 Why compare sequences? Protein sequences Proteins are chains of amino acids. 20 different types of amino acids can be found in protein sequences. Protein sequence changes over time by mutations, dele- tion, insertions. Different protein sequences may diverge from one com- mon ancestor. Their sequences may differ slightly, yet their function is often conserved. Karsten Borgwardt: Data Mining in Bioinformatics, Page 2 Why compare sequences? Biological Question: Biologists are interested in the reverse direction: Given two protein sequences, is it likely that they origi- nate from the same common ancestor? Computational Challenge: How to measure similarity between two protein se- quence, or equivalently: How to measure similarity between two strings Kernel Challenge: How to measure similarity between two strings via a ker- nel function In short: How to define a string kernel Karsten Borgwardt: Data Mining in Bioinformatics, Page 3 History of sequence comparison First phase Smith-Waterman BLAST Second phase Profiles Hidden Markov Models Third phase PSI-Blast SAM-T98 Fourth phase Kernels Karsten Borgwardt: Data Mining in Bioinformatics, Page 4 Sequence comparison: Phase 1 Idea Measure pairwise similarities between sequences with gaps Methods Smith-Waterman dynamic programming high accuracy slow (O(n2)) BLAST faster heuristic alternative with sufficient accuracy searches common substrings of fixed length extends these in both directions performs gapped alignment Karsten Borgwardt: Data Mining in Bioinformatics, Page 5 Sequence comparison: Phase 2 Idea Collect aggregate statistics from a family of sequences Compare this statistics to a single unlabeled protein Methods Hidden Markov Models (HMMs) Markov process with hidden and observable parame- ters Forward algorithm determines probability if given se- quence is output of particular HMM Profiles Profiles of sequence families are derived by multiple sequence alignment Given sequence is compared to this profile Karsten Borgwardt: Data Mining in Bioinformatics, Page 6 Sequence comparison: Phase 3 Idea Create single models from database collections of homologous sequences Methods PSI-BLAST Position specific iterative BLAST Profile from highest scoring hits in initial BLAST runs Position weighting according to degree of conserva- tion Iteration of these steps SAM-T98, now SAM-T02 database search with HMM from multiple sequence alignment Karsten Borgwardt: Data Mining in Bioinformatics, Page 7 Phase 4: Kernels and SVMs General idea Model differences between classes of sequences Use SVM classifier to distinguish classes Use kernel to measure similarity between strings Kernels for Protein Sequences SVM-Fisher kernel Composite kernel Motif kernel String kernel Karsten Borgwardt: Data Mining in Bioinformatics, Page 8 SVM-Fisher method General idea Combine HMMs and SVMs for sequence classification Won best-paper award at ISMB 1999 Sequence representation fixed-length vector components are transition and emission probabilities transformation into Fisher score Karsten Borgwardt: Data Mining in Bioinformatics, Page 9 SVM-Fisher method Algorithm Model protein family F as HMM Transform query protein X into fixed-length vector via HMM Compute kernel between X and positive and negative examples of the protein family Advantages allows to incorporate prior knowledge allows to deal with missing data is interpretable outperforms competing methods Karsten Borgwardt: Data Mining in Bioinformatics, Page 10 Composition kernels General idea Model sequence by amino acid content Bin amino acids w.r.t physico-chemical properties Sequence representation feature vector of amino acid frequencies physico-chemical properties include predicted secondary structure, hydrophobicity, normalized van der Waals volume, polarity, polarizability useful database: AAindex Karsten Borgwardt: Data Mining in Bioinformatics, Page 11 Motif kernels General idea Conserved motif in amino acid sequences indicate structural and functional relationship Model sequence s as a feature vector f representing motifs i-th component of f is 1 , s contains i-th motif Motif databases PROSITE eMOTIFs BLOCKS+ combines several databases Generated by manual construction multiple sequence alignment Karsten Borgwardt: Data Mining in Bioinformatics, Page 12 Pairwise comparison kernels General idea Employ empirical kernel map on Smith-Waterman/Blast scores Advantage Utilizes decades of practical experience with Blast Disadvantage High computational cost (O(m3)) Alleviation Employ Blast instead of Smith-Waterman Use vectorization set for empirical map only Karsten Borgwardt: Data Mining in Bioinformatics, Page 13 Phase 4: String Kernels General idea Count common substrings in two strings A substring of length k is a k-mer Variations Assign weights to k-mers Allow for mismatches Allow for gaps Include substitutions Include wildcards Karsten Borgwardt: Data Mining in Bioinformatics, Page 14 Spectrum Kernel General idea For each l-mer α 2 Σl, the coordinate indexed by α will be the number of times α occurs in sequence x. Then the l-spectrum feature map is Spectrum Φl (x) = (φα(x))α2Σl Here φα(x) is the # occurrences of α in x. The spectrum kernel is now the inner product in the fea- ture space defined by this map: 0 Spectrum Spectrum 0 kSpectrum(x; x ) =< Φl (x); Φl (x ) > Sequences are deemed the more similar, the more com- mon substrings they contain Karsten Borgwardt: Data Mining in Bioinformatics, Page 15 Spectrum Kernel Principle Spectrum kernel: Count exactly common k-mers Karsten Borgwardt: Data Mining in Bioinformatics, Page 16 Mismatch Kernel General idea Do not enforce strictly exact matches Define mismatch neighborhood of an l-mer α with up to m mismatches: Mismatch φ(l;m) (α) = (φβ(α))β2Σl For a sequence x of any length, the map is then ex- tended as Mismatch X Mismatch φ(l;m) (x) = (φ(l;m) (α)) l−mers α in x The mismatch kernel is now the inner product in feature space defined by: Mismatch 0 Mismatch Mismatch 0 k(l;m) (x; x ) =< Φ(l;m) (x); Φ(l;m) (x ) > Karsten Borgwardt: Data Mining in Bioinformatics, Page 17 Mismatch Kernel Principle Mismatch kernel: Count common k-mers with max. m mismatches Karsten Borgwardt: Data Mining in Bioinformatics, Page 18 Gappy Kernel General idea Allow for gaps in common substrings ! “subsequences” A g-mer then contributes to all its l-mer subsequences Gap φ(g;l)(α) = (φβ(α))β2Σl For a sequence x of any length, the map is then ex- tended as Gap X Gap φ(g;l)(x) = (φ(g;l)(α)) g−mers α in x The gappy kernel is now the inner product in feature space defined by: Gap 0 Gap Gap 0 k(g;l)(x; x ) =< Φ(g;l)(x); Φ(g;l)(x ) > Karsten Borgwardt: Data Mining in Bioinformatics, Page 19 Gappy Kernel Principle Gappy kernel: Count common l-subsequences of g- mers Karsten Borgwardt: Data Mining in Bioinformatics, Page 20 Substitution Kernel General idea mismatch neighborhood ! substitution neighborhood An l-mer then contributes to all l-mers in its substitution neighborhood l l X M(l,σ)(α) = fβ = b1b2 : : : bl 2 Σ : − log P (aijbi) < σg i For a sequence x of any length, the map is then ex- tended as Sub X Sub φ(l,σ)(x) = (φ(l,σ)(α)) l−mers α in x The substitution kernel is now: Sub 0 Sub Sub 0 k(l,σ)(x; x ) =< Φ(l,σ)(x); Φ(l,σ)(x ) > Karsten Borgwardt: Data Mining in Bioinformatics, Page 21 Substitution Kernel Principle Substitution kernel: Count common l-subsequences in substitution neighborhood Karsten Borgwardt: Data Mining in Bioinformatics, Page 22 Wildcard Kernels General idea augment alphabet Σ by a wildcard character ∗ ! Σ[{∗} given α from Σl and β from fΣ [ {∗}}l with maximum m occurrences of ∗ l-mer α contributes to l-mer β if their non-wildcard char- acters match For a sequence x of any length, the map is then given by W ildcard X φ(l,m,λ) (x) = (φβ(α))β2W l−mers α in x j where φβ(α) = λ if α matches pattern β containing j wildcards, φβ(α) = 0 if α does not match β, and 0 ≤ λ ≤ 1. Karsten Borgwardt: Data Mining in Bioinformatics, Page 23 Wildcard Kernel Principle Wildcard kernel: Count l-mers that match except for wildcards Karsten Borgwardt: Data Mining in Bioinformatics, Page 24 References and further reading References [1] C. Leslie, E. Eskin, and W. S. Noble. The spectrum kernel: A string kernel for SVM protein classification. In PSB, pages 564–575, 2002. [2] C. Leslie, E. Eskin, J. Weston, and W. S. Noble. Mis- match string kernels for SVM protein classification. In NIPS 2002. MIT Press. [3] C. Leslie and R. Kuang. Fast kernels for inexact string matching. In COLT, 2003. [4] B. Schölkopf, K. Tsuda, and J.-P. Vert. Kernel Methods in Computational Biology, Chapter 3 and 4. MIT Press, Cambridge, MA, 2004. Karsten Borgwardt: Data Mining in Bioinformatics, Page 25.
Recommended publications
  • Proceedings of International Conference on Electrical, Electronics & Computer Science
    Interscience Research Network Interscience Research Network Conference Proceedings - Full Volumes IRNet Conference Proceedings 12-9-2012 Proceedings of International Conference on Electrical, Electronics & Computer Science Prof.Srikanta Patnaik Mentor IRNet India, [email protected] Follow this and additional works at: https://www.interscience.in/conf_proc_volumes Part of the Electrical and Electronics Commons, and the Systems and Communications Commons Recommended Citation Patnaik, Prof.Srikanta Mentor, "Proceedings of International Conference on Electrical, Electronics & Computer Science" (2012). Conference Proceedings - Full Volumes. 61. https://www.interscience.in/conf_proc_volumes/61 This Book is brought to you for free and open access by the IRNet Conference Proceedings at Interscience Research Network. It has been accepted for inclusion in Conference Proceedings - Full Volumes by an authorized administrator of Interscience Research Network. For more information, please contact [email protected]. Proceedings of International Conference on Electrical, Electronics & Computer Science (ICEECS-2012) 9th December, 2012 BHOPAL, India Interscience Research Network (IRNet) Bhubaneswar, India Rate Compatible Punctured Turbo-Coded Hybrid Arq For Ofdm System RATE COMPATIBLE PUNCTURED TURBO-CODED HYBRID ARQ FOR OFDM SYSTEM DIPALI P. DHAMOLE1& ACHALA M. DESHMUKH2 1,2Department of Electronics and Telecommunications, Sinhgad College of Engineering, Pune 411041, Maharashtra Abstract-Now a day’s orthogonal frequency division multiplexing (OFDM) is under intense research for broadband wireless transmission because of its robustness against multipath fading. A major concern in data communication such as OFDM is to control transmission errors caused by the channel noise so that error free data can be delivered to the user. Rate compatible punctured turbo (RCPT) coded hybrid ARQ (RCPT HARQ) has been found to give improved throughput performance in a OFDM system.
    [Show full text]
  • Text Classification Using String Kernels
    Journal of Machine Learning Research 2 (2002) 419-444 Submitted 12/00;Published 2/02 Text Classification using String Kernels Huma Lodhi [email protected] Craig Saunders [email protected] John Shawe-Taylor [email protected] Nello Cristianini [email protected] Chris Watkins [email protected] Department of Computer Science, Royal Holloway, University of London, Egham, Surrey TW20 0EX, UK Editor: Bernhard Sch¨olkopf Abstract We propose a novel approach for categorizing text documents based on the use of a special kernel. The kernel is an inner product in the feature space generated by all subsequences of length k. A subsequence is any ordered sequence of k characters occurring in the text though not necessarily contiguously. The subsequences are weighted by an exponentially decaying factor of their full length in the text, hence emphasising those occurrences that are close to contiguous. A direct computation of this feature vector would involve a prohibitive amount of computation even for modest values of k, since the dimension of the feature space grows exponentially with k. The paper describes howdespite this fact the inner product can be efficiently evaluated by a dynamic programming technique. Experimental comparisons of the performance of the kernel compared with a stan- dard word feature space kernel (Joachims, 1998) show positive results on modestly sized datasets. The case of contiguous subsequences is also considered for comparison with the subsequences kernel with different decay factors. For larger documents and datasets the paper introduces an approximation technique that is shown to deliver good approximations efficiently for large datasets.
    [Show full text]
  • Fast String Kernels Using Inexact Matching for Protein Sequences
    JournalofMachineLearningResearch5(2004)1435–1455 Submitted 1/04; Revised 7/04; Published 11/04 Fast String Kernels using Inexact Matching for Protein Sequences Christina Leslie [email protected] Center for Computational Learning Systems Columbia University New York, NY 10115, USA Rui Kuang [email protected] Department of Computer Science Columbia University New York, NY 10027, USA Editor: Kristin Bennett Abstract We describe several families of k-mer based string kernels related to the recently presented mis- match kernel and designed for use with support vector machines (SVMs) for classification of pro- tein sequence data. These new kernels – restricted gappy kernels, substitution kernels, and wildcard kernels – are based on feature spaces indexed by k-length subsequences (“k-mers”) from the string alphabet Σ. However, for all kernels we define here, the kernel value K(x,y) can be computed in O(cK( x + y )) time, where the constant cK depends on the parameters of the kernel but is inde- pendent| | of the| | size Σ of the alphabet. Thus the computation of these kernels is linear in the length of the sequences, like| | the mismatch kernel, but we improve upon the parameter-dependent con- m+1 m stant cK = k Σ of the (k,m)-mismatch kernel. We compute the kernels efficiently using a trie data structure and| | relate our new kernels to the recently described transducer formalism. In protein classification experiments on two benchmark SCOP data sets, we show that our new faster kernels achieve SVM classification performance comparable to the mismatch kernel and the Fisher kernel derived from profile hidden Markov models, and we investigate the dependence of the kernels on parameter choice.
    [Show full text]
  • Efficient Global String Kernel with Random Features: Beyond Counting Substructures
    Efficient Global String Kernel with Random Features: Beyond Counting Substructures Lingfei Wu∗ Ian En-Hsu Yen Siyu Huo IBM Research Carnegie Mellon University IBM Research [email protected] [email protected] [email protected] Liang Zhao Kun Xu Liang Ma George Mason University IBM Research IBM Research [email protected] [email protected] [email protected] Shouling Ji† Charu Aggarwal Zhejiang University IBM Research [email protected] [email protected] ABSTRACT of longer lengths. In addition, we empirically show that RSE scales Analysis of large-scale sequential data has been one of the most linearly with the increase of the number and the length of string. crucial tasks in areas such as bioinformatics, text, and audio mining. Existing string kernels, however, either (i) rely on local features of CCS CONCEPTS short substructures in the string, which hardly capture long dis- • Computing methodologies → Kernel methods. criminative patterns, (ii) sum over too many substructures, such as all possible subsequences, which leads to diagonal dominance KEYWORDS of the kernel matrix, or (iii) rely on non-positive-definite similar- String Kernel, String Embedding, Random Features ity measures derived from the edit distance. Furthermore, while there have been works addressing the computational challenge with ACM Reference Format: respect to the length of string, most of them still experience qua- Lingfei Wu, Ian En-Hsu Yen, Siyu Huo, Liang Zhao, Kun Xu, Liang Ma, Shouling Ji, and Charu Aggarwal. 2019. Efficient Global String Kernel dratic complexity in terms of the number of training samples when with Random Features: Beyond Counting Substructures .
    [Show full text]
  • Proquest Dissertations
    On Email Spam Filtering Using Support Vector Machine Ola Amayri A Thesis in The Concordia Institute for Information Systems Engineering Presented in Partial Fulfillment of the Requirements for the Degree of Master of Applied Science (Information Systems Security) at Concordia University Montreal, Quebec, Canada January 2009 © Ola Amayri, 2009 Library and Archives Biblioth&que et 1*1 Canada Archives Canada Published Heritage Direction du Branch Patrimoine de l'6dition 395 Wellington Street 395, rue Wellington Ottawa ON K1A0N4 Ottawa ON K1A0N4 Canada Canada Your file Votre reference ISBN: 978-0-494-63317-5 Our file Notre r6f6rence ISBN: 978-0-494-63317-5 NOTICE: AVIS: The author has granted a non- L'auteur a accorde une licence non exclusive exclusive license allowing Library and permettant a la Bibliothgque et Archives Archives Canada to reproduce, Canada de reproduire, publier, archiver, publish, archive, preserve, conserve, sauvegarder, conserver, transmettre au public communicate to the public by par telecommunication ou par Nntemet, preter, telecommunication or on the Internet, distribuer et vendre des theses partout dans le loan, distribute and sell theses monde, a des fins commerciales ou autres, sur worldwide, for commercial or non- support microforme, papier, electronique et/ou commercial purposes, in microform, autres formats. paper, electronic and/or any other formats. The author retains copyright L'auteur conserve la propriete du droit d'auteur ownership and moral rights in this et des droits moraux qui protege cette these. Ni thesis. Neither the thesis nor la these ni des extraits substantiels de celle-ci substantial extracts from it may be ne doivent etre imprimes ou autrement printed or otherwise reproduced reproduits sans son autorisation.
    [Show full text]
  • Text Classification Using String Kernels
    Journal of Machine Learning Research 2 (2002) 419-444 Submitted 12/00;Published 2/02 Text Classification using String Kernels Huma Lodhi [email protected] Craig Saunders [email protected] John Shawe-Taylor [email protected] Nello Cristianini [email protected] Chris Watkins [email protected] Department of Computer Science, Royal Holloway, University of London, Egham, Surrey TW20 0EX, UK Editor: Bernhard Sch¨olkopf Abstract We propose a novel approach for categorizing text documents based on the use of a special kernel. The kernel is an inner product in the feature space generated by all subsequences of length k. A subsequence is any ordered sequence of k characters occurring in the text though not necessarily contiguously. The subsequences are weighted by an exponentially decaying factor of their full length in the text, hence emphasising those occurrences that are close to contiguous. A direct computation of this feature vector would involve a prohibitive amount of computation even for modest values of k, since the dimension of the feature space grows exponentially with k. The paper describes howdespite this fact the inner product can be efficiently evaluated by a dynamic programming technique. Experimental comparisons of the performance of the kernel compared with a stan- dard word feature space kernel (Joachims, 1998) show positive results on modestly sized datasets. The case of contiguous subsequences is also considered for comparison with the subsequences kernel with different decay factors. For larger documents and datasets the paper introduces an approximation technique that is shown to deliver good approximations efficiently for large datasets.
    [Show full text]