Data Mining in Bioinformatics Day 9: String & Text Mining in Bioinformatics Karsten Borgwardt
February 21 to March 4, 2011
Machine Learning & Computational Biology Research Group MPIs Tübingen
Karsten Borgwardt: Data Mining in Bioinformatics, Page 1 Why compare sequences?
Protein sequences Proteins are chains of amino acids. 20 different types of amino acids can be found in protein sequences. Protein sequence changes over time by mutations, dele- tion, insertions. Different protein sequences may diverge from one com- mon ancestor. Their sequences may differ slightly, yet their function is often conserved.
Karsten Borgwardt: Data Mining in Bioinformatics, Page 2 Why compare sequences?
Biological Question: Biologists are interested in the reverse direction: Given two protein sequences, is it likely that they origi- nate from the same common ancestor? Computational Challenge: How to measure similarity between two protein se- quence, or equivalently: How to measure similarity between two strings Kernel Challenge: How to measure similarity between two strings via a ker- nel function In short: How to define a string kernel
Karsten Borgwardt: Data Mining in Bioinformatics, Page 3 History of sequence comparison
First phase Smith-Waterman BLAST Second phase Profiles Hidden Markov Models Third phase PSI-Blast SAM-T98 Fourth phase Kernels
Karsten Borgwardt: Data Mining in Bioinformatics, Page 4 Sequence comparison: Phase 1
Idea Measure pairwise similarities between sequences with gaps Methods Smith-Waterman dynamic programming high accuracy slow (O(n2)) BLAST faster heuristic alternative with sufficient accuracy searches common substrings of fixed length extends these in both directions performs gapped alignment
Karsten Borgwardt: Data Mining in Bioinformatics, Page 5 Sequence comparison: Phase 2
Idea Collect aggregate statistics from a family of sequences Compare this statistics to a single unlabeled protein Methods Hidden Markov Models (HMMs) Markov process with hidden and observable parame- ters Forward algorithm determines probability if given se- quence is output of particular HMM Profiles Profiles of sequence families are derived by multiple sequence alignment Given sequence is compared to this profile
Karsten Borgwardt: Data Mining in Bioinformatics, Page 6 Sequence comparison: Phase 3
Idea Create single models from database collections of homologous sequences Methods PSI-BLAST Position specific iterative BLAST Profile from highest scoring hits in initial BLAST runs Position weighting according to degree of conserva- tion Iteration of these steps SAM-T98, now SAM-T02 database search with HMM from multiple sequence alignment
Karsten Borgwardt: Data Mining in Bioinformatics, Page 7 Phase 4: Kernels and SVMs
General idea Model differences between classes of sequences Use SVM classifier to distinguish classes Use kernel to measure similarity between strings Kernels for Protein Sequences SVM-Fisher kernel Composite kernel Motif kernel String kernel
Karsten Borgwardt: Data Mining in Bioinformatics, Page 8 SVM-Fisher method
General idea Combine HMMs and SVMs for sequence classification Won best-paper award at ISMB 1999 Sequence representation fixed-length vector components are transition and emission probabilities transformation into Fisher score
Karsten Borgwardt: Data Mining in Bioinformatics, Page 9 SVM-Fisher method
Algorithm Model protein family F as HMM Transform query protein X into fixed-length vector via HMM Compute kernel between X and positive and negative examples of the protein family Advantages allows to incorporate prior knowledge allows to deal with missing data is interpretable outperforms competing methods
Karsten Borgwardt: Data Mining in Bioinformatics, Page 10 Composition kernels
General idea Model sequence by amino acid content Bin amino acids w.r.t physico-chemical properties Sequence representation feature vector of amino acid frequencies physico-chemical properties include predicted secondary structure, hydrophobicity, normalized van der Waals volume, polarity, polarizability useful database: AAindex
Karsten Borgwardt: Data Mining in Bioinformatics, Page 11 Motif kernels
General idea Conserved motif in amino acid sequences indicate structural and functional relationship Model sequence s as a feature vector f representing motifs i-th component of f is 1 ⇔ s contains i-th motif Motif databases PROSITE eMOTIFs BLOCKS+ combines several databases Generated by manual construction multiple sequence alignment
Karsten Borgwardt: Data Mining in Bioinformatics, Page 12 Pairwise comparison kernels
General idea Employ empirical kernel map on Smith-Waterman/Blast scores Advantage Utilizes decades of practical experience with Blast Disadvantage High computational cost (O(m3)) Alleviation Employ Blast instead of Smith-Waterman Use vectorization set for empirical map only
Karsten Borgwardt: Data Mining in Bioinformatics, Page 13 Phase 4: String Kernels
General idea Count common substrings in two strings A substring of length k is a k-mer Variations Assign weights to k-mers Allow for mismatches Allow for gaps Include substitutions Include wildcards
Karsten Borgwardt: Data Mining in Bioinformatics, Page 14 Spectrum Kernel
General idea For each l-mer α ∈ Σl, the coordinate indexed by α will be the number of times α occurs in sequence x. Then the l-spectrum feature map is Spectrum Φl (x) = (φα(x))α∈Σl
Here φα(x) is the # occurrences of α in x. The spectrum kernel is now the inner product in the fea- ture space defined by this map: 0 Spectrum Spectrum 0 kSpectrum(x, x ) =< Φl (x), Φl (x ) > Sequences are deemed the more similar, the more com- mon substrings they contain
Karsten Borgwardt: Data Mining in Bioinformatics, Page 15 Spectrum Kernel
Principle Spectrum kernel: Count exactly common k-mers
Karsten Borgwardt: Data Mining in Bioinformatics, Page 16 Mismatch Kernel
General idea Do not enforce strictly exact matches Define mismatch neighborhood of an l-mer α with up to m mismatches: Mismatch φ(l,m) (α) = (φβ(α))β∈Σl For a sequence x of any length, the map is then ex- tended as Mismatch X Mismatch φ(l,m) (x) = (φ(l,m) (α)) l−mers α in x The mismatch kernel is now the inner product in feature space defined by: Mismatch 0 Mismatch Mismatch 0 k(l,m) (x, x ) =< Φ(l,m) (x), Φ(l,m) (x ) >
Karsten Borgwardt: Data Mining in Bioinformatics, Page 17 Mismatch Kernel
Principle Mismatch kernel: Count common k-mers with max. m mismatches
Karsten Borgwardt: Data Mining in Bioinformatics, Page 18 Gappy Kernel
General idea Allow for gaps in common substrings → “subsequences” A g-mer then contributes to all its l-mer subsequences Gap φ(g,l)(α) = (φβ(α))β∈Σl For a sequence x of any length, the map is then ex- tended as Gap X Gap φ(g,l)(x) = (φ(g,l)(α)) g−mers α in x The gappy kernel is now the inner product in feature space defined by: Gap 0 Gap Gap 0 k(g,l)(x, x ) =< Φ(g,l)(x), Φ(g,l)(x ) >
Karsten Borgwardt: Data Mining in Bioinformatics, Page 19 Gappy Kernel
Principle Gappy kernel: Count common l-subsequences of g- mers
Karsten Borgwardt: Data Mining in Bioinformatics, Page 20 Substitution Kernel
General idea mismatch neighborhood → substitution neighborhood An l-mer then contributes to all l-mers in its substitution neighborhood l l X M(l,σ)(α) = {β = b1b2 . . . bl ∈ Σ : − log P (ai|bi) < σ} i For a sequence x of any length, the map is then ex- tended as Sub X Sub φ(l,σ)(x) = (φ(l,σ)(α)) l−mers α in x The substitution kernel is now: Sub 0 Sub Sub 0 k(l,σ)(x, x ) =< Φ(l,σ)(x), Φ(l,σ)(x ) >
Karsten Borgwardt: Data Mining in Bioinformatics, Page 21 Substitution Kernel
Principle Substitution kernel: Count common l-subsequences in substitution neighborhood
Karsten Borgwardt: Data Mining in Bioinformatics, Page 22 Wildcard Kernels
General idea augment alphabet Σ by a wildcard character ∗ → Σ∪{∗} given α from Σl and β from {Σ ∪ {∗}}l with maximum m occurrences of ∗ l-mer α contributes to l-mer β if their non-wildcard char- acters match For a sequence x of any length, the map is then given by W ildcard X φ(l,m,λ) (x) = (φβ(α))β∈W l−mers α in x j where φβ(α) = λ if α matches pattern β containing j wildcards, φβ(α) = 0 if α does not match β, and 0 ≤ λ ≤ 1.
Karsten Borgwardt: Data Mining in Bioinformatics, Page 23 Wildcard Kernel
Principle Wildcard kernel: Count l-mers that match except for wildcards
Karsten Borgwardt: Data Mining in Bioinformatics, Page 24 References and further reading
References
[1] C. Leslie, E. Eskin, and W. S. Noble. The spectrum kernel: A string kernel for SVM protein classification. In PSB, pages 564–575, 2002. [2] C. Leslie, E. Eskin, J. Weston, and W. S. Noble. Mis- match string kernels for SVM protein classification. In NIPS 2002. MIT Press. [3] C. Leslie and R. Kuang. Fast kernels for inexact string matching. In COLT, 2003. [4] B. Schölkopf, K. Tsuda, and J.-P. Vert. Kernel Methods in Computational Biology, Chapter 3 and 4. MIT Press, Cambridge, MA, 2004.
Karsten Borgwardt: Data Mining in Bioinformatics, Page 25