DOCSLIB.ORG

Machine Learning for Genomic Sequence Analysis

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Machine Learning for Genomic Sequence Analysis Machine Learning for Genomic Sequence Analysis - Dissertation - vorgelegt von Dipl. Inform. S¨oren Sonnenburg aus Berlin Von der Fakult¨at IV - Elektrotechnik und Informatik der Technischen Universit¨at Berlin zur Erlangung des akademischen Grades Doktor der Naturwissenschaften — Dr. rer. nat. — genehmigte Dissertation Promotionsaussschuß Vorsitzender: Prof. Dr. Thomas Magedanz • Berichter: Dr. Gunnar R¨atsch • Berichter: Prof. Dr. Klaus-Robert Muller¨ • Tag der wissenschaftlichen Aussprache: 19. Dezember 2008 Berlin 2009 D83 Acknowledgements Above all, I would like to thank Dr. Gunnar R¨atsch and Prof. Dr. Klaus-Robert M¨uller for their guidance and inexhaustible support, without which writing this thesis would not have been possible. All of the work in this thesis has been done at the Fraunhofer Institute FIRST in Berlin and the Friedrich Miescher Laboratory in T¨ubingen. I very much enjoyed the inspiring atmosphere in the IDA group headed by K.-R. M¨uller and in G. R¨atsch’s group in the FML. Much of what we have achieved was only possible in a joint effort. As the various fruitful within-group collaborations expose — we are a team. I would like to thank Fabio De Bona, Lydia Bothe, Vojtech Franc, Sebastian Henschel, Motoaki Kawanabe, Cheng Soon Ong, Petra Philips, Konrad Rieck, Reiner Schulz, Gabriele Schweikert, Christin Sch¨afer, Christian Widmer and Alexander Zien for reading the draft, helpful discussions and moral support. I acknowledge the support from all members of the IDA group at TU Berlin and Fraunhofer FIRST and the members of the Machine Learning in Computational Biology group at the Friedrich Miescher Laboratory, especially for the tolerance of letting me run the large number of compute-jobs that were required to perform the experiments. Finally, I would like to thank the system administrators Roger Holst, Andre Noll, An- dreas Schulz, Rolf Schulz and Sebastian Stark — your support is very much appreciated. This work was supported in part by the IST Programme of the European Community, under the PASCAL Network of Excellence (IST-2002-506778), and by the Learning and Inference Platform of the Max Planck and Fraunhofer Societies. To my parents. IV Preface When I came into first contact with Machine Learning at a seminar on Support Vector Machines (SVMs) held by Klaus-Robert M¨uller and Bernhard Sch¨olkopf in 1999, I became utterly excited about this subject. During that seminar, I gave an introductory talk on SVMs to an all-knowing audience, but it took a while before I seriously picked up the SVM subject in 2002. For my student research project, Gunnar R¨atsch suggested Hidden Markov Models (HMMs) and their application to Bioinformatics. The idea was to find genes, but as this problem is too complex to start with, I started with the sub- problem of recognising splice sites. HMMs using a certain predefined topology performed quite well. However, we managed to improve on this using string kernels (derived from HMMs) and Support Vector Machines (Sonnenburg et al., 2002, Tsuda et al., 2002b). Unfortunately, training SVMs with these string kernels was computationally very costly. Even though we had a sample of more than 100,000 data points, we could barely afford training on 10,000 instances on a, at that time, cutting-edge Compaq Alpha compute cluster. In contrast to HMMs, the resulting trained SVM classifiers were not easily accessible. After a research break of about 2 years during which I became the group’s system administrator, I started work on exactly these — still valid — topics. All of this research was application driven: improving the splicing signal (and later, other signals) detection in order to construct a gene finder that is applicable on a genomic scale. To this end, I worked on computationally efficient string kernels suitable for detecting biological signals such as the aforementioned splicing signal and on large scale algo- rithms for SVMs with string kernels. The results of these efforts are presented in this doctoral thesis in Chapter 2 and 3. Trying to improve the classification performance of a string kernel SVM using the so-called weighted degree kernel, I investigated multiple kernel learning, which turned out to be useful to understand the learnt SVM classifier. Having learnt lessons about the implied string kernel feature spaces (by working on large scale learning) and since there was still room for improvements in understanding SVMs, I continued to research this subject. I consider the resulting Positional Oligomer Importance Matrices (POIMs) a big step forward toward a full understanding of string kernel based classifiers. Both concepts are described in Chapter 4. Since my research was application motivated, all algorithms were implemented in the very versatile SHOGUN toolbox that I developed in parallel (cf. Appendix C). Equipped with this tool-set, several signal detection problems, like splice site recognition and transcription start site recognition, were tackled. The work on large scale learning with string kernels enabled us to train string kernel SVMs on 10,000,000 instances, obtaining state-of-the-art accuracy, and to apply the trained SVMs to billions of instances. At the same time, I used POIMs to understand the complex SVM decision rule. These improved signal detectors were finally integrated to accurately predict gene structures (R¨atsch et al., 2007), as outlined in Chapter 6 and are used in a full gene finding system (Schweikert et al., 2008) outperforming state-of-the-art methods. V VI Zusammenfassung Die Entwicklung neuer Sequenziertechnologien ebnete den Weg fur¨ kosteneffiziente Ge- nomsequenzierung. Allein im Jahr 2008 werden etwa 250 neue Genome sequenziert. Es ist offensichtlich, dass diese gewaltigen Mengen an Daten effektive und genaue computer- gestutzte¨ Methoden zur Sequenzanalyse erfordern. Diese werden ben¨otigt, um eines der wichtigsten Probleme der Bioinformatik zu l¨osen: die akkurate Lokalisation von Genen auf der DNA. In dieser Arbeit werden auf Basis von Support Vector Machines (SVMs) genaueste genomische Signalerkenner entwickelt, die in Gensuchmaschinen verwendet werden k¨onnen. Die Arbeit untergliedert sich in folgende Themenschwerpunkte: String-Kerne Es wurden String-Kerne zur Detektion von Signalen auf dem Genom entwickelt und erweitert. Die Kerne haben eine in der L¨ange der Eingabesequenzen nur lineare Berechnungskomplexit¨at und sind fur¨ eine Vielzahl von Problemen verwendbar. Dadurch gestaltet sich die Sequenzanalyse sehr effektiv: Mit nur geringem Vorwissen ist es m¨oglich, geeignete String-Kernkombinationen auszuw¨ahlen, die hohe Erkennungsra- ten erm¨oglichen. Large-Scale-Lernen Das Training von SVMs war bisher zu rechenintensiv, um auf Da- ten genomischer Gr¨oße angewendet zu werden. Mithilfe der in dieser Arbeit entwickelter Large-Scale-Lernmethoden ist es nun in kurzer Zeit m¨oglich, string-kern-basierte SVMs auf bis zu zehn Millionen Sequenzen zu trainieren und auf uber¨ sechs Milliarden Sequenz- positionen vorherzusagen. Der entwickelte linadd-Ansatz beschleunigt die Berechnung von Linearkombinationen von String-Kernen, die bereits in linearer Zeit berechenbar sind. Dieser Ansatz erm¨oglicht den Verzicht auf einen Kern-Cache beim SVM-Training und fuhrt¨ somit zu einer drastischen Reduktion des Speicheraufwands. Interpretierbarkeit Ein h¨aufig kritisierter Nachteil von SVMs mit komplexen Kernen ist, dass ihre Entscheidungsregeln fur¨ den Menschen schwer zu verstehen sind. In dieser Arbeit wird die Black Box“ der SVM-Klassifikatoren ge¨offnet“, indem zwei Konzepte ” ” entwickeln werden, die zu ihrem Verst¨andnis beitragen: Multiple Kernel Learning (MKL) und Positional Oligomer Importance Matrices (POIMs). MKL erm¨oglicht die Interpre- tation von SVMs mit beliebigen Kernen und kann dazu verwendet werden heterogene Datenquellen zu fusionieren. POIMs sind besonders gut fur¨ String-Kerne geeignet, um die diskriminativsten Sequenzmotive zu bestimmen. Genom-Sequenzanalyse In dieser Arbeit werden SVMs mit neuentwickelten, sehr pr¨azisen String-Kernen zur Detektion einer Vielzahl von Genom-Signalen, zum Bei- spiel der Transkriptionsstart- oder Spleißstellen, angewendet. Dabei werden unerreichte Genauigkeiten erzielt. Unter Verwendung von POIMs wurden die trainierten Klassifika- toren analysiert und unter anderem viele bereits bekannte Sequenzmuster gefunden. Die verbesserten Signaldetektoren wurden dann zur genauen Genstrukturvorhersage ver- wendet. Die Doktorarbeit schließt mit einem Ausblick, wie in dieser Arbeit entwickelte Methoden die Basis fur¨ eine vollst¨andige Gensuchmaschine legen. Keywords: Support Vector Machine, Interpretierbarkeit, String-Kerne, Large-Scale- Lernen, Multiple Kernel Learning, Positional Oligomer Importance Matrices, Spleiß- stellenerkennung, Transkriptionsstartstellenerkennung, Gensuche VII VIII Summary With the development of novel sequencing technologies, the way has been paved for cost efficient, high-throughput whole genome sequencing. In the year 2008 alone, about 250 genomes will have been sequenced. It is self-evident that the handling of this wealth of data requires efficient and accurate computational methods for sequence analysis. They are needed to tackle one of the most important problems in computational biology: the localisation of genes on DNA. In this thesis, I describe the development of state-of-the- art genomic signal detectors based on Support Vector Machines (SVM) that can be used in gene finding systems. The main contributions of this work can be summarized as follows: String Kernels We have developed and extended
Load more

Recommended publications
  • The Shogun Machine Learning Toolbox
    The Shogun Machine Learning Toolbox Heiko Strathmann, Gatsby Unit, UCL London Open Machine Learning Workshop, MSR, NY August 22, 2014 A bit about Shogun I Open-Source tools for ML problems I Started 1999 by SÖren Sonnenburg & GUNnar Rätsch, made public in 2004 I Currently 8 core-developers + 20 regular contributors I Purely open-source community driven I In Google Summer of Code since 2010 (29 projects!) Ohloh - Summary Ohloh - Code Supervised Learning Given: x y n , want: y ∗ x ∗ I f( i ; i )gi=1 j I Classication: y discrete I Support Vector Machine I Gaussian Processes I Logistic Regression I Decision Trees I Nearest Neighbours I Naive Bayes I Regression: y continuous I Gaussian Processes I Support Vector Regression I (Kernel) Ridge Regression I (Group) LASSO Unsupervised Learning Given: x n , want notion of p x I f i gi=1 ( ) I Clustering: I K-Means I (Gaussian) Mixture Models I Hierarchical clustering I Latent Models I (K) PCA I Latent Discriminant Analysis I Independent Component Analysis I Dimension reduction I (K) Locally Linear Embeddings I Many more... And many more I Multiple Kernel Learning I Structured Output I Metric Learning I Variational Inference I Kernel hypothesis testing I Deep Learning (whooo!) I ... I Bindings to: LibLinear, VowpalWabbit, etc.. http://www.shogun-toolbox.org/page/documentation/ notebook Some Large-Scale Applications I Splice Site prediction: 50m examples of 200m dimensions I Face recognition: 20k examples of 750k dimensions ML in Practice I Modular data represetation I Dense, Sparse, Strings, Streams, ... I Multiple types: 8-128 bit word size I Preprocessing tools I Evaluation I Cross-Validation I Accuracy, ROC, MSE, ..
    [Show full text]
  • Python Libraries, Development Frameworks and Algorithms for Machine Learning Applications
    Published by : International Journal of Engineering Research & Technology (IJERT) http://www.ijert.org ISSN: 2278-0181 Vol. 7 Issue 04, April-2018 Python Libraries, Development Frameworks and Algorithms for Machine Learning Applications Dr. V. Hanuman Kumar, Sr. Assoc. Prof., NHCE, Bangalore-560 103 Abstract- Nowadays Machine Learning (ML) used in all sorts numerical values and is of the form y=A0+A1*x, for input of fields like Healthcare, Retail, Travel, Finance, Social Media, value x and A0, A1 are the coefficients used in the etc., ML system is used to learn from input data to construct a representation with the data that we have. The classification suitable model by continuously estimating, optimizing and model helps to identify the sentiment of a particular post. tuning parameters of the model. To attain the stated, For instance a user review can be classified as positive or python programming language is one of the most flexible languages and it does contain special libraries for negative based on the words used in the comments. Emails ML applications namely SciPy, NumPy, etc., which great for can be classified as spam or not, based on these models. The linear algebra and getting to know kernel methods of machine clustering model helps when we are trying to find similar learning. The python programming language is great to use objects. For instance If you are interested in read articles when working with ML algorithms and has easy syntax about chess or basketball, etc.., this model will search for the relatively. When taking the deep-dive into ML, choosing a document with certain high priority words and suggest framework can be daunting.
    [Show full text]
  • Tapkee: an Efficient Dimension Reduction Library
    JournalofMachineLearningResearch14(2013)2355-2359 Submitted 2/12; Revised 5/13; Published 8/13 Tapkee: An Efficient Dimension Reduction Library Sergey Lisitsyn [email protected] Samara State Aerospace University 34 Moskovskoye shosse 443086 Samara, Russia Christian Widmer∗ [email protected] Computational Biology Center Memorial Sloan-Kettering Cancer Center 415 E 68th street New York City 10065 NY, USA Fernando J. Iglesias Garcia [email protected] KTH Royal Institute of Technology 79 Valhallavagen¨ 10044 Stockholm, Sweden Editor: Cheng Soon Ong Abstract We present Tapkee, a C++ template library that provides efficient implementations of more than 20 widely used dimensionality reduction techniques ranging from Locally Linear Embedding (Roweis and Saul, 2000) and Isomap (de Silva and Tenenbaum, 2002) to the recently introduced Barnes- Hut-SNE (van der Maaten, 2013). Our library was designed with a focus on performance and flexibility. For performance, we combine efficient multi-core algorithms, modern data structures and state-of-the-art low-level libraries. To achieve flexibility, we designed a clean interface for ap- plying methods to user data and provide a callback API that facilitates integration with the library. The library is freely available as open-source software and is distributed under the permissive BSD 3-clause license. We encourage the integration of Tapkee into other open-source toolboxes and libraries. For example, Tapkee has been integrated into the codebase of the Shogun toolbox (Son- nenburg et al., 2010), giving us access to a rich set of kernels, distance measures and bindings to common programming languages including Python, Octave, Matlab, R, Java, C#, Ruby, Perl and Lua.
    [Show full text]
  • ML Cheatsheet Documentation
    ML Cheatsheet Documentation Team Sep 02, 2021 Basics 1 Linear Regression 3 2 Gradient Descent 21 3 Logistic Regression 25 4 Glossary 39 5 Calculus 45 6 Linear Algebra 57 7 Probability 67 8 Statistics 69 9 Notation 71 10 Concepts 75 11 Forwardpropagation 81 12 Backpropagation 91 13 Activation Functions 97 14 Layers 105 15 Loss Functions 117 16 Optimizers 121 17 Regularization 127 18 Architectures 137 19 Classification Algorithms 151 20 Clustering Algorithms 157 i 21 Regression Algorithms 159 22 Reinforcement Learning 161 23 Datasets 165 24 Libraries 181 25 Papers 211 26 Other Content 217 27 Contribute 223 ii ML Cheatsheet Documentation Brief visual explanations of machine learning concepts with diagrams, code examples and links to resources for learning more. Warning: This document is under early stage development. If you find errors, please raise an issue or contribute a better definition! Basics 1 ML Cheatsheet Documentation 2 Basics CHAPTER 1 Linear Regression • Introduction • Simple regression – Making predictions – Cost function – Gradient descent – Training – Model evaluation – Summary • Multivariable regression – Growing complexity – Normalization – Making predictions – Initialize weights – Cost function – Gradient descent – Simplifying with matrices – Bias term – Model evaluation 3 ML Cheatsheet Documentation 1.1 Introduction Linear Regression is a supervised machine learning algorithm where the predicted output is continuous and has a constant slope. It’s used to predict values within a continuous range, (e.g. sales, price) rather than trying to classify them into categories (e.g. cat, dog). There are two main types: Simple regression Simple linear regression uses traditional slope-intercept form, where m and b are the variables our algorithm will try to “learn” to produce the most accurate predictions.
    [Show full text]
  • GNU/Linux AI & Alife HOWTO
    GNU/Linux AI & Alife HOWTO GNU/Linux AI & Alife HOWTO Table of Contents GNU/Linux AI & Alife HOWTO......................................................................................................................1 by John Eikenberry..................................................................................................................................1 1. Introduction..........................................................................................................................................1 2. Traditional Artificial Intelligence........................................................................................................1 3. Connectionism.....................................................................................................................................1 4. Evolutionary Computing......................................................................................................................1 5. Alife & Complex Systems...................................................................................................................1 6. Agents & Robotics...............................................................................................................................1 7. Programming languages.......................................................................................................................2 8. Missing & Dead...................................................................................................................................2 1. Introduction.........................................................................................................................................2
    [Show full text]
  • Arxiv:1702.01460V5 [Cs.LG] 10 Dec 2018 1
    Journal of Machine Learning Research 1 (2016) 1-15 Submitted 8/93;; Published 9/93 scikit-multilearn: A scikit-based Python environment for performing multi-label classification Piotr Szyma´nski [email protected],illimites.edu.plg Department of Computational Intelligence Wroc law University of Science and Technology Wroc law, Poland illimites foundation Wroc law, Poland Tomasz Kajdanowicz [email protected] Department of Computational Intelligence Wroc law University of Science and Technology Wroc law, Poland Editor: Leslie Pack Kaelbling Abstract scikit-multilearn is a Python library for performing multi-label classification. The library is compatible with the scikit/scipy ecosystem and uses sparse matrices for all internal op- erations. It provides native Python implementations of popular multi-label classification methods alongside a novel framework for label space partitioning and division. It includes modern algorithm adaptation methods, network-based label space division approaches, which extracts label dependency information and multi-label embedding classifiers. It pro- vides python wrapped access to the extensive multi-label method stack from Java libraries and makes it possible to extend deep learning single-label methods for multi-label tasks. The library allows multi-label stratification and data set management. The implementa- tion is more efficient in problem transformation than other established libraries, has good test coverage and follows PEP8. Source code and documentation can be downloaded from http://scikit.ml and also via pip. The library follows BSD licensing scheme. Keywords: Python, multi-label classification, label-space clustering, multi-label embed- ding, multi-label stratification arXiv:1702.01460v5 [cs.LG] 10 Dec 2018 1.
    [Show full text]
  • The SHOGUN Machine Learning Toolbox
    Introduction Machine Learning The SHOGUN Machine Learning Toolbox The SHOGUN Machine Learning Toolbox (and its python interface) S¨orenSonnenburg1;2, Gunnar R¨atsch2,Sebastian Henschel2,Christian Widmer2,Jonas Behr2,Alexander Zien2,Fabio de Bona2,Alexander Binder1,Christian Gehl1, and Vojtech Franc3 1 Berlin Institute of Technology, Germany 2 Friedrich Miescher Laboratory, Max Planck Society, Germany 3 Center for Machine Perception, Czech Republic fml Introduction Machine Learning The SHOGUN Machine Learning Toolbox Outline 1 Introduction 2 Machine Learning 3 The SHOGUN Machine Learning Toolbox Introduction Machine Learning The SHOGUN Machine Learning Toolbox About me Who Am I? About Me 1997-2002 studied Computer Science 2002-2009 doing ML & Bioinformatics research at Fraunhofer Institute FIRST and Max Planck Society since 2002 2008 PhD: Machine Learning for Genomic Sequence Analysis 2009- Researcher at Berlin Institute of Technology Open Source Involvement Debian Developer http://www.debian.org Machine Learning OSS http://mloss.org Machine Learning Data http://mldata.org Main author of SHOGUN - this talk More about me http://sonnenburgs.de/soeren Introduction Machine Learning The SHOGUN Machine Learning Toolbox Machine Learning - Learning from Data What is Machine Learning and what can it do for you? What is ML? AIM: Learning from empirical data! Applications speech and handwriting recognition search engines, natural language processing medical diagnosis, bioinformatics, chemoinformatics detecting credit card fraud computer vision, object
    [Show full text]
  • Learning Kernels -Tutorial Part IV: Software Solutions
    Learning Kernels -Tutorial Part IV: Software Solutions Corinna Cortes Mehryar Mohri Afshin Rostami Google Research Courant Institute & UC Berkeley [email protected] Google Research arostami@eecs. [email protected] berkeley.edu This Part Software Solutions Demo Corinna Cortes, Mehryar Mohri, Afshin Rostami - ICML 2011 Tutorial. page 2 Open-Source Software Provide easy-to-use and efficient implementations of useful learning kernel algorithms. Allows end-users to combine standard as well as specialized domain-specific kernels. Allow researchers to easily compare against established learning kernel algorithms. Allow developers to makes use of and extend existing algorithms. Corinna Cortes, Mehryar Mohri, Afshin Rostami - ICML 2011 Tutorial. page 3 Open-Source Software Libraries and single algorithms - a starting point: • SHOGUN http://www.shogun-toolbox.org • OpenKernel.org http://www.openkernel.org • DOGMA (online alg: UFO) http://dogma.sourceforge.net/index.html • MKL-SMO, HKL http://www.di.ens.fr/~fbach/index.htm#software • SMO q-norm, GMKL http://research.microsoft.com/ ̃manik/ • SimpleMKL http://asi.insa-rouen.fr/enseignants/~arakotom/code/mklindex.html • Mixed-Norm http://www.cse.iitb.ac.in/saketh/research.html • DC-program http://www.cs.ucl.ac.uk/staff/A.Argyriou/code/dc/ • LP-B, LP-β http://www.vision.ee.ethz.ch/~pgehler/ • MC-MKL http://http://www.fml.tuebingen.mpg.de/raetsch/ Corinna Cortes, Mehryar Mohri, Afshin Rostami - ICML 2011 Tutorial. page 4 SHOGUN www.shogun-toolbox.org S. Sonnenburg, G. Raetsch, S. Henschel Large scale kernel methods, focusing on SVM. MATLAB, R, Octave and Python interfaces. Choice of LibSVM, Liblinear, SVMLight for internal solver. Standard kernels (e.g.
    [Show full text]
  • The Shogun Machine Learning Toolbox
    The Shogun Machine Learning Toolbox [email protected] Europython 2014 July 24, 2014 Outline Overview Machine Learning Features Technical Features Community A bit about me I Heiko Strathmann - http://herrstrathmann.de/ I 2004-2008 - Jazz guitar I 2008-2013 - BSc in CS, MSc in ML I Phd in Neuroscience & Machine Learning, UCL I Kernels, Bayesian & MCMC, Bioinformatics, Open-Source I Involved in Shogun since 2010 A bit about Machine Learning I Science of patterns in information I For example for recognising I Frauds in aeroplane wings I Skin cancer I Abuse of bank accounts I ...or predicting I Treatment resistance of HIV I Brain activity I Products on Amazon/Spotify/Netix I Not limited to, but including I Statistics I Big Data & <place-buzzword> I Robot world domination A bit about Shogun I Open-Source tools for ML problems I Made public in 2004 I Currently 8 core-developers + 20 regular contributors I Originally academic background I In Google Summer of Code since 2010 (29 projects!) I Upcoming workshop: July 27, 28, c-base Ohloh - Summary Ohloh - Code Ohloh - Commits Outline Overview Machine Learning Features Technical Features Community Supervised Learning Given: x y n , want: y ∗ x ∗ I f( i ; i )gi=1 j I Classication: y discrete I Support Vector Machine I Gaussian Processes I Logistic Regression I Decision Trees I Nearest Neighbours I Naive Bayes I Regression: y continuous I Gaussian Processes I Support Vector Regression I (Kernel) Ridge Regression I (Group) LASSO Unsupervised Learning Given: x n , want notion of p x I f i gi=1 ( ) I Clustering: I K-Means I (Gaussian) Mixture Models I Hierarchical clustering I Latent Models I (K) PCA I Latent Discriminant Analysis I Independent Component Analysis I Dimension reduction I (K) Locally Linear Embeddings I Many more..
    [Show full text]
  • The SHOGUN Machine Learning Toolbox (And Its R Interface)
    Introduction Features Code Example Summary The SHOGUN Machine Learning Toolbox (and its R interface) S¨orenSonnenburg1;2, Gunnar R¨atsch2,Sebastian Henschel2, Christian Widmer2,Jonas Behr2,Alexander Zien2,Fabio De Bona2,Alexander Binder1,Christian Gehl1, and Vojtech Franc3 1 Berlin Institute of Technology, Germany 2 Friedrich Miescher Laboratory, Max Planck Society, Germany 3 Center for Machine Perception, Czech Republic fml Introduction Features Code Example Summary Outline 1 Introduction 2 Features 3 Code Example 4 Summary fml Introduction Features Code Example Summary Introduction What can you do with the SHOGUN Machine Learning Toolbox [6]? Types of problems: Clustering (no labels) Classification (binary labels) Regression (real valued labels) Structured Output Learning (structured labels) Main focus is on Support Vector Machines (SVMs) Also implements a number of other ML methods like Hidden Markov Models (HMMs) Linear Discriminant Analysis (LDA) Kernel Perceptrons fml Introduction Features Code Example Summary Support Vector Machine Given: Points xi 2 X (i = 1;:::; N) with labels yi 2 {−1; +1g Task: Find hyperplane that maximizes margin Decision function f (x) = w · x + b fml Introduction Features Code Example Summary SVM with Kernels SVM decision function in kernel feature space: N X f (x) = yi αi Φ(x) · Φ(xi ) + b (1) i=1 | {z } =k(x;xi ) Training: Find parameters α Corresponds to solving quadratic optimization problem (QP) fml Introduction Features Code Example Summary Large-Scale SVM Implementations Different SVM solvers employ
    [Show full text]
  • ML-Flex: a Flexible Toolbox for Performing Classification Analyses
    JournalofMachineLearningResearch13(2012)555-559 Submitted 6/11; Revised 2/12; Published 3/12 ML-Flex: A Flexible Toolbox for Performing Classification Analyses In Parallel Stephen R. Piccolo [email protected] Department of Pharmacology and Toxicology, School of Pharmacy University of Utah Salt Lake City, UT 84112, USA Lewis J. Frey [email protected] Huntsman Cancer Institute Department of Biomedical Informatics, School of Medicine University of Utah Salt Lake City, UT 84112, USA Editor: Geoff Holmes Abstract Motivated by a need to classify high-dimensional, heterogeneous data from the bioinformatics do- main, we developed ML-Flex, a machine-learning toolbox that enables users to perform two-class and multi-class classification analyses in a systematic yet flexible manner. ML-Flex was writ- ten in Java but is capable of interfacing with third-party packages written in other programming languages. It can handle multiple input-data formats and supports a variety of customizations. ML- Flex provides implementations of various validation strategies, which can be executed in parallel across multiple computing cores, processors, and nodes. Additionally, ML-Flex supports aggre- gating evidence across multiple algorithms and data sets via ensemble learning. This open-source software package is freely available from http://mlflex.sourceforge.net. Keywords: toolbox, classification, parallel, ensemble, reproducible research 1. Introduction The machine-learning community has developed a wide array of classification algorithms, but they are implemented in diverse programming languages, have heterogeneous interfaces, and require disparate file formats. Also, because input data come in assorted formats, custom transformations often must precede classification. To address these challenges, we developed ML-Flex, a general- purpose toolbox for performing two-class and multi-class classification analyses.
    [Show full text]
  • Variability of the Neuroimaging Results Across OS, and How to Avoid It
    Center for Open Neuroscience cogito ergo sum C Variability of the Neuroimaging results across OS, and how to avoid it Yaroslav O. Halchenko Dartmouth College, Hanover, NH, USA May 2018 http://datalad.org http://www.pymvpa.org Houston, we’ve got a problem... Houston, we’ve got a problem... Houston, we’ve got a problem... Gronenschild et al. (2012) Houston, we’ve got a problem... Glatard et al. (2015) How neuroimaging study should be implemented to make it verifiably re-executable? Simple re-executable neuroimaging publication Ghosh et al. (2017) Simple re-executable neuroimaging publication Goal Investigate and document procedures to generate a verifiable re-executable publication: obtain the same result using the same software on the same data make it easy to check if results hold when computational environment changes Approach make entire analysis fully scripted and full computing environment unambiguously specified complement classical publication with all necessary code, data and documentation for anyone to (re)run the analysis (on same or different data/environment) Simple re-executable: Data input dataset created by querying NITRC Image Repository (NITRC-IR; nitrc.org/ir) for MRI images subjects age = 10–15 years magnet field strength = 3 24 subjects from 1000 Functional Connectomes project (Biswal et al., 2010) age=13.5 +/- 1.4 years; 16 males, 8 females; 8 right handed, 1 left and 15 unknown data provided into analysis as a simple list of URLs Simple re-executable: Analysis software FSL fsl.fmrib.ox.ac.uk/fsl/fslwiki brain segmentation
    [Show full text]