DOCSLIB.ORG

The Khmer Software Package: Enabling Efficient Nucleotide Sequence Analysis [Version 1; Peer Review: 2 Approved, 1 Approved with Reservations]

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
The Khmer Software Package: Enabling Efficient Nucleotide Sequence Analysis [Version 1; Peer Review: 2 Approved, 1 Approved with Reservations] F1000Research 2015, 4:900 Last updated: 23 JUL 2021 SOFTWARE TOOL ARTICLE The khmer software package: enabling efficient nucleotide sequence analysis [version 1; peer review: 2 approved, 1 approved with reservations] Michael R. Crusoe 1, Hussien F. Alameldin2, Sherine Awad3, Elmar Boucher4, Adam Caldwell5, Reed Cartwright6, Amanda Charbonneau7, Bede Constantinides8, Greg Edvenson9, Scott Fay10, Jacob Fenton11, Thomas Fenzl12, Jordan Fish11, Leonor Garcia-Gutierrez13, Phillip Garland14, Jonathan Gluck15, Iván González16, Sarah Guermond17, Jiarong Guo18, Aditi Gupta1, Joshua R. Herr1, Adina Howe 19, Alex Hyer20, Andreas Härpfer21, Luiz Irber11, Rhys Kidd22, David Lin23, Justin Lippi24, Tamer Mansour3,25, Pamela McA'Nulty26, Eric McDonald11, Jessica Mizzi27, Kevin D. Murray28, Joshua R. Nahum29, Kaben Nanlohy30, Alexander Johan Nederbragt31, Humberto Ortiz-Zuazaga32, Jeramia Ory33, Jason Pell11, Charles Pepe-Ranney34, Zachary N. Russ35, Erich Schwarz36, Camille Scott11, Josiah Seaman37, Scott Sievert38, Jared Simpson39,40, Connor T. Skennerton41, James Spencer42, Ramakrishnan Srinivasan43, Daniel Standage44,45, James A. Stapleton46, Susan R. Steinman47, Joe Stein48, Benjamin Taylor11, Will Trimble49, Heather L. Wiencko50, Michael Wright11, Brian Wyss11, Qingpeng Zhang11, en zyme51, C. Titus Brown1,3,11 1Microbiology and Molecular Genetics, Michigan State University, East Lansing, MI, USA 2Department of Plant, Soil and Microbial Sciences, Michigan State University, East Lansing, MI, USA 3Population Health and Reproduction, University of California, Davis, Davis, CA, USA 4Department of Biomedical Engineering, Oregon Health and Science University, Portland, OR, USA 5Biology Department, San Jose State University, San Jose, CA, USA 6School of Life Sciences and The Biodesign Institute, Arizona State University, Tempe, AZ, USA 7Genetics, Michigan State University, East Lansing, MI, USA 8Computational and Evolutionary Biology, Faculty of Life Sciences, University of Manchester, Manchester, UK 9Micron Technology, Seattle, WA, USA 10Invitae, San Francisco, CA, USA 11Computer Science and Engineering, Michigan State University, East Lansing, MI, USA 12Independent Researcher, Munich, Germany 13Mathematics Institute, University of Warwick, Warwick, UK 14Eastlake Data, Seattle, WA, USA 15Graduate Program, University of Maryland, College Park, MD, USA 16Athinoula A. Martinos Center for Biomedical Imaging, Department of Radiology, Massachusetts General Hospital, Charlestown, MA, USA 17Independent Researcher, Seattle, WA, USA 18Center for Microbial Ecology, Michigan State University, East Lansing, MI, USA 19Department of Agricultural and Biosystems Engineering, Iowa State University, Ames, IA, USA 20Department of Biology, University of Utah, Salt Lake City, UT, USA 21 Page 1 of 13 F1000Research 2015, 4:900 Last updated: 23 JUL 2021 ConSol* Software GmbH, Munchen, Germany 22Independent Researcher, Sydney, Australia 23Verdematics, Fremont, CA, USA 24Independent Researcher, San Francisco, CA, USA 25Clinical Pathology, Mansoura University, Mansoura, Egypt 26Addgene, Cambridge, MA, USA 27Biochemistry and Molecular Biology, Michigan State University, East Lansing, MI, USA 28ARC Centre of Excellence in Plant Energy Biology, The Australian National University, Canberra, ACT, Australia 29BEACON Center, Michigan State University, East Lansing, MI, USA 30Independent Researcher, New Orleans, LA, USA 31Centre for Ecological and Evolutionary Synthesis, Dept. of Biosciences, University of Oslo, Oslo, Norway 32Department of Computer Science, Rio Piedras Campus, University of Puerto Rico, San Juan, Puerto Rico 33Biochemistry, St. Louis College of Pharmacy, St. Louis, MO, USA 34Crop and Soil Sciences, Cornell University, Ithaca, NY, USA 35Department of Bioengineering, UC Berkeley, Berkeley, CA, USA 36Department of Molecular Biology and Genetics, Cornell University, Ithaca, NY, USA 37Data Visualization, Newline Technical Innovations, Windsor, CO, USA 38Electrical and Computer Engineering, University of Minnesota, Minneapolis, MN, USA 39Ontario Institute for Cancer Research, Toronto, ON, Canada 40Computer Science, University of Toronto, Toronto, ON, Canada 41Division of Geological and Planetary Sciences, California Institute of Technology, Pasadena, CA, USA 42Dept of Physics and Dept of Materials, Imperial College London, London, UK 43Genetics and Genomic Sciences, Icahn School of Medicine at Mount Sinai, New York, NY, USA 44Department of Biology, Indiana University, Bloomington, IN, USA 45Bioinformatics and Computational Biology Graduate Program, Iowa State University, Ames, IA, USA 46Chemical Engineering & Materials Science, Michigan State University, East Lansing, MIS, USA 47The New York Eye and Ear Infirmary of Mount Sinai, New York, NY, USA 48Independent Researcher, Providence, RI, USA 49Mathematics and Computer Science Division, Argonne National Laboratory, Lemont, IL, USA 50Department of Genetics, Smurfit Institute, Trinity College Dublin, Dublin, Ireland 51Independent Researcher, Boston, MA, USA v1 First published: 25 Sep 2015, 4:900 Open Peer Review https://doi.org/10.12688/f1000research.6924.1 Latest published: 25 Sep 2015, 4:900 https://doi.org/10.12688/f1000research.6924.1 Reviewer Status Invited Reviewers Abstract The khmer package is a freely available software library for working 1 2 3 efficiently with fixed length DNA words, or k-mers. khmer provides implementations of a probabilistic k-mer counting data structure, a version 1 compressible De Bruijn graph representation, De Bruijn graph 25 Sep 2015 report report report partitioning, and digital normalization. khmer is implemented in C++ and Python, and is freely available under the BSD license at 1. Ewan Birney , European Bioinformatics https://github.com/dib-lab/khmer/. Institute, Hinxton, UK Keywords bioinformatics , dna sequencing analysis , k-mer , kmer , khmer , 2. Daniel S. Katz , University of Chicago, online, low-memory , streaming Chicago, USA 3. Rob Patro , Stony Brook University, Stony Brook, USA Page 2 of 13 F1000Research 2015, 4:900 Last updated: 23 JUL 2021 Any reports and responses or comments on the This article is included in the Iowa State article can be found at the end of the article. University collection. Corresponding author: C. Titus Brown ([email protected]) Competing interests: No competing interests were disclosed. Grant information: khmer development has largely been supported by AFRI Competitive Grant no. 2010-65205-20361 from the USDA NIFA, and is now funded by the National Human Genome Research Institute of the National Institutes of Health under Award Number R01HG007513, as well as by the the Gordon and Betty Moore Foundation under Award number GBMF4551, all to CTB. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. Copyright: © 2015 Crusoe MR et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. How to cite this article: Crusoe MR, Alameldin HF, Awad S et al. The khmer software package: enabling efficient nucleotide sequence analysis [version 1; peer review: 2 approved, 1 approved with reservations] F1000Research 2015, 4:900 https://doi.org/10.12688/f1000research.6924.1 First published: 25 Sep 2015, 4:900 https://doi.org/10.12688/f1000research.6924.1 Page 3 of 13 F1000Research 2015, 4:900 Last updated: 23 JUL 2021 Introduction Operation DNA words of a fixed-length k, or “k-mers”, are a common abstrac- khmer is primarily developed on Linux for Python 2.7 and 64-bit tion in DNA sequence analysis that enable alignment-free sequence processors, and several core developers use Mac OS X. The project analysis and comparison. With the advent of second-generation is tested regularly using the Jenkins continuous integration system sequencing and the widespread adoption of De Bruijn graph-based running on Ubuntu 14.04 LTS and Mac OS X 10.10; the current assemblers, k-mers have become even more widely used in recent development branch is also tested under Python 3.3, 3.4, and 3.5. years. However, the dramatically increased rate of sequence data Releases are tested against many Linux distributions, including generation from Illumina sequencers continues to challenge the RedHat Enterprise Linux, Debian, Fedora, and Ubuntu. khmer basic data structures and algorithms for k-mer storage and manip- should work on most UNIX derivatives with little modification. ulation. This has led to the development of a wide range of data Windows is explicitly not supported. structures and algorithms that explore possible improvements to k-mer-based approaches. Memory requirements for using khmer vary with the complexity of data and are user configurable. Several core data structures can Here we present version 2.0 of the khmer software package, a high- trade memory for false positives, and we have explored these details performance library implementing memory- and time-efficient in several papers, most notably Pell et al. 20122 and Zhang et al. algorithms for the manipulation and analysis of short-read data sets. 20141. For example, most single organism mRNAseq data sets can khmer contains reference implementations of several approaches, be processed in under 16 GB of RAM3,8, while memory require- including a probabilistic k-mer counter based on the CountMin ments for metagenome data sets may vary from dozens of gigabytes Sketch1, a compressible
Load more

Recommended publications
  • BIOGRAPHICAL SKETCH NAME: Berger
    BIOGRAPHICAL SKETCH NAME: Berger, Bonnie eRA COMMONS USER NAME (credential, e.g., agency login): BABERGER POSITION TITLE: Simons Professor of Mathematics and Professor of Electrical Engineering and Computer Science EDUCATION/TRAINING (Begin with baccalaureate or other initial professional education, such as nursing, include postdoctoral training and residency training if applicable. Add/delete rows as necessary.) EDUCATION/TRAINING DEGREE Completion (if Date FIELD OF STUDY INSTITUTION AND LOCATION applicable) MM/YYYY Brandeis University, Waltham, MA AB 06/1983 Computer Science Massachusetts Institute of Technology SM 01/1986 Computer Science Massachusetts Institute of Technology Ph.D. 06/1990 Computer Science Massachusetts Institute of Technology Postdoc 06/1992 Applied Mathematics A. Personal Statement Advances in modern biology revolve around automated data collection and sharing of the large resulting datasets. I am considered a pioneer in the area of bringing computer algorithms to the study of biological data, and a founder in this community that I have witnessed grow so profoundly over the last 26 years. I have made major contributions to many areas of computational biology and biomedicine, largely, though not exclusively through algorithmic innovations, as demonstrated by nearly twenty thousand citations to my scientific papers and widely-used software. In recognition of my success, I have just been elected to the National Academy of Sciences and in 2019 received the ISCB Senior Scientist Award, the pinnacle award in computational biology. My research group works on diverse challenges, including Computational Genomics, High-throughput Technology Analysis and Design, Biological Networks, Structural Bioinformatics, Population Genetics and Biomedical Privacy. I spearheaded research on analyzing large and complex biological data sets through topological and machine learning approaches; e.g.
    [Show full text]
  • Lior Pachter Genome Informatics 2013 Keynote
    Stories from the Supplement Lior Pachter! Department of Mathematics and Molecular & Cell Biology! UC Berkeley! ! November 1, 2013! Genome Informatics, CSHL The Cufflinks supplements • C. Trapnell, B.A. Williams, G. Pertea, A. Mortazavi, G. Kwan, M.J. van Baren, S.L. Salzberg, B.J. Wold and L. Pachter, Transcript assembly and quantification by RNA-Seq reveals unannotated transcripts and isoform switching during cell differentiation, Nature Biotechnology 28, (2010), p 511–515. Supplementary Material: 42 pages. • C. Trapnell, D.G. Hendrickson, M. Sauvageau, L. Goff, J.L. Rinn and L. Pachter, Differential analysis of gene regulation at transcript resolution with RNA-seq, Nature Biotechnology, 31 (2012), p 46–53. Supplementary Material: 70 pages. A supplementary arithmetic progression? • C. Trapnell, B.A. Williams, G. Pertea, A. Mortazavi, G. Kwan, M.J. van Baren, S.L. Salzberg, B.J. Wold and L. Pachter, Transcript assembly and quantification by RNA-Seq reveals unannotated transcripts and isoform switching during cell differentiation, Nature Biotechnology 28, (2010), p 511–515. Supplementary Material: 42 pages. • C. Trapnell, D.G. Hendrickson, M. Sauvageau, L. Goff, J.L. Rinn and L. Pachter, Differential analysis of gene regulation at transcript resolution with RNA- seq, Nature Biotechnology, 31 (2012), p 46–53. Supplementary Material: 70 pages. • A. Roberts and L. Pachter, Streaming algorithms for fragment assignment, Nature Methods 10 (2013), p 71—73. Supplementary Material: 98 pages? The nature methods manuscript checklist The nature methods manuscript checklist Emperor Joseph II: My dear young man, don't take it too hard. Your work is ingenious. It's quality work. And there are simply too many notes, that's all.
    [Show full text]
  • Modeling and Analysis of RNA-Seq Data: a Review from a Statistical Perspective
    Modeling and analysis of RNA-seq data: a review from a statistical perspective Wei Vivian Li 1 and Jingyi Jessica Li 1;2;∗ Abstract Background: Since the invention of next-generation RNA sequencing (RNA-seq) technolo- gies, they have become a powerful tool to study the presence and quantity of RNA molecules in biological samples and have revolutionized transcriptomic studies. The analysis of RNA-seq data at four different levels (samples, genes, transcripts, and exons) involve multiple statistical and computational questions, some of which remain challenging up to date. Results: We review RNA-seq analysis tools at the sample, gene, transcript, and exon levels from a statistical perspective. We also highlight the biological and statistical questions of most practical considerations. Conclusion: The development of statistical and computational methods for analyzing RNA- seq data has made significant advances in the past decade. However, methods developed to answer the same biological question often rely on diverse statical models and exhibit dif- ferent performance under different scenarios. This review discusses and compares multiple commonly used statistical models regarding their assumptions, in the hope of helping users select appropriate methods as needed, as well as assisting developers for future method development. 1 Introduction RNA sequencing (RNA-seq) uses the next generation sequencing (NGS) technologies to reveal arXiv:1804.06050v3 [q-bio.GN] 1 May 2018 the presence and quantity of RNA molecules in biological samples. Since its invention, RNA- seq has revolutionized transcriptome analysis in biological research. RNA-seq does not require any prior knowledge on RNA sequences, and its high-throughput manner allows for genome-wide profiling of transcriptome landscapes [1,2].
    [Show full text]
  • Human and Mouse Gene Structure: Comparative Analysis and Application to Exon Prediction
    Downloaded from genome.cshlp.org on October 4, 2021 - Published by Cold Spring Harbor Laboratory Press Letter Human and Mouse Gene Structure: Comparative Analysis and Application to Exon Prediction Serafim Batzoglou,1,4,7 Lior Pachter,2,7 Jill P. Mesirov,3 Bonnie Berger,1,4,6 and Eric S. Lander3,5,6 1Laboratory for Computer Science, Massachusetts Institute of Technology, Cambridge, Massachusetts 02139 USA; 2Department of Mathematics, University of California Berkeley, Berkeley, California 94720 USA; 3Whitehead Institute for Biomedical Research, Cambridge, Massachusetts 02142 USA; 4Department of Mathematics, Massachusetts Institute of Technology, Cambridge, Massachusetts 02139 USA; 5Department of Biology, Massachusetts Institute of Technology, Cambridge, Massachusetts 02139 USA We describe a novel analytical approach to gene recognition based on cross-species comparison. We first undertook a comparison of orthologous genomic loci from human and mouse, studying the extent of similarity in the number, size and sequence of exons and introns. We then developed an approach for recognizing genes within such orthologous regions by first aligning the regions using an iterative global alignment system and then identifying genes based on conservation of exonic features at aligned positions in both species. The alignment and gene recognition are performed by new programs called GLASS and ROSETTA, respectively. ROSETTA performed well at exact identification of coding exons in 117 orthologous pairs tested. A fundamental task in analyzing genomes is to identify by comparison of syntenic human and mouse genomic the genes. This is relatively straightforward for organ- sequences. isms with compact genomes (such as bacteria, yeast, It is well known that cross-species sequence com- flies and worms) because exons tend to be large and the parison can help highlight important functional ele- introns are either non-existent or tend to be short.
    [Show full text]
  • Eric S. Lander
    Chancellor's Distinguished Fellows Program 2004-2005 Selective Bibliography UC Irvine Libraries Eric S. Lander February 25, 2005 Prepared by: John E. Sisson Biological Sciences Librarian [email protected] Journal Articles (Selected from over 220 articles he has authored or co-authored) Poirier, C., Y. J. Qin, C.P. Adams, Y. Anaya, J. B. Singer, A. E. Hill, Eric S. Lander, J.H. Nadeau, and C. E. Bishop. "A complex interaction of imprinted and maternal-effect genes modifies sex determination in odd sex (Ods) mice." Genetics 168, no. 3 (2004): 1557-1562. Skuse, D. H., S. Purcell, M. J. Daly, R. J. Dolan, J. S. Morris, K. Lawrence, Eric S. Lander, and P. Sklar. "What can studies on Turner syndrome tell us about the role of X- linked genes in social cognition?." American Journal of Medical Genetics Part B- Neuropsychiatric Genetics 130B, no. 1 (2004): 8-9. Rioux, J. D., H. Karinen, K. Kocher, S. G. McMahon, P. Karkkainen, E. Janatuinen, M. Heikkinen, R. Julkunen, J. Pihlajamaki, A. Naukkarinen, V. M. Kosma, M. J. Daly, Eric S. Lander, and M. Laakso. "Genomewide search and association studies in a Finnish celiac disease population: Identification of a novel locus and replication of the HLA and CTLA4 loci." American Journal of Medical Genetics Part A 130A, no. 4 (2004): 345- 350. Michalkiewicz, M., T. Michalkiewicz, R. A. Ettinger, E. A. Rutledge, J. M. Fuller, D. H. Moralejo, B. Van Yserloo, A. J. MacMurray, A. E. Kwitek, H. J. Jacob, Eric S. Lander, and A. Lernmark. "Transgenic rescue demonstrates involvement of the Ian5 gene in T cell development in the rat." Physiological Genomics 19, no.
    [Show full text]
  • The Anatomy of Successful Computational Biology Software
    FEATURE The anatomy of successful computational biology software Stephen Altschul1, Barry Demchak2, Richard Durbin3, Robert Gentleman4, Martin Krzywinski5, Heng Li6, Anton Nekrutenko7, James Robinson6, Wayne Rasband8, James Taylor9 & Cole Trapnell10 Creators of software widely used in computational biology discuss the factors that contributed to their success he year was 1989 and Stephen Altschul What factors determine whether scientific tant thing is that Thad a problem. Sam Karlin, the brilliant software is successful? Cufflinks, Bowtie mathematician whose help he needed, was Stephen Altschul: BLAST was the first program (which is mainly Ben so convinced of the power of a mathemati- to assign rigorous statistics to useful scores of Langmead’s work) cally tractable but biologically constrained local sequence alignments. Before then people and TopHat were in measure of protein sequence similarity that had derived many large part at the right he would not listen to Altschul (or anyone different scoring sys- place at the right time. else for that matter). So Altschul essentially tems, and it wasn’t We were stepping into tricked him into solving the problem sty- clear why any should Cole Trapnell fields that were poised mying the field of computational biology have a particular developed the Tophat/ to explode, but which by posing it in terms of pure mathematics, advantage. I had made Cufflinks suite of short- really had a vacuum in devoid of any reference to biology. The treat a conjecture that read analysis tools. terms of usable tools. from that trick became known as the Karlin- every scoring system You get two things Altschul statistics that are a key part of that people proposed from being first.
    [Show full text]
  • BIOGRAPHICAL SKETCH NAME: Bonnie Berger POSITION TITLE
    BIOGRAPHICAL SKETCH NAME: Bonnie Berger POSITION TITLE: Simons Professor of Mathematics and Professor of Electrical Engineering & Computer Science EDUCATION/TRAINING (Begin with baccalaureate or other initial professional education, such as nursing, include postdoctoral training and residency training if applicable. Add/delete rows as necessary.) DEGREE Completion (if Date FIELD OF STUDY INSTITUTION AND LOCATION applicable) MM/YYYY Brandeis University, Waltham, MA AB 06/1983 Computer Science Massachusetts Institute of Technology SM 01/1986 Computer Science Massachusetts Institute of Technology Ph.D. 06/1990 Computer Science Massachusetts Institute of Technology Postdoc 06/1992 Applied Mathematics A. Personal Statement Many advances in modern biology revolve around automated data collection and the large resulting data sets. I am considered a pioneer in the area of bringing computer algorithms to the study of biological data, and a founder in this community that I have witnessed grow so profoundly over the last 20 years. I have made major contributions to many areas of computational biology and biomedicine, largely, though not exclusively through algorithmic insights, as demonstrated by ten thousand citations to my scientific papers and widely-used software. My research group works on diverse challenges, including and Computational Genomics, Structural Bioinformatics, High-throughput Technology Analysis and Design, Network Inference, and Data Privacy. We collaborate closely with biologists, MDs, and software engineers, implementing these new techniques in order to design experiments to maximally leverage the power of computation for biological exploration. Over the past five years I have been particularly active analyzing large and complex biological data sets; for example, my lab has played integral roles in modENCODE (non-coding RNA analysis), MPEG (biological data compression standard), and the Broad Institute’s sequence analysis efforts.
    [Show full text]
  • Statistical Models for Genome Assembly and Analysis by Atif
    Statistical Models for Genome Assembly and Analysis by Atif Hasan Rahman A dissertation submitted in partial satisfaction of the requirements for the degree of Doctor of Philosophy in Computer Science and the Designated Emphasis in Computational and Genomic Biology in the Graduate Division of the University of California, Berkeley Committee in charge: Professor Lior Pachter, Chair Professor Daniel Rokhsar Professor Yun S. Song Summer 2015 Statistical Models for Genome Assembly and Analysis Copyright 2015 by Atif Hasan Rahman 1 Abstract Statistical Models for Genome Assembly and Analysis by Atif Hasan Rahman Doctor of Philosophy in Computer Science and the Designated Emphasis in Computational and Genomic Biology University of California, Berkeley Professor Lior Pachter, Chair Genome assembly is the process of merging fragments of DNA sequences produced by shotgun sequencing in order to reconstruct the original genome. It is complicated by repeated regions in genomes, sequencing errors, and experimental biases. Here we focus on our efforts to confront some of the challenges in genome assembly and analysis of genomes to find regions associated with phenotypes using statistical models. Assembly algorithms have been extensively benchmarked using simulated data so that results can be compared to ground truth. However, in de novo assembly, only crude metrics such as contig number and size are typically used to evaluate assembly quality. We present CGAL, a novel likelihood-based approach to assembly assessment in the absence of a ground truth. We show that likelihood is more accurate than other metrics currently used for evaluating assemblies, and describe its application to the optimization and comparison of assembly algorithms.
    [Show full text]
  • Big Data Challenges in Genome Informatics
    Big Data Challenges in Genome Informatics Ka-Chun Wong March 28, 2018 Abstract In recent years, we have witnessed a dramatic data explosion in ge- nomics, thanks to the improvement in sequencing technologies and the drastically decreasing costs. We are entering the era of millions of avail- able genomes. Notably, each genome can be composed of billions of nu- cleotides stored as plain text files in GigaBytes (GBs). It is undeniable that those genome data impose unprecedented data challenges for us. In this article, we briefly discuss the big data challenges associated with ge- nomics in recent years. Introduction Since 1990s, the whole genomes of different species have been sequenced by their corresponding genome sequencing projects. In 1995, the first free-living organism Haemophilus influenzae was sequenced by the Institute for Genomic Research. In 1996, the first eukaryotic genome (Saccharomyces cerevisiase) was completely sequenced. In 2000, the first plant genome, Arabidopsis thaliana, was also sequenced by Arabidopsis Genome Initiative. In 2003, the Human Genome Project (HGP) announced its completion. Following the HGP, the En- cyclopedia of DNA Elements (ENCODE) project was started, revealing massive functional elements on the human genome in 2011 [9]. The drastically decreas- ing cost of sequencing also enables the 1000 Genomes Project and Roadmap Epigenomics Project to be carried out. Their results have been published in 2012 and 2015 respectively [8, 15]. Nonetheless, the massive genomic data gen- erated by those projects impose an unforeseen challenge for big data analysis at the scale of GigaBytes (GBs) or even TeraBytes (TBs). arXiv:1803.09632v1 [q-bio.OT] 20 Mar 2018 In particular, Next-Generation Sequencing (NGS) technologies have enabled massive data generation for different genomes [30, 20]; for instance, DNA se- quencing, protein-DNA binding occupancy [28] (e.g.
    [Show full text]
  • Combinatorics of Least-Squares Trees
    Combinatorics of Least-Squares Trees Author(s): Radu Mihaescu and Lior Pachter Source: Proceedings of the National Academy of Sciences of the United States of America, Vol. 105, No. 36 (Sep. 9, 2008), pp. 13206-13211 Published by: National Academy of Sciences Stable URL: http://www.jstor.org/stable/25464022 Accessed: 06-03-2017 22:47 UTC REFERENCES Linked references are available on JSTOR for this article: http://www.jstor.org/stable/25464022?seq=1&cid=pdf-reference#references_tab_contents You may need to log in to JSTOR to access the linked references. JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range of content in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new forms of scholarship. For more information about JSTOR, please contact [email protected]. Your use of the JSTOR archive indicates your acceptance of the Terms & Conditions of Use, available at http://about.jstor.org/terms National Academy of Sciences is collaborating with JSTOR to digitize, preserve and extend access to Proceedings of the National Academy of Sciences of the United States of America This content downloaded from 131.215.71.224 on Mon, 06 Mar 2017 22:47:23 UTC All use subject to http://about.jstor.org/terms Combinatorics of least-squares trees Radu Mihaescu" and Lior Pachter*5 + Departments of Mathematics and ?Computer Science, University of California, Berkeley, CA 94704 Edited by Peter J. Bickel, University of California, Berkeley, CA, and approved May 21, 2008 (received for review March 3, 2008) A recurring theme in the least-squares approach to phylogenet length of e is a projection ofthe dissimilarity map onto the coordi ics has been the discovery of elegant combinatorial formulas for nates given by pairs of leaves whose joining path contains at least the least-squares estimates of edge lengths.
    [Show full text]
  • Download (8MB)
    New Nonlinear Machine Learning Algorithms with Applications to Biomedical Data Science by Xiaoqian Wang BS, Zhejiang University, 2013 Submitted to the Graduate Faculty of the Swanson School of Engineering in partial fulfillment of the requirements for the degree of Doctor of Philosophy University of Pittsburgh 2019 UNIVERSITY OF PITTSBURGH SWANSON SCHOOL OF ENGINEERING This dissertation was presented by Xiaoqian Wang It was defended on May 31, 2019 and approved by Heng Huang, PhD, John A. Jurenko Endowed Professor, Department of Electrical and Computer Engineering Zhi-Hong Mao, PhD, Professor, Department of Electrical and Computer Engineering Wei Gao, PhD, Associate Professor, Department of Electrical and Computer Engineering Jingtong Hu, PhD, Assistant Professor, Department of Electrical and Computer Engineering Jian Ma, PhD, Associate Professor, Computational Biology Department, Carnegie Mellon University Dissertation Director: Heng Huang, PhD, John A. Jurenko Endowed Professor, Department of Electrical and Computer Engineering ii Copyright c by Xiaoqian Wang 2019 iii New Nonlinear Machine Learning Algorithms with Applications to Biomedical Data Science Xiaoqian Wang, PhD University of Pittsburgh, 2019 Recent advances in machine learning have spawned innovation and prosperity in various fields. In machine learning models, nonlinearity facilitates more flexibility and ability to better fit the data. However, the improved model flexibility is often accompanied by challenges such as overfitting, higher computational complexity, and less interpretability. Thus, it is an important problem of how to design new feasible nonlinear machine learning models to address the above different challenges posed by various data scales, and bringing new discoveries in both theory and applications. In this thesis, we propose several newly designed nonlinear machine learning algorithms, such as additive models and deep learning methods, to address these challenges and validate the new models via the emerging biomedical applications.
    [Show full text]
  • Whole-Genome Alignments and Polytopes for Comparative Genomics
    Whole-Genome Alignments and Polytopes for Comparative Genomics Colin Noel Dewey Electrical Engineering and Computer Sciences University of California at Berkeley Technical Report No. UCB/EECS-2006-104 http://www.eecs.berkeley.edu/Pubs/TechRpts/2006/EECS-2006-104.html August 3, 2006 Copyright © 2006, by the author(s). All rights reserved. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission. Whole-Genome Alignments and Polytopes for Comparative Genomics by Colin Noel Dewey B.S. (University of California, Berkeley) 2001 A dissertation submitted in partial satisfaction of the requirements for the degree of Doctor of Philosophy in Engineering - Electrical Engineering and Computer Sciences and the Designated Emphasis in Computational and Genomic Biology in the GRADUATE DIVISION of the UNIVERSITY OF CALIFORNIA, BERKELEY Committee in charge: Professor Lior Pachter, Chair Professor Richard Karp Professor Michael Eisen Fall 2006 The dissertation of Colin Noel Dewey is approved: Chair Date Date Date University of California, Berkeley Fall 2006 Whole-Genome Alignments and Polytopes for Comparative Genomics Copyright 2006 by Colin Noel Dewey 1 Abstract Whole-Genome Alignments and Polytopes for Comparative Genomics by Colin Noel Dewey Doctor of Philosophy in Engineering - Electrical Engineering and Computer Sciences Designated Emphasis in Computational and Genomic Biology University of California, Berkeley Professor Lior Pachter, Chair Whole-genome sequencing of many species has presented us with the opportunity to de- duce the evolutionary relationships between each and every nucleotide.
    [Show full text]