DOCSLIB.ORG

Optimization of Samtools Sorting Using Openmp Tasks Nathan T

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Optimization of Samtools Sorting Using Openmp Tasks Nathan T View metadata, citation and similar papers at core.ac.uk brought to you by CORE provided by Digital Repository @ Iowa State University Computer Science Publications Computer Science 4-26-2017 Optimization of SAMtools sorting using OpenMP tasks Nathan T. Weeks Iowa State University, [email protected] Glenn R. Luecke Iowa State University, [email protected] Follow this and additional works at: http://lib.dr.iastate.edu/cs_pubs Part of the Computer Sciences Commons, and the Mathematics Commons The ompc lete bibliographic information for this item can be found at http://lib.dr.iastate.edu/ cs_pubs/10. For information on how to cite this item, please visit http://lib.dr.iastate.edu/ howtocite.html. This Article is brought to you for free and open access by the Computer Science at Iowa State University Digital Repository. It has been accepted for inclusion in Computer Science Publications by an authorized administrator of Iowa State University Digital Repository. For more information, please contact [email protected]. Optimization of SAMtools Sorting Using OpenMP Tasks Nathan T. Weeks · Glenn R. Luecke Published 28 May 2017. The final publication is available at Springer via https://dx.doi.org/10.1007/s10586-017-0874-8 Abstract SAMtools is a widely-used genomics appli- 1 Introduction cation for post-processing high-throughput sequence alignment data. Such sequence alignment data are The advent of high-throughput sequencing (HTS) has commonly sorted to make downstream analysis more resulted in a rapid decline in DNA sequencing costs, efficient. However, this sorting process itself can be outpacing the growth in transistor density from Moore's computationally- and I/O-intensive: high-throughput Law [23]. As a result, genome sequencing is increas- sequence alignment files in the de facto standard Binary ing at a rapid pace. Genomics data generation is pre- Alignment/Map (BAM) format can be many gigabytes dicted to potentially dwarf Twitter, YouTube, and as- in size, and may need to be decompressed before sort- trophysics data combined by the year 2025 [19]. How- ing and compressed afterwards. As a result, BAM-file ever, performance-sensitive genomics applications have sorting can be a bottleneck in genomics workflows. This generally not even scaled with Moore's Law, mainly be- paper describes a case study on the performance analy- cause many such applications are unprepared to fully sis and optimization of SAMtools for sorting large BAM utilize increasingly-parallel processors. Consequently, files. OpenMP task parallelism and memory optimiza- algorithmic improvements, specialized computing hard- tion techniques resulted in a speedup of 5.9X versus the ware, and new storage technologies are needed to upstream SAMtools 1.3.1 for an internal (in-memory) cope with storing, processing, and analyzing increasing sort of 24.6 GiB of compressed BAM data (102.6 GiB amounts of genomics data. uncompressed) with 32 processor cores, while a 1.98X HTS workflows often include sequence alignment speedup was achieved for an external (out-of-core) sort to a reference genome. SAMtools [12] is a utility for of a 271.4 GiB BAM file. performing operations on the resulting sequence align- ment data such as sorting, indexing, selecting subsets, compressing, and reporting various statistics. These se- quence alignments can be represented in one of three Keywords Bioinformatics, High-Throughput Se- industry-standard formats: the Sequence Alignmen- quencing, OpenMP, Sorting, Burst Buffer t/Map (SAM) text format; its binary equivalent, the Binary Alignment/Map (BAM) format; or the more re- cent CRAM format, which utilizes reference-sequence- based compression and lossy quality scores. N. Weeks SAMtools utilizes the codeveloped HTSlib library Department of Computer Science, Iowa State University, Ames, IA, USA for reading, parsing, and compressing/decompressing E-mail: [email protected] SAM/BAM/CRAM data. As SAMtools and HTSlib G. Luecke are developed in lockstep with the SAM/BAM/CRAM Department of Mathematics, Iowa State University, specifications, many consider these to be the reference Ames, IA, USA implementations among similar software that work with E-mail: [email protected] these formats. 2 Nathan T. Weeks, Glenn R. Luecke Sequence alignments are typically sorted to make for a subset of performance-critical SAMtools function- downstream analysis more efficient. However, the sort- ality. Written from the ground up in the D program- ing operation can itself be a bottleneck in HTS work- ming language to implement parallelism, Sambamba flows [8, 18]. Sorting large data sets is well-known strives to fully exploit multi-core CPUs. to be a resource-intensive problem that can bene- elPrep [9] is a Common Lisp/Python application for fit from parallelism [7]. The SAMtools 1.3.1 sorting multi-threaded, in-memory execution of the subset of pipeline is already parallelized using the POSIX threads SAMtools functionality that focuses on preparing se- (pthreads) API. But, as the performance analysis in quence alignment data for variant calling. While el- Section 3 illustrates, this parallelization is incomplete, Prep's in-memory processing benefits performance, it and exhibits inefficiencies. Because of the widespread requires a lot of memory for sorting: the elPrep 2.5 doc- adoption of SAMtools|including by HTS workflows umentation states "As a rule of thumb, elPrep requires that run on large-scale public cloud resources, such as 6x times more RAM memory than the size of the in- Churchill [11], and massively parallel supercomputers, put file in .sam format when it is used for sorting". such as MegaSeq [16]|performance enhancements to it For comparison, SAMtools can accomplish an internal would have broad impact. sort of the 24.6 GiB BAM (∼85:5 GiB SAM) data set This work1 extends our previous effort to improve described in Sect. 3 on a 128 GiB-memory compute the performance of SAMtools for the purpose of sort- node. elPrep can sort such BAM files within an accessi- ing BAM files [22], primarily by 1) utilizing an alterna- ble memory limit at the cost of extra disk space and tive algorithm employing fine-grained tasking for com- I/O overhead: the protocol involves splitting the in- pression during BAM encoding to allow adequate load put BAM files (each containing alignments with respect balancing with a larger number of threads, 2) utiliz- to a different reference sequence/chromosome), sorting ing circular buffers to reduce calls to memcpy() and en- each BAM file independently, and merging the result- hance concurrency, 3) improving the performance of ex- ing sorted BAM files to produce a single sorted BAM ternal sorting by keeping sorted BAM records in mem- file. Since elPrep uses SAMtools as a library for decod- ory where possible instead of writing them to secondary ing/encoding BAM files, the optimizations described in storage before merging, and 4) benchmarking the exter- this paper could benefit elPrep as well. nal sort with a much larger data set, leveraging Cori's Similar to elPrep, biobambam2 [21] focuses on align- newly-available Burst Buffer to store input, output, and ment preprocessing in preparation for variant calling. intermediate BAM files. Coded in C++, biobambam2 can perform a multi- The rest of this paper is organized as follows. Sec- threaded SAM/BAM/CRAM sort by coordinate under tion 2 lists other software tools that aim to efficiently the condition that the input is already sorted by query implement performance-critical SAMtools functional- (read) name, or sort by query name when the input is ity. Section 3 characterizes the performance of SAM- in any order. tools for internal sorting of a large BAM file. Section 4 Some sequence aligners, such as Isaac [17], sort describes optimizations implemented in this work to ad- alignments in-memory before output. This strategy dress the identified performance bottlenecks. The im- avoids the I/O overhead associated with emitting align- pact of this work on performance is measured in Sec- ments from a sequence alignment process and reading tion 5. Section 6 explores other possible optimizations. them into a separate sort process. Section 7 summarizes this effort to address the perfor- DNANexus has proposed a fork of SAMtools that mance limitations of SAMtools. leverages RocksDB for improved sorting/merging per- formance [13], as well as a patch to HTSlib that im- proves concurrency in BGZF encoding2. 2 Related Work Both Intel and CloudFlare have created optimized versions of the zlib compression library. These have Picard [1] from the Broad Institute is a Java analog to been shown to benefit SAMtools compression perfor- SAMtools, providing similar HTS-processing function- mance3. ality. Picard currently supports only serial SAM/BAM The HTSLib library utilized by SAMtools for sorting. SAM/BAM/CRAM I/O supports multi-threaded en- Similar software exists with the primary goal of coding/decoding of CRAM data using a custom outperforming SAMtools. One such software, Sam- pthreads-based work-stealing thread pool (adapted bamba [20], aims to be a high-performance replacement from the implementation in the Scramble [4] library). 1 Source code for SAMtools optimizations available at https://doi.org/10.5281/zenodo.262169, and HTSlib opti- 2 https://github.com/samtools/htslib/pull/51 mizations at https://doi.org/10.5281/zenodo.262161 3 http://www.htslib.org/benchmarks/zlib.html Optimization of SAMtools Sorting Using OpenMP Tasks 3 As of this writing, support for multi-threaded BAM en- originally 19.9 GiB (102.6 GiB uncompressed), the file coding/decoding using this thread pool implementation size increased to 24.6 GiB. This could be attributed at has been committed to the development branch of HT- least partially to the reduced likelihood of overlapping Slib4. The alternative approach presented in this paper sequences appearing within the same 64 KiB BGZF uses OpenMP 4.0 constructs to implement concurrency. block, providing less repetition for the compression al- The OpenMP API is supported by almost all major C gorithm to leverage.
Load more

Recommended publications
  • De Novo Genomic Analyses for Non-Model Organisms: an Evaluation of Methods Across a Multi-Species Data Set
    De novo genomic analyses for non-model organisms: an evaluation of methods across a multi-species data set Sonal Singhal [email protected] Museum of Vertebrate Zoology University of California, Berkeley 3101 Valley Life Sciences Building Berkeley, California 94720-3160 Department of Integrative Biology University of California, Berkeley 1005 Valley Life Sciences Building Berkeley, California 94720-3140 June 5, 2018 Abstract High-throughput sequencing (HTS) is revolutionizing biological research by enabling sci- entists to quickly and cheaply query variation at a genomic scale. Despite the increasing ease of obtaining such data, using these data effectively still poses notable challenges, especially for those working with organisms without a high-quality reference genome. For every stage of analysis – from assembly to annotation to variant discovery – researchers have to distinguish technical artifacts from the biological realities of their data before they can make inference. In this work, I explore these challenges by generating a large de novo comparative transcriptomic dataset data for a clade of lizards and constructing a pipeline to analyze these data. Then, using a combination of novel metrics and an externally validated variant data set, I test the efficacy of my approach, identify areas of improvement, and propose ways to minimize these errors. I arXiv:1211.1737v1 [q-bio.GN] 8 Nov 2012 find that with careful data curation, HTS can be a powerful tool for generating genomic data for non-model organisms. Keywords: de novo assembly, transcriptomes, suture zones, variant discovery, annotation Running Title: De novo genomic analyses for non-model organisms 1 Introduction High-throughput sequencing (HTS) is poised to revolutionize the field of evolutionary genetics by enabling researchers to assay thousands of loci for organisms across the tree of life.
    [Show full text]
  • MATCH-G Program
    MATCH-G Program The MATCH-G toolset MATCH-G (Mutational Analysis Toolset Comparing wHole Genomes) Download the Toolset The toolset is prepared for use with pombe genomes and a sample genome from Sanger is included. However, it is written in such a way as to allow it to utilize any set or number of chromosomes. Simply follow the naming convention "chromosomeX.contig.embl" where X=1,2,3 etc. For use in terminal windows in Mac OSX or Unix-like environments where Perl is present by default. Toolset Description Version History 2.0 – Bug fixes related to scaling to other genomes. First version of GUI interface. 1.0 – Initial Release. Includes build, alignment, snp, copy, and gap resolution routines in original form. Please note this project in under development and will undergo significant changes. Project Description The MATCH-G toolset has been developed to facilitate the evaluation of whole genome sequencing data from yeasts. Currently, the toolset is written for pombe strains, however the toolset is easily scalable to other yeasts or other sequenced genomes. The included tools assist in the identification of SNP mutations, short (and long) insertion/deletions, and changes in gene/region copy number between a parent strain and mutant or revertant strain. Currently, the toolset is run from the command line in a unix or similar terminal and requires a few additional programs to run noted below. Installation It is suggested that a separate folder be generated for the toolset and generated files. Free disk space proportional to the original read datasets is recommended. The toolset utilizes and requires a few additional free programs: Bowtie rapid alignment software: http://bowtie-bio.sourceforge.net/index.shtml (Langmead B, Trapnell C, Pop M, Salzberg SL.
    [Show full text]
  • Large Scale Genomic Rearrangements in Selected Arabidopsis Thaliana T
    bioRxiv preprint doi: https://doi.org/10.1101/2021.03.03.433755; this version posted March 7, 2021. The copyright holder for this preprint (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made available under aCC-BY 4.0 International license. 1 Large scale genomic rearrangements in selected 2 Arabidopsis thaliana T-DNA lines are caused by T-DNA 3 insertion mutagenesis 4 5 Boas Pucker1,2+, Nils Kleinbölting3+, and Bernd Weisshaar1* 6 1 Genetics and Genomics of Plants, Center for Biotechnology (CeBiTec), Bielefeld University, 7 Sequenz 1, 33615 Bielefeld, Germany 8 2 Evolution and Diversity, Department of Plant Sciences, University of Cambridge, Cambridge, 9 United Kingdom 10 3 Bioinformatics Resource Facility, Center for Biotechnology (CeBiTec, Bielefeld University, 11 Sequenz 1, 33615 Bielefeld, Germany 12 + authors contributed equally 13 * corresponding author: Bernd Weisshaar 14 15 BP: [email protected], ORCID: 0000-0002-3321-7471 16 NK: [email protected], ORCID: 0000-0001-9124-5203 17 BW: [email protected], ORCID: 0000-0002-7635-3473 18 19 20 21 22 23 24 25 26 27 28 29 30 page 1 bioRxiv preprint doi: https://doi.org/10.1101/2021.03.03.433755; this version posted March 7, 2021. The copyright holder for this preprint (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made available under aCC-BY 4.0 International license.
    [Show full text]
  • An Open-Sourced Bioinformatic Pipeline for the Processing of Next-Generation Sequencing Derived Nucleotide Reads
    bioRxiv preprint doi: https://doi.org/10.1101/2020.04.20.050369; this version posted May 28, 2020. The copyright holder for this preprint (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made available under aCC-BY 4.0 International license. An open-sourced bioinformatic pipeline for the processing of Next-Generation Sequencing derived nucleotide reads: Identification and authentication of ancient metagenomic DNA Thomas C. Collin1, *, Konstantina Drosou2, 3, Jeremiah Daniel O’Riordan4, Tengiz Meshveliani5, Ron Pinhasi6, and Robin N. M. Feeney1 1School of Medicine, University College Dublin, Ireland 2Division of Cell Matrix Biology Regenerative Medicine, University of Manchester, United Kingdom 3Manchester Institute of Biotechnology, School of Earth and Environmental Sciences, University of Manchester, United Kingdom [email protected] 5Institute of Paleobiology and Paleoanthropology, National Museum of Georgia, Tbilisi, Georgia 6Department of Evolutionary Anthropology, University of Vienna, Austria *Corresponding Author Abstract The emerging field of ancient metagenomics adds to these Bioinformatic pipelines optimised for the processing and as- processing complexities with the need for additional steps sessment of metagenomic ancient DNA (aDNA) are needed in the separation and authentication of ancient sequences from modern sequences. Currently, there are few pipelines for studies that do not make use of high yielding DNA cap- available for the analysis of ancient metagenomic DNA ture techniques. These bioinformatic pipelines are tradition- 1 4 ally optimised for broad aDNA purposes, are contingent on (aDNA) ≠ The limited number of bioinformatic pipelines selection biases and are associated with high costs.
    [Show full text]
  • Sequence Alignment/Map Format Specification
    Sequence Alignment/Map Format Specification The SAM/BAM Format Specification Working Group 3 Jun 2021 The master version of this document can be found at https://github.com/samtools/hts-specs. This printing is version 53752fa from that repository, last modified on the date shown above. 1 The SAM Format Specification SAM stands for Sequence Alignment/Map format. It is a TAB-delimited text format consisting of a header section, which is optional, and an alignment section. If present, the header must be prior to the alignments. Header lines start with `@', while alignment lines do not. Each alignment line has 11 mandatory fields for essential alignment information such as mapping position, and variable number of optional fields for flexible or aligner specific information. This specification is for version 1.6 of the SAM and BAM formats. Each SAM and BAMfilemay optionally specify the version being used via the @HD VN tag. For full version history see Appendix B. Unless explicitly specified elsewhere, all fields are encoded using 7-bit US-ASCII 1 in using the POSIX / C locale. Regular expressions listed use the POSIX / IEEE Std 1003.1 extended syntax. 1.1 An example Suppose we have the following alignment with bases in lowercase clipped from the alignment. Read r001/1 and r001/2 constitute a read pair; r003 is a chimeric read; r004 represents a split alignment. Coor 12345678901234 5678901234567890123456789012345 ref AGCATGTTAGATAA**GATAGCTGTGCTAGTAGGCAGTCAGCGCCAT +r001/1 TTAGATAAAGGATA*CTG +r002 aaaAGATAA*GGATA +r003 gcctaAGCTAA +r004 ATAGCT..............TCAGC -r003 ttagctTAGGC -r001/2 CAGCGGCAT The corresponding SAM format is:2 1Charset ANSI X3.4-1968 as defined in RFC1345.
    [Show full text]
  • Assembly Exercise
    Assembly Exercise Turning reads into genomes Where we are • 13:30-14:00 – Primer Design to Amplify Microbial Genomes for Sequencing • 14:00-14:15 – Primer Design Exercise • 14:15-14:45 – Molecular Barcoding to Allow Multiplexed NGS • 14:45-15:15 – Processing NGS Data – de novo and mapping assembly • 15:15-15:30 – Break • 15:30-15:45 – Assembly Exercise • 15:45-16:15 – Annotation • 16:15-16:30 – Annotation Exercise • 16:30-17:00 – Submitting Data to GenBank Log onto ILRI cluster • Log in to HPC using ILRI instructions • NOTE: All the commands here are also in the file - assembly_hands_on_steps.txt • If you are like me, it may be easier to cut and paste Linux commands from this file instead of typing them in from the slides Start an interactive session on larger servers • The interactive command will start a session on a server better equipped to do genome assembly $ interactive • Switch to csh (I use some csh features) $ csh • Set up Newbler software that will be used $ module load 454 A norovirus sample sequenced on both 454 and Illumina • The vendors use different file formats unknown_norovirus_454.GACT.sff unknown_norovirus_illumina.fastq • I have converted these files to additional formats for use with the assembly tools unknown_norovirus_454_convert.fasta unknown_norovirus_454_convert.fastq unknown_norovirus_illumina_convert.fasta Set up and run the Newbler de novo assembler • Create a new de novo assembly project $ newAssembly de_novo_assembly • Add read data to the project $ addRun de_novo_assembly unknown_norovirus_454.GACT.sff
    [Show full text]
  • Supplemental Material Nanopore Sequencing of Complex Genomic
    Supplemental Material Nanopore sequencing of complex genomic rearrangements in yeast reveals mechanisms of repeat- mediated double-strand break repair McGinty, et al. Contents: Supplemental Methods Supplemental Figures S1-S6 Supplemental References Supplemental Methods DNA extraction: The DNA extraction protocol used is a slight modification of a classic ethanol precipitation. First, yeast cultures were grown overnight in 2 ml of complete media (YPD) and refreshed for 4 hours in an additional 8 ml YPD to achieve logarithmic growth. The cultures were spun down, and the pellets were resuspended in 290 µl of solution containing 0.9M Sorbitol and 0.1M EDTA at pH 7.5. 10 µl lyticase enzyme was added, and the mixture was incubated for 30 minutes at 37oC in order to break down the yeast cell wall. The mixture was centrifuged at 8,000 rpm for two minutes, and the pellet was resuspended in 270 µl of solution containing 50mM Tris 20mM EDTA at pH 7.5, and 30 µl of 10% SDS. Following five minutes incubation at room temperature, 150 µl of chilled 5M potassium acetate solution was added. This mixture was incubated for 10 min at 4oC and centrifuged for 10 min at 13,000 rpm. The supernatant was then combined with 900 µl of pure ethanol, which had been chilled on ice. This solution was stored overnight at -20oC, and then centrifuged for 20 minutes at 4,000 rpm in a refrigerated centrifuge. The pellet was then washed twice with 70% ethanol, allowed to dry completely, and then resuspended in 50 µl TE (10mM Tris, 1mM EDTA, pH 8).
    [Show full text]
  • Providing Bioinformatics Services on Cloud
    Providing Bioinformatics Services on Cloud Christophe Blanchet, Clément Gauthey InfrastructureC. Blanchet Distributed and C. Gauthey for Biology IDB-IBCPEGI CF13, CNRS Manchester, FR3302 - LYON 9 April - FRANCE2013 http://idee-b.ibcp.fr Infrastructure Distributed for Biology - IDB CNRS-IBCP FR3302, Lyon, FRANCE IDB acknowledges co-funding by the European Community's Seventh Framework Programme (INFSO-RI-261552) and the French National Research Agency's Arpege Programme (ANR-10-SEGI-001) Bioinformatics Today • Biological data are big data • 1512 online databases (NAR Database Issue 2013) • Institut Sanger, UK, 5 PB • Beijing Genome Institute, China, 4 sites, 10 PB ➡ Big data in lot of places • Analysing such data became difficult • Scale-up of the analyses : gene/protein to complete genome/ proteome, ... • Lot of different daily-used tools • That need to be combined in workflows • Usual interfaces: portals, Web services, federation,... ➡ Datacenters with ease of access/use • Distributed resources ADN Experimental platforms: NGS, imaging, ... • ADN • Bioinformatics platforms BI Federation of datacenters M ➡ BI CC ADN ADN BI ADN EGI CF13, Manchester, 9 April 2013 ADN ADN BI ADN ADN M M Sequencing Genomes Complete genome sequencing become a lab commodity with NGS (cheap and efficient) source: www.genomesonline.org source: www.politigenomics.com/next-generation-sequencing-informatics EGI CF13, Manchester, 9 April 2013 Infrastructures in Biology Lot of tools and web services to treat and vizualize lot of data EGI CF13, Manchester, 9 April 2013
    [Show full text]
  • Syntax Highlighting for Computational Biology Artem Babaian1†, Anicet Ebou2, Alyssa Fegen3, Ho Yin (Jeffrey) Kam4, German E
    bioRxiv preprint doi: https://doi.org/10.1101/235820; this version posted December 20, 2017. The copyright holder has placed this preprint (which was not certified by peer review) in the Public Domain. It is no longer restricted by copyright. Anyone can legally share, reuse, remix, or adapt this material for any purpose without crediting the original authors. bioSyntax: Syntax Highlighting For Computational Biology Artem Babaian1†, Anicet Ebou2, Alyssa Fegen3, Ho Yin (Jeffrey) Kam4, German E. Novakovsky5, and Jasper Wong6. 5 10 15 Affiliations: 1. Terry Fox Laboratory, BC Cancer, Vancouver, BC, Canada. [[email protected]] 2. Departement de Formation et de Recherches Agriculture et Ressources Animales, Institut National Polytechnique Felix Houphouet-Boigny, Yamoussoukro, Côte d’Ivoire. [[email protected]] 20 3. Faculty of Science, University of British Columbia, Vancouver, BC, Canada [[email protected]] 4. Faculty of Mathematics, University of Waterloo, Waterloo, ON, Canada. [[email protected]] 5. Department of Medical Genetics, University of British Columbia, Vancouver, BC, 25 Canada. [[email protected]] 6. Genome Science and Technology, University of British Columbia, Vancouver, BC, Canada. [[email protected]] Correspondence†: 30 Artem Babaian Terry Fox Laboratory BC Cancer Research Centre 675 West 10th Avenue Vancouver, BC, Canada. V5Z 1L3. 35 Email: [[email protected]] bioRxiv preprint doi: https://doi.org/10.1101/235820; this version posted December 20, 2017. The copyright holder has placed this preprint (which was not certified by peer review) in the Public Domain. It is no longer restricted by copyright. Anyone can legally share, reuse, remix, or adapt this material for any purpose without crediting the original authors.
    [Show full text]
  • Software List for Biology, Bioinformatics and Biostatistics CCT
    Software List for biology, bioinformatics and biostatistics v CCT - Delta Software Version Application short read assembler and it works on both small and large (mammalian size) ALLPATHS-LG 52488 genomes provides a fast, flexible C++ API & toolkit for reading, writing, and manipulating BAMtools 2.4.0 BAM files a high level of alignment fidelity and is comparable to other mainstream Barracuda 0.7.107b alignment programs allows one to intersect, merge, count, complement, and shuffle genomic bedtools 2.25.0 intervals from multiple files Bfast 0.7.0a universal DNA sequence aligner tool analysis and comprehension of high-throughput genomic data using the R Bioconductor 3.2 statistical programming BioPython 1.66 tools for biological computation written in Python a fast approach to detecting gene-gene interactions in genome-wide case- Boost 1.54.0 control studies short read aligner geared toward quickly aligning large sets of short DNA Bowtie 1.1.2 sequences to large genomes Bowtie2 2.2.6 Bowtie + fully supports gapped alignment with affine gap penalties BWA 0.7.12 mapping low-divergent sequences against a large reference genome ClustalW 2.1 multiple sequence alignment program to align DNA and protein sequences assembles transcripts, estimates their abundances for differential expression Cufflinks 2.2.1 and regulation in RNA-Seq samples EBSEQ (R) 1.10.0 identifying genes and isoforms differentially expressed EMBOSS 6.5.7 a comprehensive set of sequence analysis programs FASTA 36.3.8b a DNA and protein sequence alignment software package FastQC
    [Show full text]
  • Computational Protocol for Assembly and Analysis of SARS-Ncov-2 Genomes
    PROTOCOL Computational Protocol for Assembly and Analysis of SARS-nCoV-2 Genomes Mukta Poojary1,2, Anantharaman Shantaraman1, Bani Jolly1,2 and Vinod Scaria1,2 1CSIR Institute of Genomics and Integrative Biology, Mathura Road, Delhi 2Academy of Scientific and Innovative Research (AcSIR) *Corresponding email: MP [email protected]; AS [email protected]; BJ [email protected]; VS [email protected] ABSTRACT SARS-CoV-2, the pathogen responsible for the ongoing Coronavirus Disease 2019 pandemic is a novel human-infecting strain of Betacoronavirus. The outbreak that initially emerged in Wuhan, China, rapidly spread to several countries at an alarming rate leading to severe global socio-economic disruption and thus overloading the healthcare systems. Owing to the high rate of infection of the virus, as well as the absence of vaccines or antivirals, there is a lack of robust mechanisms to control the outbreak and contain its transmission. Rapid advancement and plummeting costs of high throughput sequencing technologies has enabled sequencing of the virus in several affected individuals globally. Deciphering the viral genome has the potential to help understand the epidemiology of the disease as well as aid in the development of robust diagnostics, novel treatments and prevention strategies. Towards this effort, we have compiled a comprehensive protocol for analysis and interpretation of the sequencing data of SARS-CoV-2 using easy-to-use open source utilities. In this protocol, we have incorporated strategies to assemble the genome of SARS-CoV-2 using two approaches: reference-guided and de novo. Strategies to understand the diversity of the local strain as compared to other global strains have also been described in this protocol.
    [Show full text]
  • Alignment-Free Sequence Comparison: Benefits, Applications, and Tools Andrzej Zielezinski1, Susana Vinga2, Jonas Almeida3 and Wojciech M
    Zielezinski et al. Genome Biology (2017) 18:186 DOI 10.1186/s13059-017-1319-7 REVIEW Open Access Alignment-free sequence comparison: benefits, applications, and tools Andrzej Zielezinski1, Susana Vinga2, Jonas Almeida3 and Wojciech M. Karlowski1* Abstract Alignment-free sequence analyses have been applied to problems ranging from whole-genome phylogeny to the classification of protein families, identification of horizontally transferred genes, and detection of recombined sequences. The strength of these methods makes them particularly useful for next-generation sequencing data processing and analysis. However, many researchers are unclear about how these methods work, how they compare to alignment-based methods, and what their potential is for use for their research. We address these questions and provide a guide to the currently available alignment-free sequence analysis tools. Introduction most alignment programs also model inserted/deleted The 1980s and 1990s were a flourishing time not only states (gaps). However, as our understanding of complex for pop music but also for bioinformatics, where the evolutionary scenarios and our knowledge about the pat- emergence of sequence comparison algorithms revolu- terns and properties of biological sequences advanced, tionized the computational and molecular biology fields. we gradually uncovered some downsides of sequence At that time, many computational biologists quickly be- comparisons based solely on alignments. came stars in the field by developing programs for se- quence alignment, which is a method that positions the Five cases where alignment-based sequence ana- biological sequences’ building blocks to identify regions lysis might be troublesome of similarity that may have consequences for functional, First, alignment-producing programs assume that hom- structural, or evolutionary relationships.
    [Show full text]