ManuscriptbioRxiv preprint doi: https://doi.org/10.1101/024448; this version posted August 21, 2015. The copyright holder for this preprint (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made available under Click here to download Manuscript: bmc_article.tex aCC-BY 4.0 International license. Click here to view linked References Shelton and Brown 1 2 3 4 5 RESEARCH 6 7 8 9 Fasta-O-Matic: a tool to sanity check and if 10 11 needed reformat FASTA files 12 13 Jennifer M Shelton1 and Susan J Brown1* 14 15 *Correspondence: [email protected] 16 1KSU/K-INBRE Bioinformatics Abstract 17 Center, Division of Biology, 18 Kansas State University, Background: As the sheer volume of bioinformatic sequence data increases, the 19 Manhattan, KS, USA only way to take advantage of this content is to more completely automate Full list of author information is 20 available at the end of the article robust analysis workflows. Analysis bottlenecks are often mundane and 21 overlooked processing steps. Idiosyncrasies in reading and/or writing 22 bioinformatics file formats can halt or impair analysis workflows by interfering 23 with the transfer of data from one informatics tools to another. 24 Results: Fasta-O-Matic automates handling of common but minor formatissues 25 that otherwise may halt pipelines. The need for automation must be balanced by 26 the need for manual confirmation that any formatting error is actually minor 27 28 rather than indicative of a corrupt data file. To that end Fasta-O-Matic reports 29 any issues detected to the user with optionally color coded and quiet or verbose 30 logs. 31 Fasta-O-Matic can be used as a general pre-processing tool inbioinformatics 32 workflows (e.g. to automatically wrap FASTA files so that they can be read by 33 BioPerl). It was also developed as a sanity check for bioinformatic core facilities 34 that tend to repeat common analysis steps on FASTA files received from 35 disparate sources. Fasta-O-Matic can be set with format requirements specific to 36 downstream tools as a first step in a larger analysis workflow. 37 Availability: Fasta-O-Matic is available free of charge to academic and 38 non-profit institutions on GitHub. 39 40 Keywords: FASTA; sequence data; bioinformatics file format 41 42 43 44 45 Background 46 Sequence data can be stored as text with each letter representing a nucleic acid 47 48 (DNA and RNA) or amino acid (protein). The linear nature of these molecules 49 makes it natural to represent them as strings, finite sequences of characters. Al- 50 though it has been argued that a graph, a network of edges connected by vertices, 51 is a more accurate way to store genomic sequences because graphs allow the inclu- 52 53 sion of alternate alleles and alternate possible assemblies [1] all of the most common 54 methods for storing sequences (FASTA, FASTQ, SAM/BAM) use a linear strings. 55 Other decisions about how to represent sequence data can be more arbitrary. For 56 57 example, any character that is not used as a base or an amino acid could be used 58 to indicate the beginning of a new sequence. Additionally text could be wrapped to 59 limit the information content in any one line of a file. The advantage of wrapping 60 text is that some programs can then be designed to work one line at time limiting 61 62 the burden of each step (e.g. the program would never have to process an entire 63 64 65 bioRxiv preprint doi: https://doi.org/10.1101/024448; this version posted August 21, 2015. The copyright holder for this preprint (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made available under aCC-BY 4.0 International license. Shelton and Brown Page 2 of 8 1 2 3 4 5 chromosome of sequence data in a single step). The disadvantage is that code must 6 7 be slightly more complex to load an entire sequence record into the working memory. 8 9 0.1 FASTA file format specifications versus recommendations 10 The popular FASTA file format stores sequence records and has very minimal format 11 12 requirements [2]. Each sequence is preceded by a header/description line that begins 13 with a > symbol. Sequence lines can include any standard International Unionof 14 Pure and Applied Chemistry (IUPAC) single character symbols for nucleic acids 15 or amino acids or the ambiguous codes that indicate possible residues or bases [3]. 16 They can also include - to indicate alignment gaps and * to indicate stop codons. 17 18 NCBI recommends wrapping FASTA file sequences lines [2]. It is also common 19 practice to use the first ‘word’ in a header (i.e. any character string to the left of 20 the first space in the header) as the unique sequence id. Although these features 21 are common they are not required leading to format compatibility issues with tools 22 23 that treat these conventions as required. 24 25 0.2 Customizing FASTA files to ensure that information is properly interpreted by 26 downstream tools 27 Regardless of whether a FASTA file is technically improperly formatted or it’s for- 28 29 mat merely violates a popular convention, it is critical to quality analysis workflows 30 that data is converted into a format that will be correctly interpreted by down- 31 stream tools. Formatting issues can fall into multiple categories including actual 32 format errors and formats that are not technically wrong but are non-standard, 33 34 causing some tools to throw an error. 35 Some format errors indicate a major problem like an attempt to use the wrong 36 data format (e.g. the first line is not a FASTA header because it does not begin 37 with a > character). These types of errors will be subsequently referred to as fatal. 38 Alternately, some formatting issues occur commonly without indicating the FASTA 39 40 file is corrupt (e.g. improperly wrapped/unwrapped sequence lines, missing final new 41 line characters, unusual new line characters like \r). These issues will be referred 42 to as non-fatal. Fatal formatting issues should cause processing to stop. Non-fatal 43 formatting issues should be automatically corrected according to the most common 44 45 resolution for this type of error. While downstream processing continues, the analyst 46 can double check the automated decision to reformat non-fatal issues. This way 47 workflow would not be slowed for trivial reformatting steps and the more rare 48 problems (e.g. when a missing last new line was caused by incomplete file transfer) 49 50 could still be caught. 51 52 0.3 Existing tools 53 Existing bioinformatics tools address FASTA format inconsistencies.Howevermany 54 tools either halt and exit with an error (e.g. BioPerl [4], [5], [6]) or can produce 55 56 reformatted output FASTA but cannot determine if there is a formatting issue to 57 begin with (e.g. EMBOSS Seqret [7]). 58 The BioPerl module DB::Fasta will halt if a FASTA is inconsistently wrapped or 59 if a line of sequence is too long (as in an unwrapped genome FASTA). This has the 60 disadvantage of requiring human intervention to wrap and restart analysis. 61 62 Code: 63 64 65 bioRxiv preprint doi: https://doi.org/10.1101/024448; this version posted August 21, 2015. The copyright holder for this preprint (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made available under aCC-BY 4.0 International license. Shelton and Brown Page 3 of 8 1 2 3 4 5 #!/usr/bin/perl 6 7 use Bio::Seq; 8 use Bio::SeqIO; 9 use Bio::DB::Fasta; #makes a searchable db from FASTA file 10 11 my $out_file_temp = ’/home/bionano/test_db/all.fa’; 12 #Create new FASTA outfile object 13 my $seq_out = Bio::SeqIO->new(’-file’ => ">$out_file_temp",’-format’ => ’fasta’); 14 #Load FASTA file as DB 15 16 my $db = Bio::DB::Fasta->new("/home/bionano/test_db/miswrapped.fa"); 17 my $seq_obj = $db->get_Seq_by_id(’seq’); # get FASTA records using headers 18 #(where header = first ’word’ so really header whitespace should also be 19 #removed for this file) 20 21 $seq_out->write_seq($seq_obj); 22 Input: 23 >seq 1 24 25 ACTGTGTGCAATCGCTGNNNNCTCTCATCGGATCTTGCAATCGCTNNNCTCTCATCGGATTGCAATCGCTNNNCTtcatcCGGAT 26 CGCTGNNNNCTGTGTGCAATCGCTGNNNNCTCCTGATCGCTGNNNNCTGTGTGCAATCGCTGNNNNCTCCTGCAATCGCTGNNNN 27 CTCCTGTTCGNATCGatcctctgtttatgcttatagctagctgatcgtagnnntcaacgt 28 CTAGAGCGCAGCTCTGGGGGATTACTACTCACTACATCATTAGATCAGATacgactcann 29 30 >seq 2 31 cttatagctagctgatAATCGCTGNNTCATCGGATCTTGCCTTGCAATCGtcatcCGtcC 32 CGCTGNNNNCTGTGTGCAnnnnnnnnnnncgtaaaacgcctcctccgactcgTCTCTAGG 33 CTAGAGCGCAGCTCTGGGGGATTACTACTCACTACATCATTAGATCAGATacgactcann 34 35 nnnctacgCTATCAGGTCTCGAG 36 >seq 3 37 ATCAGCGCTCTATATGGCTCTGATTATAGTTTGCATTCATATGCTGATCTTctcagnntc 38 cttgacgctcgctATCTGTAGATCTGTACTtcagacagctcTCAGCAGNNNCTCAGCAGC 39 40 CTACGACAGTcatgcagactagcagt 41 Output: 42 ------EXCEPTION ------43 44 MSG: Each line of the fasta entry must be the same length exceptthelast. 45 Line above #5 ’CTAGAGCGCAGCTCTGGGGG..’ is 61 != 86 chars... 46 EMBOSS seqret was designed as a very flexible tool to convert from one properly 47 48 formatted file to another properly but distinctly formatted file. It also was designed 49 to accept poorly formatted data (e.g. a FASTA missing the final new line that is 50 improperly wrapped) and export a reformatted file (e.g. wrapped after 60 bases 51 with a final new line). 52 53 Code: 54 seqret -stdout -sequence test.fa -outseq test_reformat.fa 55 Input: 56 57 >my header 58 AAAAAAAAAAAATTTTTTCCCCGGCGCGCGCGCTATAGCGCTATANNNNNNNNNNNNNNN 59 ATATATATATAT 60 ATTATTATATATATATTCTCTCTGGGCTCGCGTCTCGCTATTTATATATATATATATATTGCGCTCTCGTCTCCT 61 62 Output: 63 64 65 bioRxiv preprint doi: https://doi.org/10.1101/024448; this version posted August 21, 2015. The copyright holder for this preprint (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made available under aCC-BY 4.0 International license. Shelton and Brown Page 4 of 8 1 2 3 4 5 >my header 6 7 AAAAAAAAAAAATTTTTTCCCCGGCGCGCGCGCTATAGCGCTATANNNNNNNNNNNNNNN 8 ATATATATATATATTATTATATATATATTCTCTCTGGGCTCGCGTCTCGCTATTTATATA 9 TATATATATATTGCGCTCTCGTCTCCT 10 However, seqret does not log the detected errors in the format. Another feature of 11 12 Seqret is that an output file is created even if the output is identical to the input. 13 Storing two identical files is an inefficient use of disk space. Seqtk [8] is another 14 example of a tool that can automate FASTA reformatting but does not first check 15 original format or report format issues. 16 17 Another case to note is when an improperly formatted FASTA file is actually dis- 18 tributed as a component of a bioinformatics tool. Trimmomatic adapter sequences 19 [9], for example, are distributed versions of the proprietary Illumina sequencing 20 adapters but the FASTA files are missing final new lines. This can cause issues 21 downstream if a workflow includes common analysis techniques like FASTA file 22 23 concatenation. 24 The process of restarting analysis manually after wrapping a FASTA file may 25 only take minutes. The time consuming aspect of this interruption is the time it 26 takes the analyst to become available and the number of jobs this step must be 27 28 repeated for. Likewise, storage of one extra FASTA file is trivial unless the FASTA 29 file in question stores a whole genome in which case the burden can add up for a 30 bioinformatics core. Efficiency and automation are crucial as bioinformatic analysis 31 projects become more numerous and time consuming. Many tools can either detect a 32 33 format issue or repair a format issue. No existing tool was found that both validates 34 FASTA format and reformats automatically only where required for auserdefined 35 list of non-fatal FASTA format issues. 36 37 1Implementation 38 39 Fasta-O-Matic was designed to fit seamlessly into an analysis workflow. It detects 40 which format issues are actually present in the FASTA file and then only produces 41 a reformatted file if the current file violates the user defined format requirements. 42 43 1.1 Portability 44 45 Where possible Fasta-O-Matic was designed to be easy to distribute and use. Fasta- 46 O-Matic is distributed on GitHub under the MIT license to allow for easy access to 47 or customization of the code. The tool was also built and tested on both Python2.7 48 and Python3.3 to minimize incompatibility with existing linux environments.The 49 50 script generates complete help menus when called from the command line with 51 the --help command and from within python with help(fasta_o_matic).Ad- 52 ditionally, Fasta-O-Matic includes a sample FASTA file with missing new lines, 53 inconsistent wrapping and spaces in headers along with a tutorial which describes 54 how to reformat the sample. These features ensure that Fasta-O-Matic is easy to 55 56 incorporate into existing workflows. 57 58 1.2 Automate where appropriate 59 The script was designed to efficiently execute the most likely solution given the 60 presence or absence of format issues. Fasta-O-Matic returns a filename for the out- 61 62 put FASTA file that conforms to the user defined format. If the original file already 63 64 65 bioRxiv preprint doi: https://doi.org/10.1101/024448; this version posted August 21, 2015. The copyright holder for this preprint (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made available under aCC-BY 4.0 International license. Shelton and Brown Page 5 of 8 1 2 3 4 5 conforms, then Fasta-O-Matic returns the original filename rather than outputting 6 7 a redundant FASTA file under a new name. 8 Fasta-O-Matic will exit and report an error if the FASTA file cannot beread,the 9 default or defined output directory cannot be written to, the input FASTA file does 10 not begin with a > or if any sequence line includes a non-IUPAC character. The last 11 12 two errors are considered to be fatal FASTA format errors. 13 Inconsistent or unwrapped sequence lines, spaces in headers and missing or non- 14 standard new lines are considered non-fatal errors. Testing for these issues is op- 15 tional. If they are detected, the decision is made to reformat as requested, report 16 17 the issue to the analyst and continue the workflow. 18 Testing the uniqueness of the header/description line can return a non-fatal warn- 19 ing and a reformatted file or a fatal error. Testing for uniqueness is optional. If the 20 first word in each header/description line is unique then it follows that all descrip- 21 22 tion lines are unique. If the first words are not unique then it is possible that is 23 because the header ids include whitespace ‘>seq 1’or‘>seq1’. In this case a 24 resolution is to replace the whitespace with a character. Fasta-O-Matic replaces the 25 whitespace with an underscore and retests for the uniqueness of the first words in 26 27 the headers. If this version passes than the user is warned that whitespace effected 28 header uniqueness and was removed from headers. If removing whitespace also fails 29 to resolve the issue the lack of uniqueness is considered a fatal error. The fatal error 30 is reported and the program halts. 31 32 The script also automatically adjusts to run the minimal number of steps sufficient 33 to fix and report format issues. If it is included in the set of quality control (QC) 34 steps then wrapping is the first format issue tested because while repairing FASTA 35 wrapping both headers and new lines can be corrected. New lines are given priority 36 37 after wrapping because while repairing new lines it is also trivial to repair headers. 38 Next, uniqueness of the header lines is tested. Finally, headers are evaluated for 39 whitespace. If an early test returns a format issue and launches a reformatting that 40 automatically repairs any remaining format issues then Fasta-O-Matic still tests for 41 42 any additional format errors in the original file. 43 All format issues are reported in the programs logs in case they indicate an unex- 44 pected issue with the data. Logs can be optionally color coded so that red indicates 45 errors, yellow indicates warnings (e.g. a non-fatal issue was found and automati- 46 47 cally reformatted) and green indicates status information. This method of logging 48 is designed to draw the attention of the bioinformatics analyst to relevant warn- 49 ings or errors even if they have grown accustomed to seeing Fasta-O-Matic output 50 frequently. 51 52 53 1.3 Workflow integration 54 Sequence FASTA files are often passed as arguments to commandline tools. For 55 example FASTA files can be passed as an argument to bowtie2-build to be indexed 56 57 as an alignment reference [10] or passed to trimmomatic as adapters to detect 58 sequencing artifacts. The output filename used by Fast-O-Matic varies to reflect 59 the reformatting performed. For seamless integration into automated workflows 60 Fasta-O-Matic returns the full path of the new properly formatted FASTA file or 61 62 the original file (if it is already formatted properly). This can be captured as a 63 64 65 bioRxiv preprint doi: https://doi.org/10.1101/024448; this version posted August 21, 2015. The copyright holder for this preprint (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made available under aCC-BY 4.0 International license. Shelton and Brown Page 6 of 8 1 2 3 4 5 variable and used as an argument in subsequent commands. The Bashcommands 6 7 below show and example of capturing the FASTA file name as a variable. 8 Code (backslashes are used to indicate a new line that is for display in the article 9 rather than the new lines being included in the actual code): 10 filename="$(python fasta_o_matic.py -f NC_010473_mock_scaffolds.fna \ 11 12 -o ~/out_fasta_o_matic -c)" 13 echo $filename 14 15 2Results 16 17 2.1 Data 18 FASTA format tools were tested on the Vicugna pacos-2.0.1 whole genome shotgun 19 sequence scaffolds because the 2.17 Gb Vicugna pacos genome is large (> 1Gb)and 20 has many scaffolds (276727) [11]. The large genome size and high number of individ- 21 22 ual sequences should approximate a typical large FASTA file. The FASTA file was 23 downloaded from the National Center for Biotechnology Information (NCBI) FTP 24 as NW 005882702.1 Vicugna pacos isolate Carlotta (AHFN-0088) Vicugna pacos- 25 2.0.1 assembly scaffolds. An additional unwrapped sequence was added to the end 26 27 of the file. This sequence was also missing a new line. Each FASTA record in the 28 file also had spaces within the text of the headers. 29 The additional simulated FASTA record is available on GitHub. 30 31 32 2.2 Reformatting tests 33 No tool was found with all of Fasta-O-Matic’s functions. Therefore sequence line 34 wrapping was compared between Fasta-O-Matic and two other common reformat- 35 ting tools, seqtk and seqret. Fasta-O-Matic was run with the --qc_steps flag set 36 37 to either wrap new_line header_whitespace unique (all), wrap (W) new_line 38 (NL), unique (U) or header_whitespace (HW). Seqtk was run with the arguments 39 seq -l 60. Seqret was run using only the -sequence and -outseq arguments. Code 40 used in tests or to produce figures can be found on GitHub. Run time and max mem- 41 42 ory was reported for each tool. Tests were run on a Xeon Phi server with 48x12-core 43 Intel Xeon CPUs, 256GB of RAM, Linux CentOS 7 and Python2.7. 44 45 2.3 Comparison between results 46 47 All tools could reformat the improperly wrapped FASTA file. Fasta-O-Matic had 48 the lowest maximum memory requirements (Figure 1, Table 1). This maybeuseful 49 if working on a large genome on a local machine or cluster headnode where memory 50 usage is restricted. Fasta-O-Matic took several minutes rather than seconds (seqtk 51 52 and seqret took < 13 s) (Figure 2, Table 1). 53 Fully re-formatted simulated FASTA record (backslashes are used to indicate a 54 new line that is for display in the article rather than the new lines being included 55 in the actual FASTA record): 56 57 >NW_000000000.0 Vicugna pacos isolate Carlotta (AHFN-0088) FAKE genomic scaffold, \ 58 Vicugna_pacos-2.0.1 Scaffold-, whole genome shotgun sequence 59 ATACAACCATAAAGGTGCTATTCAGTCCATGGTTACAGGACATAACTACAACACACACCC 60 ACGTACACATGCGCATGCGCATGCACACACCCACGTACACGTACACGTACGCATACACAC 61 62 CCACGTACACGTACACGTACGCATACACACCCACGTACACGTACACGTACGCATACACAC 63 64 65 bioRxiv preprint doi: https://doi.org/10.1101/024448; this version posted August 21, 2015. The copyright holder for this preprint (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made available under aCC-BY 4.0 International license. Shelton and Brown Page 7 of 8 1 2 3 4 5 CCACGTACACGTACACGTACGCATACACACCCACGTACACGTACACGTACGCATACACAC 6 7 CCACGTACGCACACACGTACACGTGTAGGCACGCATTTAGCAAGTATTTAGCTTGCTTAA 8 ACAAACCCCCCCTACCCCCCACGAGCCCCACCTTATATACCAGACAGTCTTGCCAAACCC 9 CAAAAACAAGACATAGCGCATAAGCTATAGAACCCGGACAAACCTTTGCCCACAAACCCA 10 ACTTCTTAAATAATCACATGGCCAAATCGTACCAATGTGTTACTCTAGTATATTAAAAAT 11 12 ATACAGACAGCTATCTCCCTAGATCCGCCAAAATTTTTAAAACAGAATTCAACAACCTTT 13 TTAATGGCACCCCCCCCCCCCATAAATGACC 14 15 3Conclusions 16 17 Overall, both memory and run time requirements were small for all three programs. 18 However, the extra minutes taken by Fasta-O-Matic to test for fatal and non-fatal 19 format issues may prevent hours lost waiting for an analyst to manually restart 20 analysis or worse discover that a file was corrupt only after analysis is complete. 21 Fasta-O-Matic was also the only tool identified that skips reformatting if none is 22 23 required balancing the need to prepare data to be properly interpreted by bioin- 24 formatics tools with the practical need to conserve disk space. Fasta-O-Matic is a 25 portable and easy to use tool to facilitate bioinformatics analysis by automating 26 FASTA file inspection in busy bioinformatics cores. 27 28 29 4Availabilityandrequirements 30 Project name: Fasta-O-Matic tool 31 Project home page: The Fasta-O-Matic script and tutorial are available at 32 https://github.com/i5K-KINBRE-script-share/read-cleaning-format-conversion/tree/master/KSU_bioi 33 34 Operating system(s): Linux (tested on CentOS 7, Gentoo and Ubuntu). 35 Programming language: Python2.7+, Python3.3+ 36 License: Tool and tutorial are available free of charge to academic and non-profit 37 institutions. 38 39 Any restrictions to use by non-academics: Please contact authors for com- 40 mercial use. 41 Dependencies: Fasta-O-Matic requires the python modules Colorer and general 42 which are distributed in the same git repository. 43 44 45 Abbreviations 46 IUPAC - International Union of Pure and Applied Chemistry QC - quality control 47 NCBI - National Center for Biotechnology Information 48 W-wrap 49 NL - new line 50 HW - header whitespace 51 U-unique 52 Competing interests 53 The authors declare that they have no competing interests. 54 Author’s contributions 55 JMS wrote most of the code for Fasta-O-Matic. JMS and SJB did the writing. Both authors read and approved the 56 final manuscript. 57 Acknowledgements 58 Thanks to Sheldon McKay https://github.com/mckays630 for contributing to the editing of the Fast-O-Matic 59 program. This project was supported by an Institutional Development Award (IDeA) from the National Institute of General 60 Medical Sciences of the National Institutes of Health under grant number P20 GM103418. The content is solely the 61 responsibility of the authors and does not necessarily represent the official views of the National Institute of General 62 Medical Sciences or the National Institutes of Health. 63 64 65 bioRxiv preprint doi: https://doi.org/10.1101/024448; this version posted August 21, 2015. The copyright holder for this preprint (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made available under aCC-BY 4.0 International license. Shelton and Brown Page 8 of 8 1 2 3 4 5 Author details 6 1KSU/K-INBRE Bioinformatics Center, Division of Biology, Kansas State University, Manhattan, KS, USA. 7 2BioNano Genomics, San Diego, CA, USA. 8 References 9 1. Jaffe, D., et al.: The FASTG Format Specification (v1.00). http (2012) 10 2. NCBI: Accepted input formats (Accessed: 2015-08-06). 11 http://blast.ncbi.nlm.nih.gov/blastcgihelp.shtml 12 3. Comm, I.-I.: Abbreviations and symbols for nucleic acids,polynucleotides,andtheirconstituents.Biochemistry 9(20), 4022–4027 (1970) 13 4. Stajich, J.E., Block, D., Boulez, K., Brenner, S.E., Chervitz, S.A., Dagdigian, C., Fuellen, G., Gilbert, J.G., 14 Korf, I., Lapp, H., Lehvaslaiho, H., Matsalla, C., Mungall, C.J., Osborne, B.I., Pocock, M.R., Schattner, P., 15 Senger, M., Stein, L.D., Stupka, E., Wilkinson, M.D., Birney, E.: The Bioperl toolkit: Perl modules for the life 16 sciences. Genome Res 12(10), 1611–8 (2002) 5. Stein, L.D., Mungall, C., Shu, S., Caudy, M., Mangone, M., Day, A., Nickerson, E., Stajich, J.E., Harris, T.W., 17 Arva, A., Lewis, S.: The generic genome browser: a building block for a model organism system database. 18 Genome Res 12(10), 1599–610 (2002) 19 6. Stajich, J.E., Hahn, M.W.: Disentangling the effects of demography and selection in human history. Molecular 20 biology and evolution 22(1), 63–73 (2005). doi:10.1093/molbev/msh252 7. Rice, P., Longden, I., Bleasby, A.: EMBOSS: the European Molecular Biology Open Software Suite. Trends in 21 genetics : TIG 16(6), 276–277 (2000). doi:10.1016/s0168-9525(00)02024-2 22 8. Li, H., seqtk GitHub repository, 4feb6e81444ab6bc44139dd3a125068f81ae4ad8. 23 https://github.com/lh3/seqtk 24 9. Bolger, A.M., Lohse, M., Usadel, B.: Trimmomatic: a flexible trimmer for Illumina sequence data. Bioinformatics, 170 (2014) 25 10. Langmead, B., Salzberg, S.L.: Fast gapped-read alignment with Bowtie 2. Nature methods 9(4), 357–359 26 (2012) 27 11. Lindblad-Toh, K., Garber, M., Zuk, O., Lin, M.F., Parker,B.J.,Washietl,S.,Kheradpour,P.,Ernst,J.,Jordan, G., Mauceli, E., Ward, L.D., Lowe, C.B., Holloway, A.K., Clamp, M., Gnerre, S., Alf¨oldi, J., Beal, K., Chang, 28 J., Clawson, H., Cuff, J., Palma, F.D., Fitzgerald, S., Flicek, P., Guttman, M., Hubisz, M.J., Jaffe, D.B., 29 Jungreis, I., Kent, W.J., Kostka, D., Lara, M., Martins, A.L., Massingham, T., Moltke, I., Raney, B.J., 30 Rasmussen, M.D., Robinson, J., Stark, A., Vilella, A.J., Wen, J., Xie, X., Zody, M.C., Baldwin, J., Bloom, T., 31 Chin, C.W., Heiman, D., Nicol, R., Nusbaum, C., Young, S., Wilkinson, J., Worley, K.C., Kovar, C.L., Muzny, D.M., Gibbs, R.A., Cree, A., Dihn, H.H., Fowler, G., Jhangiani, S., Joshi, V., Lee, S., Lewis, L.R., Nazareth, 32 L.V., Okwuonu, G., Santibanez, J., Warren, W.C., Mardis, E.R., Weinstock, G.M., Wilson, R.K., Delehaunty, 33 K., Dooling, D., Fronik, C., Fulton, L., Fulton, B., Graves, T., Minx, P., Sodergren, E., Birney, E., Margulies, 34 E.H., Herrero, J., Green, E.D., Haussler, D., Siepel, A., Goldman, N., Pollard, K.S., Pedersen, J.S., Lander, 35 E.S., Kellis, M.: A high-resolution map of human evolutionary constraint using 29 mammals. Nature 478(7370), 476–482 (2011). doi:10.1038/nature10530 36 37 Figures 38 39 40 Figure 1 Max memory used by various FASTA tools. Tools were run on the Vicugna pacos isolate Carlotta (AHFN-0088) Vicugna pacos-2.0.1 whole genome shotgun sequence 41 NW 005882702.1 with additional unwrapped FASTA sequence record. 42 43 44 45 46 Figure 2 Run time for various FASTA tools. Tools were run on the Vicugna pacos isolate Carlotta (AHFN-0088) Vicugna pacos-2.0.1 whole genome shotgun sequence NW 005882702.1 47 with additional unwrapped FASTA sequence record. 48 49 50 Tables 51 52 Table 1 Runtime and max memory used by various FASTA tools. Tools wererunontheVicugna pacos isolate Carlotta (AHFN-0088) Vicugna pacos-2.0.1 whole genome shotgun sequence 53 NW 005882702.1. 54 Program Max mem (kbytes) Run time (s) 55 Fasta-O-Matic(HW) 7084 112.02 56 Fasta-O-Matic(NL) 7084 93.21 57 Fasta-O-Matic(W) 10996 141.41 Fasta-O-Matic(U) 30800 105.65 58 seqtk 38352 3.17 59 Fasta-O-Matic(all) 69452 162.86 60 seqret 137840 12.50 61 62 63 64 65 bioRxiv preprint doi: https://doi.org/10.1101/024448; this version posted August 21, 2015. The copyright holder for this preprint (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made available under aCC-BY 4.0 International license.

Figure_1_memory Click here to download Figure: Fig_1_memory.pdf Max memory (kbytes) 0 40000 80000 140000

Fasta−O−Matic (HW) Fasta−O−Matic (NL) Fasta−O−Matic (W) Fasta−O−Matic (U) seqtk Fasta−O−Matic (all) seqret

FASTA file tool bioRxiv preprint doi: https://doi.org/10.1101/024448; this version posted August 21, 2015. The copyright holder for this preprint (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made available under aCC-BY 4.0 International license.

Figure_2_run_time Click here to download Figure: Fig_2_run_time.pdf Run time (s) 0 20 60 100 140

Fasta−O−Matic (HW) Fasta−O−Matic (NL) Fasta−O−Matic (W) Fasta−O−Matic (U) seqtk Fasta−O−Matic (all) seqret

FASTA file tool