DOCSLIB.ORG

A Virtual Architecture for Durable Compressed Archives

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
A Virtual Architecture for Durable Compressed Archives VXA: A Virtual Architecture for Durable Compressed Archives Bryan Ford Computer Science and Artificial Intelligence Laboratory Massachusetts Institute of Technology http://pdos.csail.mit.edu/~baford/vxa/ The Ubiquity of Data Compression Everything is compressed these days – Archive/Backup/Distribution: ZIP, tar.gz, ... – Multimedia streams: mp3, ogg, wmv, ... – Office documents: XML-in-ZIP – Digital cameras: JPEG, proprietary RAW, ... – Video camcorders: DV, MPEG-2, ... Compressed Data Formats Observation #1: Data compression formats evolve rapidly s es pr rc 2 C O a p R p Q U om R O H IP zi A zi z S L c A Z L Z g R b 7 Ð Ð Ð Ð Ð Ð Ð Ð Ð Ð Ð Lossless Compression 1980 1985 1990 1995 2000 2005 O b C s D e o rv m a a ta t p c io r o n e L m o s ss # le p 1 s ss r e C e : o Ð SQ d Im m s p Ð LU s ag r e e io D E ss n io n c n Ð compress a V odi f id n Ð ARC o t eo g r a E Ð ZOO m nc o F A d Ð BMP a ud in Ð LHarc t io g Ð ILBM s o E Ð TIFF Ð ZIP n e r c v 1 o Ð GIF m 9 di o 8 0 ng Ð PCX lv Ð TGA Ð gzip e a r ts Ð 8SVX Ð ANIM Ð RAR ap 1 Ð FLIC i 9 d 85 Ð bzip2 l Ð AIFF Ð QuickTime Ð JPEG y Ð PNG Ð AIFF-C Ð MPEG-1 Ð 7z Ð WAV 1 Ð MPEG-2 9 9 0 Ð DV Ð MP3 Ð JPEG 2000 Ð RealAudio Ð MPEG-4 19 Ð Sorenson 9 5 Ð AAC Ð WMV7 Ð WMV8 Ð WMA7 Ð WMV9 20 Ð FLAC 0 0 Ð Vorbis Ð WMA9 2 0 0 5 Compressed Data Formats Observation #1: Data compression formats evolve rapidly Problems: – Inconvenient: each new algorithm requires decoder install/upgrade – Impedes data portability: data unusable on systems without supported decoder – Threatens long-term data usability: old decoders may not run on new operating systems Archiving Compressed Data Observation #2: Processor architectures evolve more conservatively r) r) o to ) ct c 4 ) 6 it X e e -6 it 6 8 b v E 6 b 3 - M t S v 8 - 08 0 2 n S P x 4 8 8 (3 M (i (F (6 Ð Ð Ð Ð Ð x86 Architecture 1980 1985 1990 1995 2000 2005 Archiving Compressed Data Observation #2: Processor architectures evolve more conservatively Fully Backward Compatible Extensions r) r) o to ) ct c 4 ) 6 it X e e -6 it 6 8 b v E 6 b 3 - M t S v 8 - 08 0 2 n S P x 4 8 8 (3 M (i (F (6 Ð Ð Ð Ð Ð x86 Architecture 1980 1985 1990 1995 2000 2005 Archiving Compressed Data Observation #2: Processor architectures evolve more conservatively r) r) o to ) ct c 4 ) 6 it X e e -6 it 6 8 b v E 6 b 3 - M t S v 8 - 08 0 2 n S P x 4 8 8 (3 M (i (F (6 Ð Ð Ð Ð Ð a x86 Architecture h C C lp S P 0 C I r A m 0 S R M R e iu 0 P A - w C 8 I P R A o E tan 6 M S A P P D I Ð Ð Ð Ð Ð Ð Ð Ð Other Architectures 1980 1985 1990 1995 2000 2005 Archiving Compressed Data Observation #2: Processor architectures evolve more conservatively r) r) o to ) ct c 4 ) 6 it X e e -6 it 6 8 b v E 6 b 3 - M t S v 8 - 08 0 2 n S P x 4 8 8 (3 M (i (F (6 Ð Ð Ð Ð Ð a x86 Architecture h C C lp S P 0 C I r A m 0 S R M R e iu 0 P A - w C 8 I P R A o E tan 6 M S A P P D I Ð Ð Ð Ð Ð Ð Ð Ð Other Architectures 1980 1985 1990 1995 2000 2005 Archiving Compressed Data Observation #2: Processor architectures evolve more conservatively r) r) o to ) ct c 4 ) 6 it X e e -6 it 6 8 b v E 6 b 3 - M t S v 8 - 08 0 2 n S P x 4 8 8 (3 M (i (F (6 Ð Ð Ð Ð Ð a x86 Architecture h C C lp C S P 0 I r A m 0 S R M -R e C iu 0 IP A w 8 P R A o E tan 6 M S A P P D I Ð Ð Ð Ð Ð Ð Ð Ð Other Architectures 1980 1985 1990 1995 2000 2005 Archiving Compressed Data Observation #2: Processor architectures evolve more conservatively r) r) o to ) ct c 4 ) 6 it X e e -6 it 6 8 b v E 6 b 3 - M t S v 8 - 08 0 2 n S P x 4 8 8 (3 M (i (F (6 Ð Ð Ð Ð Ð a x86 Architecture h C C lp C S P 0 I r A m 0 S R M R e iu 0 P A - w C 8 I P R A o E tan 6 M S A P P D I Ð Ð Ð Ð Ð Ð Ð Ð Other Architectures 1980 1985 1990 1995 2000 2005 Archiving Compressed Data Observation #2: Processor architectures evolve more conservatively r) r) o to ) ct c 4 ) 6 it X e e -6 it 6 8 b v E 6 b 3 - M t S v 8 - 08 0 2 n S P x 4 8 8 (3 M (i (F (6 Ð Ð Ð Ð Ð a x86 Architecture h C C lp C S P 0 I r A m 0 S R M R e iu 0 P A - w C 8 I P R A o E tan 6 M S A P P D I Ð Ð Ð Ð Ð Ð Ð Ð Other Architectures 1980 1985 1990 1995 2000 2005 ic n a t I VXA: Virtual Executable Archives Observation 1+2: Instruction formats are historically more durable than compressed data formats Make archive self-extracting (data + executable decoder) To extract data, archive reader runs embedded decoder Archive Archive Archive Writer Reader Encoder Decoder D D Goals of VXA Make self-extracting archives... Archive Archive Archive Writer Reader Encoder Decoder D D Goals of VXA Make self-extracting archives... 1. Safe: malicious decoders can©t compromise host 2. Future-proof: simple, well-defined architecture [Lorie] Archive Archive Archive Writer Reader Encoder Emulator Decoder D D Goals of VXA Make self-extracting archives... 1. Safe: malicious decoders can©t compromise host 2. Future-proof: simple, well-defined architecture [Lorie] 3. Easy: allow reuse of existing code, languages, tools Archive Archive Archive Writer Reader Encoder x86 Emulator Decoder D D Goals of VXA Make self-extracting archives... 1. Safe: malicious decoders can©t compromise host 2. Future-proof: simple, well-defined architecture [Lorie] 3. Easy: allow reuse of existing code, languages, tools 4. Efficient: practical for short term data packaging too Archive Archive Archive Writer Reader Encoder Fast x86 Emulator Decoder D D Outline ● Archiver Operation ● vxZIP Archive Format ● Decoder Architecture ● Emulator Design & Implementation ● Evaluation (performance, storage overhead) ● Conclusion Archive Writer Operation VXA Archiver Archive Archive Writer Operation Uncompressed Input Files VXA Archiver General Compressor Decoder1 D1 Archive Archive Writer Operation Uncompressed Input Files VXA Archiver General Compressor Decoder1 D1 Archive Archive Writer Operation Uncompressed Input Files General Image Audio Compressor Compressor Compressor Decoder1 Decoder2 Decoder3 D1 D2 D3 Archive Archive Writer Operation Uncompressed Input Files Pre-Compressed Input Files General Image Audio Compressor Compressor Compressor Decoder1 Decoder2 Decoder3 D1 D2 D3 Archive Archive Writer Operation Uncompressed Input Files Pre-Compressed Input Files General Image Audio Image Format Audio Format Compressor Compressor Compressor Recognizer Recognizer Decoder1 Decoder2 Decoder3 Decoder4 Decoder5 D1 D2 D3 D4 D5 Archive Archive Reader Operation VXA Archive Reader x86 Emulator D1 D2 D3 D4 D5 Archive Archive Reader Operation Original Uncompressed Files VXA Archive Reader x86 Emulator Decoder1 D1 D2 D3 D4 D5 Archive Archive Reader Operation Original Uncompressed Files VXA Archive Reader x86 Emulator Decoder1 Decoder2 Decoder3 D1 D2 D3 D4 D5 Archive Archive Reader Operation Original Uncompressed Files Original Pre-Compressed Files VXA Archive Reader x86 Emulator Decoder1 Decoder2 Decoder3 D1 D2 D3 D4 D5 Archive Archive Reader Operation Original Uncompressed Files De-compressed Files VXA Archive Reader x86 Emulator Decoder1 Decoder2 Decoder3 Decoder4 Decoder5 D1 D2 D3 D4 D5 Archive vxZIP Archive Format ● Backward compatible with legacy ZIP format Image file Audio file Audio file Central Directory vxZIP Archive vxZIP Archive Format ● Backward compatible JP2 Decoder with legacy ZIP format Image file ● Decoders intermixed FLAC Decoder with archived files Audio file Audio file Central Directory vxZIP Archive vxZIP Archive Format ● Backward compatible JP2 Decoder with legacy ZIP format Image file (JP2-encoded) ● Decoders intermixed FLAC Decoder with archived files Audio file ● Archived files have (FLAC-encoded) Audio file new extension header (FLAC-encoded) pointing to decoder Central Directory vxZIP Archive vxZIP Archive Format ● Backward compatible JP2 Decoder (deflated) with legacy ZIP format Image file (JP2-encoded) ● Decoders intermixed FLAC Decoder with archived files (deflated) Audio file ● Archived files have (FLAC-encoded) Audio file new extension header (FLAC-encoded) pointing to decoder Central Directory ● Decoders are hidden, vxZIP Archive ªdeflatedº (gzip) vxZIP Decoder Architecture ● Decoders are ELF executables for x86-32 – Can be written in any language, safe or unsafe – Compiled using ordinary tools (GCC) ● Decoders have access to five ªsystem callsº: – read stdin, write stdout, malloc, next file, exit ● Decoders cannot: – open files, windows, devices, network connections, ..
Load more

Recommended publications
  • Leveraging Activation Sparsity for Training Deep Neural Networks
    Compressing DMA Engine: Leveraging Activation Sparsity for Training Deep Neural Networks Minsoo Rhu Mike O’Connor Niladrish Chatterjee Jeff Pool Stephen W. Keckler NVIDIA Santa Clara, CA 95050 fmrhu, moconnor, nchatterjee, jpool, [email protected] Abstract—Popular deep learning frameworks require users to the time needed to copy data back and forth through PCIe is fine-tune their memory usage so that the training data of a deep smaller than the time the GPU spends computing the DNN neural network (DNN) fits within the GPU physical memory. forward and backward propagation operations, vDNN does Prior work tries to address this restriction by virtualizing the memory usage of DNNs, enabling both CPU and GPU memory not affect performance. For networks whose memory copying to be utilized for memory allocations. Despite its merits, vir- operation is bottlenecked by the data transfer bandwidth of tualizing memory can incur significant performance overheads PCIe however, vDNN can incur significant overheads with an when the time needed to copy data back and forth from CPU average 31% performance loss (worst case 52%, Section III). memory is higher than the latency to perform the computations The trend in deep learning is to employ larger and deeper required for DNN forward and backward propagation. We introduce a high-performance virtualization strategy based on networks that leads to large memory footprints that oversub- a “compressing DMA engine” (cDMA) that drastically reduces scribe GPU memory [13, 14, 15]. Therefore, ensuring the the size of the data structures that are targeted for CPU-side performance scalability of the virtualization features offered allocations.
    [Show full text]
  • Genetic Databases
    Stefano Lonardi March, 2000 Compression of Biological Sequences by Greedy Off-line Textual Substitution Alberto Apostolico Stefano Lonardi Purdue University Università di Padova Genetic Databases § Massive § Growing exponentially Example: GenBank contains approximately 4,654,000,000 bases in 5,355,000 sequence records as of December 1999 Data Compression Conference 2000 1 Stefano Lonardi March, 2000 DNA Sequence Records Composed by annotations (in English) and DNA bases (on the alphabet {A,C,G,T,U,M,R,W,S,Y,K,V,H,D,B,X,N}) >RTS2 RTS2 upstream sequence, from -200 to -1 TCTGTTATAGTACATATTATAGTACACCAATGTAAATCTGGTCCGGGTTACACAACACTT TGTCCTGTACTTTGAAAACTGGAAAAACTCCGCTAGTTGAAATTAATATCAAATGGAAAA GTCAGTATCATCATTCTTTTCTTGACAAGTCCTAAAAAGAGCGAAAACACAGGGTTGTTT GATTGTAGAAAATCACAGCG >MEK1 MEK1 upstream sequence, from -200 to -1 TTCCAATCATAAAGCATACCGTGGTYATTTAGCCGGGGAAAAGAAGAATGATGGCGGCTA AATTTCGGCGGCTATTTCATTCATTCAAGTATAAAAGGGAGAGGTTTGACTAATTTTTTA CTTGAGCTCCTTCTGGAGTGCTCTTGTACGTTTCAAATTTTATTAAGGACCAAATATACA ACAGAAAGAAGAAGAGCGGA >NDJ1 NDJ1 upstream sequence, from -200 to -1 ATAAAATCACTAAGACTAGCAACCACGTTTTGTTTTGTAGTTGAGAGTAATAGTTACAAA TGGAAGATATATATCCGTTTCGTACTCAGTGACGTACCGGGCGTAGAAGTTGGGCGGCTA TTTGACAGATATATCAAAAATATTGTCATGAACTATACCATATACAACTTAGGATAAAA ATACAGGTAGAAAAACTATA Problem Textual compression of DNA data is difficult, i.e., “standard” methods do not seem to exploit the redundancies (if any) inherent to DNA sequences cfr. C.Nevill-Manning, I.H.Witten, “Protein is incompressible”, DCC99 Data Compression Conference 2000 2 Stefano Lonardi March, 2000
    [Show full text]
  • Datasheet-Smartsignage
    Fanless Embedded Box PC with Intel® Celeron® SmartSignage 4-core Processor J1900 EBC-3311 Features Fanless digital signage solution Cableless design Low power consumption 1 x USB 3.0 Line-out 1 x USB 2.0 2 x RJ-45 VGA DC INPUT 12V On board Intel® Celeron® 4-core Processor EBC-3311B High definition video output J1900 (Bay Trail) Supports DDR3L SO-DIMM 1333 max. up to 8GB 1 x High definition video output & 1 VGA or 2 x High definition video output 1 x USB 3.0 Line-out 1 x USB 2.0 2 x RJ-45 DC INPUT 12V Front Side 1 mSATA supported High definition video output Supports VESA mount Rear Side Power Switch Applications Digital Signage Kiosk Engine POS PC IoT Gateway Specifications Image format Jpeg, tiff, png, (anim) gif, bmp System Optional for 802.11 b/g/n CPU Intel® Celeron® 4-core Processor J1900 WiFi (Bay Trail) Expansion Slot 1 x full-size Mini card Chipset SoC integrated (for mSATA) System Memory 1 x DDR3L 1333 max up to 8GB 1 x half-size PCI Express Mini card Storage 1 x mSATA supported for optional WiFi module Watchdog Timer 255 levels, 1-255 sec. Power Supply 12V DC-in I/O 1 x USB 2.0 , 1 x USB 3.0 2 x 10/100/1000Mbps Ethernet Dimensions (W x D x H) 170 x 120 x 40 mm 1 x power on/off button 6.69 x 4.72 x 1.57” 1 x Audio (Line-out) Packing Dimension 320 x 205 x 65 mm Video I/O EBC-3311 EBC-3311B (W x D x H) 12.59 x 8.07 x 2.55” 1 x High 2 x High definition Weight (net/gross) 0.7kg(1.5lb)/1.2kg(2.61b) definition video output video output Environmental 1 x VGA Operation Temperature 0°C – +40°C,(32°F – 104°F) Video MPEG-4, MPEG-2, MPEG-1, H.264,
    [Show full text]
  • ARC-LEAP User Instructions for The
    ARC-LEAP User Instructions Appalachian Regional Commission Local Economic Assessment Package prepared for the Appalachian Regional Commission prepared by Economic Development Research Group, Inc. Glen Weisbrod Teresa Lynch Margaret Collins January, 2004 ARC-LEAP User Instructions Appalachian Regional Commission Local Economic Assessment Package prepared for the Appalachian Regional Commission prepared by Economic Development Research Group, Inc. 2 Oliver Street, 9th Floor, Boston, MA 02109 Telephone 617.338.6775 Fax 617.338.1174 e-mail [email protected] Website www.edrgroup.com January, 2004 ARC-LEAP User Instructions PREFACE LEAP is a software tool that was designed and developed by Economic Development Research Group, Inc. (www.edrgroup.com) to assist practitioners in evaluating local economic development needs and opportunities. ARC-LEAP is a version of this tool developed specifically for the Appalachian Regional Commission (ARC) and it’s Local Development Districts (LDDs). Development of this user guide was funded by ARC as a companion to the ARC-LEAP analysis system. This document presents user instructions and technical documentation for ARC-LEAP. It is organized into three parts: I. overview of the ARC-LEAP tool II. instructions for users to obtain input information and run the analysis model III. interpretation of output tables . A separate Handbook document provides more detailed discussion of the economic development assessment process, including analysis of local economic performance, diagnosis of local strengths and weaknesses, and application of business opportunity information for developing an economic development strategy. Economic Development Research Group i ARC-LEAP User Instructions I. OVERVIEW The ARC-LEAP model serves to three related purposes, each aimed at helping practitioners identify target industries for economic development.
    [Show full text]
  • NXP Eiq Machine Learning Software Development Environment For
    AN12580 NXP eIQ™ Machine Learning Software Development Environment for QorIQ Layerscape Applications Processors Rev. 3 — 12/2020 Application Note Contents 1 Introduction 1 Introduction......................................1 Machine Learning (ML) is a computer science domain that has its roots in 2 NXP eIQ software introduction........1 3 Building eIQ Components............... 2 the 1960s. ML provides algorithms capable of finding patterns and rules in 4 OpenCV getting started guide.........3 data. ML is a category of algorithm that allows software applications to become 5 Arm Compute Library getting started more accurate in predicting outcomes without being explicitly programmed. guide................................................7 The basic premise of ML is to build algorithms that can receive input data and 6 TensorFlow Lite getting started guide use statistical analysis to predict an output while updating output as new data ........................................................ 8 becomes available. 7 Arm NN getting started guide........10 8 ONNX Runtime getting started guide In 2010, the so-called deep learning started. It is a fast-growing subdomain ...................................................... 16 of ML, based on Neural Networks (NN). Inspired by the human brain, 9 PyTorch getting started guide....... 17 deep learning achieved state-of-the-art results in various tasks; for example, A Revision history.............................17 Computer Vision (CV) and Natural Language Processing (NLP). Neural networks are capable of learning complex patterns from millions of examples. A huge adaptation is expected in the embedded world, where NXP is the leader. NXP created eIQ machine learning software for QorIQ Layerscape applications processors, a set of ML tools which allows developing and deploying ML applications on the QorIQ Layerscape family of devices.
    [Show full text]
  • Oak Park Area Visitor Guide
    OAK PARK AREA VISITOR GUIDE COMMUNITIES Bellwood Berkeley Broadview Brookfield Elmwood Park Forest Park Franklin Park Hillside Maywood Melrose Park Northlake North Riverside Oak Park River Forest River Grove Riverside Schiller Park Westchester www.visitoakpark.comvisitoakpark.com | 1 OAK PARK AREA VISITORS GUIDE Table of Contents WELCOME TO THE OAK PARK AREA ..................................... 4 COMMUNITIES ....................................................................... 6 5 WAYS TO EXPERIENCE THE OAK PARK AREA ..................... 8 BEST BETS FOR EVERY SEASON ........................................... 13 OAK PARK’S BUSINESS DISTRICTS ........................................ 15 ATTRACTIONS ...................................................................... 16 ACCOMMODATIONS ............................................................ 20 EATING & DRINKING ............................................................ 22 SHOPPING ............................................................................ 34 ARTS & CULTURE .................................................................. 36 EVENT SPACES & FACILITIES ................................................ 39 LOCAL RESOURCES .............................................................. 41 TRANSPORTATION ............................................................... 46 ADVERTISER INDEX .............................................................. 47 SPRING/SUMMER 2018 EDITION Compiled & Edited By: Kevin Kilbride & Valerie Revelle Medina Visit Oak Park
    [Show full text]
  • Dynamic Resource Management of Network-On-Chip Platforms for Multi-Stream Video Processing
    DYNAMIC RESOURCE MANAGEMENT OF NETWORK-ON-CHIP PLATFORMS FOR MULTI-STREAM VIDEO PROCESSING Hashan Roshantha Mendis Doctor of Engineering University of York Computer Science March 2017 2 Abstract This thesis considers resource management in the context of parallel multiple video stream de- coding, on multicore/many-core platforms. Such platforms have tens or hundreds of on-chip processing elements which are connected via a Network-on-Chip (NoC). Inefficient task allo- cation configurations can negatively affect the communication cost and resource contention in the platform, leading to predictability and performance issues. Efficient resource management for large-scale complex workloads is considered a challenging research problem; especially when applications such as video streaming and decoding have dynamic and unpredictable workload characteristics. For these type of applications, runtime heuristic-based task mapping techniques are required. As the application and platform size increase, decentralised resource management techniques are more desirable to overcome the reliability and performance bot- tlenecks in centralised management. In this work, several heuristic-based runtime resource management techniques, targeting real-time video decoding workloads are proposed. Firstly, two admission control approaches are proposed; one fully deterministic and highly predictable; the other is heuristic-based, which balances predictability and performance. Secondly, a pair of runtime task mapping schemes are presented, which make use of limited known application properties, communication cost and blocking-aware heuristics. Combined with the proposed deterministic admission con- troller, these techniques can provide strict timing guarantees for hard real-time streams whilst improving resource usage. The third contribution in this thesis is a distributed, bio-inspired, low-overhead, task re-allocation technique, which is used to further improve the timeliness and workload distribution of admitted soft real-time streams.
    [Show full text]
  • Chapter 3 Image Data Representations
    Chapter 3 Image Data Representations 1 IT342 Fundamentals of Multimedia, Chapter 3 Outline • Image data types: • Binary image (1-bit image) • Gray-level image (8-bit image) • Color image (8-bit and 24-bit images) • Image file formats: • GIF, JPEG, PNG, TIFF, EXIF 2 IT342 Fundamentals of Multimedia, Chapter 3 Graphics and Image Data Types • The number of file formats used in multimedia continues to proliferate. For example, Table 3.1 shows a list of some file formats used in the popular product Macromedia Director. Table 3.1: Macromedia Director File Formats File Import File Export Native Image Palette Sound Video Anim. Image Video .BMP, .DIB, .PAL .AIFF .AVI .DIR .BMP .AVI .DIR .GIF, .JPG, .ACT .AU .MOV .FLA .MOV .DXR .PICT, .PNG, .MP3 .FLC .EXE .PNT, .PSD, .WAV .FLI .TGA, .TIFF, .GIF .WMF .PPT 3 IT342 Fundamentals of Multimedia, Chapter 3 1-bit Image - binary image • Each pixel is stored as a single bit (0 or 1), so also referred to as binary image. • Such an image is also called a 1-bit monochrome image since it contains no color. It is satisfactory for pictures containing only simple graphics and text. • A 640X480 monochrome image requires 38.4 kilobytes of storage (= 640x480 / (8x1000)). 4 IT342 Fundamentals of Multimedia, Chapter 3 8-bit Gray-level Images • Each pixel has a gray-value between 0 and 255. Each pixel is represented by a single byte; e.g., a dark pixel might have a value of 10, and a bright one might be 230. • Bitmap: The two-dimensional array of pixel values that represents the graphics/image data.
    [Show full text]
  • Sequence Alignment/Map Format Specification
    Sequence Alignment/Map Format Specification The SAM/BAM Format Specification Working Group 3 Jun 2021 The master version of this document can be found at https://github.com/samtools/hts-specs. This printing is version 53752fa from that repository, last modified on the date shown above. 1 The SAM Format Specification SAM stands for Sequence Alignment/Map format. It is a TAB-delimited text format consisting of a header section, which is optional, and an alignment section. If present, the header must be prior to the alignments. Header lines start with `@', while alignment lines do not. Each alignment line has 11 mandatory fields for essential alignment information such as mapping position, and variable number of optional fields for flexible or aligner specific information. This specification is for version 1.6 of the SAM and BAM formats. Each SAM and BAMfilemay optionally specify the version being used via the @HD VN tag. For full version history see Appendix B. Unless explicitly specified elsewhere, all fields are encoded using 7-bit US-ASCII 1 in using the POSIX / C locale. Regular expressions listed use the POSIX / IEEE Std 1003.1 extended syntax. 1.1 An example Suppose we have the following alignment with bases in lowercase clipped from the alignment. Read r001/1 and r001/2 constitute a read pair; r003 is a chimeric read; r004 represents a split alignment. Coor 12345678901234 5678901234567890123456789012345 ref AGCATGTTAGATAA**GATAGCTGTGCTAGTAGGCAGTCAGCGCCAT +r001/1 TTAGATAAAGGATA*CTG +r002 aaaAGATAA*GGATA +r003 gcctaAGCTAA +r004 ATAGCT..............TCAGC -r003 ttagctTAGGC -r001/2 CAGCGGCAT The corresponding SAM format is:2 1Charset ANSI X3.4-1968 as defined in RFC1345.
    [Show full text]
  • Official Service Contractor for The2018 International Bluegrass Music Association
    International Bluegrass Musical Association OFFICIAL SERVICE September 26 - 29, 2018 CONTRACTOR Raleigh Convention Center Hall C Raleigh, NC Information and Order Forms Table of Contents General Information General Information................................................................2, 3 Payment Policy & Credit Card Authorization.........................4 Third Party Billing & Credit Card Charge Authorization..5,6 Color Chart for Drape, Table Skirts and Carpet...................7 Decorating Services Custom Booth Packages...........................................................8 Furnishing Rentals.......................................................................9 Mailing Address: Custom Signs and Graphics...................................................10 P. O. Box 49837 Greensboro, NC 27419 Labor Cleaning Service.........................................................................11 Street Address: Installation and Dismantle Labor............................................12 121 North Chimney Rock Road Exhibitor Appointed Contractor.........................................13,14 Greensboro, NC 27409 Material Handling Phone: (336) 315-5225 Material Handling General Information............................15,16 Fax: (336) 315-5220 Material Handling Rate Schedule and Order Form.......17,18 Shipping Labels........................................................................19 2 General 2 Information HOLLINS Exposition Services is pleased to have been selected as the Official Service Contractor for the2018 International
    [Show full text]
  • Galileo Formats
    Galileo Formats October 1998 edition Chapters INDEX Introduction Booking File Air Transportation Fares Cars Hotels LeisureShopper Document Production Queues Client File/TravelScreen Travel Information Miscellaneous SECURITY Sign On H/SON SON/Z217 or Sign on at own office SON/ followed by Z and a 1 to 3 character I.D.; the I.D. can be SON/ZHA initials, a number or a combination of both SON/ZGL4HA Sign on at branch agency SON/ followed by Z, own pseudo city code and a 1 to 3 character I.D. SON/Z7XX1/UMP Sign on at 4 character PCC branch agency SON/ followed by Z, own pseudo city code, second delimiter and 1 to 3 character I.D. SB Change to work area B SA/TA Change to work area A; different duty code TA (Training) SAI/ZHA Sign back into all work areas at own office SAI/ZGL4HA Sign back into all work areas at branch agency; SAI/ followed by Z, own pseudo city code and a 1 to 3 character I.D. Sign Off SAO Temporary sign out; incomplete Booking Files must be ignored or completed SOF Sign off; incomplete Booking Files must be ignored or completed SOF/ZHA Sign off override (at own office); incomplete transactions are not protected SOF/ZGL4HA Sign off override (at branch agency); incomplete transactions are not protected; SOF/ followed by Z, own pseudo city code and a 1 to 3 character I.D. SECURITY Security Profile STD/ZHA Display security profile, for sign on HA; once displayed, password may be changed SDA List security profiles created by user (second level authoriser and above) SDA/ZXXØ List security profiles associated with agency XXØ (second level authoriser and above) STD/ZXX1UMP or Display profile STD/ followed by Z, own pseudo city code, second delimiter if pseudo STD/Z7XX1/UMP city code is 4 characters and 1 to 3 character I.D.
    [Show full text]
  • Archive and Compressed [Edit]
    Archive and compressed [edit] Main article: List of archive formats • .?Q? – files compressed by the SQ program • 7z – 7-Zip compressed file • AAC – Advanced Audio Coding • ace – ACE compressed file • ALZ – ALZip compressed file • APK – Applications installable on Android • AT3 – Sony's UMD Data compression • .bke – BackupEarth.com Data compression • ARC • ARJ – ARJ compressed file • BA – Scifer Archive (.ba), Scifer External Archive Type • big – Special file compression format used by Electronic Arts for compressing the data for many of EA's games • BIK (.bik) – Bink Video file. A video compression system developed by RAD Game Tools • BKF (.bkf) – Microsoft backup created by NTBACKUP.EXE • bzip2 – (.bz2) • bld - Skyscraper Simulator Building • c4 – JEDMICS image files, a DOD system • cab – Microsoft Cabinet • cals – JEDMICS image files, a DOD system • cpt/sea – Compact Pro (Macintosh) • DAA – Closed-format, Windows-only compressed disk image • deb – Debian Linux install package • DMG – an Apple compressed/encrypted format • DDZ – a file which can only be used by the "daydreamer engine" created by "fever-dreamer", a program similar to RAGS, it's mainly used to make somewhat short games. • DPE – Package of AVE documents made with Aquafadas digital publishing tools. • EEA – An encrypted CAB, ostensibly for protecting email attachments • .egg – Alzip Egg Edition compressed file • EGT (.egt) – EGT Universal Document also used to create compressed cabinet files replaces .ecab • ECAB (.ECAB, .ezip) – EGT Compressed Folder used in advanced systems to compress entire system folders, replaced by EGT Universal Document • ESS (.ess) – EGT SmartSense File, detects files compressed using the EGT compression system. • GHO (.gho, .ghs) – Norton Ghost • gzip (.gz) – Compressed file • IPG (.ipg) – Format in which Apple Inc.
    [Show full text]