DOCSLIB.ORG

Máster En Bioinformática Y Biología Computacional Trabajo Fin De

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Máster En Bioinformática Y Biología Computacional Trabajo Fin De Universidad Autónoma de Madrid Escuela politécnica superior Máster en Bioinformática y Biología Computacional Trabajo Fin de Máster PLASMIDNET. SISTEMA DE INFORMACIÓN DE MÓDULOS FUNCIONALES DE PLÁSMIDOS Autor: Fernando Freire Gómez Tutor: David Abia Holgado Ponente: Gonzalo Martínez Muñoz Febrero 2020 PLASMIDNET. SISTEMA DE INFORMACIÓN DE MÓDULOS FUNCIONALES DE PLÁSMIDOS Autor: Fernando Freire Gómez Tutor: David Abia Holgado Ponente: Gonzalo Martínez Muñoz Escuela Politécnica Superior Universidad Autónoma de Madrid Febrero 2020 i Resumen Resumen La publicación en web de los datos obtenidos en proyectos bioinformáticos o la exposición de sus algoritmos o procesos de forma que puedan ser utilizados con fuentes de datos alternativas suministradas por el usuario, requiere el diseño y construcción de sistemas capaces de lidiar con grandes volúmenes de datos, habillitados para configurar, distribuir, ejecutar y monitorear flujos de procesos distribuidos entre varios nodos, y conseguir presentar la información de forma interactiva, numérica y gráficamente, confiriendo significación a los datos y a las relaciones entre las diferentes entidades de información. El sistema desarrollado afronta estos retos, teniendo en cuenta además las limitaciones de los entornos humanos de investigación, donde los recursos están sobre todo orientados a la investigación en sí misma. Por ello el sistema se ha construido bajo un estricto criterio de au- tomatización máxima, desde el empaquetamiento y distribución de software hasta la generación de la documentación, eludiendo al máximo la instalación de productos software que conllevarían dificultades de administración y actualización, homogeneizando las pautas de diseño y construc- ción en todos los ámbitos de la aplicación (servidor, navegador), construyendo un conjunto de utilidades para facilitar las pruebas y utilizando los patrones de diseño de software mínimos y necesarios para acometer las diferentes tareas. El sistema se ha construido para soportar las necesidades de los usuarios de la base de datos de módulos funcionales de plásmidos, pero puede ser aplicado fácilmente a cualquier otro proyecto bioinformático. Palabras Clave Módulo funcional, plásmido, proteína, redes, superfamilia, PWA, nodejs, service worker, pipeline, docker, lean, literate programming iii PlasmidNet. Sistema de Información de Módulos Funcionales de Plásmidos Abstract The web publication of the data obtained by bioinformatics projects or the accessibility of their algorithms or processes so that they can be used with alternative data sources provided by the user, requires the design and construction of systems capable of dealing with large volumes of data. Also these systems must be enabled to configure, distribute, execute and monitorice pipelines of batch processes distributed among several nodes, and should have the ability to draw the information interactively, numerically and graphically, conferring meaning to the data and to the relationships between the different information entities. The system developed addresses these challenges, also taking into account the human resour- ces limitations in research environments, where the staff deal primarily towards research itself. Therefore, the system has been built under a strict criteria of maximum automation, from the packaging and distribution of software to the generation of documentation, avoiding the insta- llation of software products that would lead to complexities in administration and updating. To achieve this goal we homogenize to a maximum extent the guidelines of design and construction in all areas of the application (server, browser, pipeline scheduling), building a set of utilities to ease testing and using the minimum and necessary software design patterns to undertake the different tasks. The system has been built to support the user needs of the plasmid functional modules database, but it can be easily extended to any other bioinformatic project. Key words Functional module, plasmid, protein, networks, superfamily, PWA, nodejs, service worker, pipeline, docker, lean, literate programming iv Agradecimientos Puedo escribir estas líneas gracias a los avances científicos. Si no hubiera sido por aquel antibiótico suministrado a tiempo para curar esa neumonía, por aquellos anticoagulantes que eliminaron los trombos antes de que el daño se convirtiera en irreversible, si no fuera por tantas otras cosas pequeñas y grandes, yo ya no estaría en este mundo. Por ello, después de que la ciencia me regalara tiempo en forma de supervivencia y después de que mi última empresa me obsequiara con tiempo para dedicarme a lo que más me gustara, me planteé colaborar con departamentos de investigación con el ánimo de aportar algo de mi experiencia profesional en entornos TIC, algo que ayudara en el trabajo diario de las personas dedicadas a la ardua tarea de interpretar la realidad y cambiarla. Con el fin de lograrlo resultaba imprescindible recibir la formación necesaria para adentrarme en un área gobernada por otra semántica, por otras reglas y fundamentos ajenos en gran medida a los relacionados con mi experiencia previa. El ser elegido para cursar este máster significó un enorme privilegio que no dudé en aprove- char. Después, cuando el CBM, por iniciativa de David Abia, me propuso construir un sistema web para publicar información funcional de plásmidos, tampoco titubeé en aceptar la encomien- da. Este trabajo es un intento de construir una primera versión de este sistema. He intentado utilizar las mejores estrategias de diseño y construcción que he ido aprendiendo durante mi vida profesional previa, soslayando los errores, desde una época que arranca para mí en los inicios de Internet y las comunicaciones de banda ancha, donde todo estaba por hacer, por inventar. Pero este trabajo tiene también mucho de invención, de investigación en estrategias que jamás había utilizado ni he visto tan siquiera plantear. Espero de verdad que con esta dedicación pueda retornar al menos una ínfima parte de lo que he recibido, más allá de que signifique el trámite final para culminar este máster de Bioinformática y Biología Computacional. v PlasmidNet. Sistema de Información de Módulos Funcionales de Plásmidos vi Índice general Índice de Figuras xi Índice de Tablas xiii 1. Introducción 1 1.1. Motivación del proyecto . .1 1.2. Objetivos y enfoque . .2 1.3. Metodología y plan de trabajo . .3 2. Navegador 9 2.1. Planteamiento . .9 2.2. Carga dinámica de módulos javascript . 13 2.2.1. Circuito . 14 2.3. Artefactos injertables . 15 2.4. Base de datos local . 17 2.5. Tablas de datos . 20 2.6. Gráficos vectoriales interactivos. 20 2.7. Subsistema de caché . 21 2.7.1. Service Worker . 22 2.8. Subsistema multiversión . 26 2.9. Aplicación web progresiva . 27 2.10. Diseño de la aplicación . 29 2.11. Documentación como PWA . 31 3. Servidor web 35 3.1. Planteamiento . 35 3.2. Requisitos principales . 36 3.3. Implementación . 36 3.4. Configuración . 37 3.5. Encaminamientos . 38 3.5.1. Solicitudes de estáticos . 38 vii PlasmidNet. Sistema de Información de Módulos Funcionales de Plásmidos 3.5.2. Solicitudes de servicios REST ......................... 38 3.6. Sistema de log ...................................... 39 3.7. Carga dinámica de módulos . 39 4. Bases de Datos 41 4.1. Planteamiento . 41 4.2. Implementación . 42 4.3. Módulos funcionales de plásmidos . 42 4.4. DSL de consulta . 42 4.4.1. Componentes. 43 4.5. Carga inicial de la base de datos . 45 5. Gestor de contenidos 47 5.1. Planteamiento . 47 5.2. Implementación . 49 6. Orquestador distribuido de tareas 51 6.1. Planteamiento . 51 6.2. Definiciones . 52 6.3. Roles . 52 6.4. Casos de uso . 53 6.4.1. Ejecución de una tarea . 53 6.4.2. Ejecución de un paso . 55 6.5. Definición de flujos . 56 6.6. Base de datos de orquestación . 60 6.7. Comunicaciones . 61 7. Automatización Extrema 63 7.1. Generación Documental . 63 7.1.1. Primeros pasos . 63 7.1.2. Enfoque inicial . 64 7.1.3. Implementación . 66 7.1.4. Bibliografía . 69 7.1.5. Conclusiones . 71 7.2. Pruebas y despliegue de software . 72 7.3. Despliegue basado en contenedores . 73 7.3.1. Creación de imágenes . 76 7.3.2. Ejecución de contenedores . 80 7.4. Otras automatizaciones . 80 viii ÍNDICE GENERAL PlasmidNet. Sistema de Información de Módulos Funcionales de Plásmidos 8. Conclusiones y trabajo futuro 83 Glosario de términos y acrónimos 85 Bibliografia 89 A. Manual de utilización 91 A.1. Servidor web ...................................... 91 A.2. Bases de Datos . 91 A.3. Gestor de contenidos . 92 A.4. Orquestador distribuido de tareas . 94 A.4.1. Base de datos de orquestación . 97 A.5. Automatización Extrema . 99 A.5.1. Generación Documental . 99 A.5.2. Pruebas y despliegue de software . 101 A.5.3. Despliegue basado en contenedores . 104 A.5.4. Otras automatizaciones . 107 B. Manual del programador 111 B.1. Estructura Global . 111 B.1.1. Módulo principal . 111 B.2. Navegador . 112 B.2.1. plasmidnet.js . 112 B.2.2. plasmidnet_widgets.js . 114 B.2.3. plasmidnet_loki.js . 125 B.2.4. plasmidnet_graphs.js . 131 B.2.5. plasmidnet_global.js . 141 B.2.6. plasmidnet_sw.js . 144 B.2.7. sw.js . ..
Load more

Recommended publications
  • LLNL Computation Directorate Annual Report (2014)
    PRODUCTION TEAM LLNL Associate Director for Computation Dona L. Crawford Deputy Associate Directors James Brase, Trish Damkroger, John Grosh, and Michel McCoy Scientific Editors John Westlund and Ming Jiang Art Director Amy Henke Production Editor Deanna Willis Writers Andrea Baron, Rose Hansen, Caryn Meissner, Linda Null, Michelle Rubin, and Deanna Willis Proofreader Rose Hansen Photographer Lee Baker LLNL-TR-668095 3D Designer Prepared by LLNL under Contract DE-AC52-07NA27344. Ryan Chen This document was prepared as an account of work sponsored by an agency of the United States government. Neither the United States government nor Lawrence Livermore National Security, LLC, nor any of their employees makes any warranty, expressed or implied, or assumes any legal liability or responsibility for the accuracy, completeness, or usefulness of any information, apparatus, product, or process disclosed, or represents that its use would not infringe privately owned rights. Reference herein to any specific commercial product, process, or service by trade name, trademark, Print Production manufacturer, or otherwise does not necessarily constitute or imply its endorsement, recommendation, or favoring by the United States government or Lawrence Livermore National Security, LLC. The views and opinions of authors expressed herein do not necessarily state or reflect those of the Charlie Arteago, Jr., and Monarch Print Copy and Design Solutions United States government or Lawrence Livermore National Security, LLC, and shall not be used for advertising or product endorsement purposes. CONTENTS Message from the Associate Director . 2 An Award-Winning Organization . 4 CORAL Contract Awarded and Nonrecurring Engineering Begins . 6 Preparing Codes for a Technology Transition . 8 Flux: A Framework for Resource Management .
    [Show full text]
  • Replication, History, and Grafting in the Ori File System
    Replication, History, and Grafting in the Ori File System Ali Jose´ Mashtizadeh, Andrea Bittau, Yifeng Frank Huang, David Mazieres` Stanford University Abstract backup, versioning, access from any device, and multi- user sharing—over storage capacity. But is the cloud re- Ori is a file system that manages user data in a modern ally the best place to implement data management fea- setting where users have multiple devices and wish to tures, or could the file system itself directly implement access files everywhere, synchronize data, recover from them better? disk failure, access old versions, and share data. The In terms of technological changes, disk space has in- key to satisfying these needs is keeping and replicating creased dramatically and has outgrown the increase in file system history across devices, which is now prac- wide-area bandwidth. In 1990, a typical desktop machine tical as storage space has outpaced both wide-area net- had a 60 MB hard disk, whose entire contents could tran- work (WAN) bandwidth and the size of managed data. sit a 9,600 baud modem in under 14 hours [2]. Today, Replication provides access to files from multiple de- $120 can buy a 3 TB disk, which requires 278 days to vices. History provides synchronization and offline ac- transfer over a 1 Mbps broadband connection! Clearly, cess. Replication and history together subsume backup cloud-based storage solutions have a tough time keep- by providing snapshots and avoiding any single point of ing up with disk capacity. But capacity is also outpac- failure. In fact, Ori is fully peer-to-peer, offering oppor- ing the size of managed data—i.e., the documents on tunistic synchronization between user devices in close which users actually work (as opposed to large media proximity and ensuring that the file system is usable so files or virtual machine images that would not be stored long as a single replica remains.
    [Show full text]
  • Sistemas De Archivos Distribuido Para Clúster HPC Utilizando Ceph
    Departamento de Telecomunicaciones y Electrónica Título: Sistemas de archivos distribuido para Clúster HPC utilizando Ceph Autor: Daniel Placencia Alvarez Tutor: Ing. Javier Antonio Ruiz Bosch , Junio, 2019 Este documento es Propiedad Patrimonial de la Universidad Central “Marta Abreu” de Las Villas, y se encuentra depositado en los fondos de la Biblioteca Universitaria “Chiqui Gómez Lubian” subordinada a la Dirección de Información Científico Técnica de la mencionada casa de altos estudios. Se autoriza su utilización bajo la licencia siguiente: Atribución- No Comercial- Compartir Igual Para cualquier información contacte con: Dirección de Información Científico Técnica. Universidad Central “Marta Abreu” de Las Villas. Carretera a Camajuaní. Km 5½. Santa Clara. Villa Clara. Cuba. CP. 54 830 Teléfonos.: +53 01 42281503-1419 Hago constar que el presente trabajo de diploma fue realizado en la Universidad Central “Marta Abreu” de Las Villas como parte de la culminación de estudios de la especialidad de Ingeniería en Telecomunicaciones y Electrónica, autorizando a que el mismo sea utilizado por la Institución, para los fines que estime conveniente, tanto de forma parcial como total y que además no podrá ser presentado en eventos, ni publicados sin autorización de la Universidad. Firma del Autor Los abajo firmantes certificamos que el presente trabajo ha sido realizado según acuerdo de la dirección de nuestro centro y el mismo cumple con los requisitos que debe tener un trabajo de esta envergadura referido a la temática señalada. Firma del Tutor Firma del Jefe de Departamento donde se defiende el trabajo Firma del Responsable de Información Científico-Técnica i PENSAMIENTO Muchos de los fracasos en la vida lo experimentan personas que no se dan cuenta de cuan cerca estuvieron del éxito cuando decidieron darse por vencidos.
    [Show full text]
  • The Evolution of File Systems
    The Evolution of File Systems Thomas Rivera, Hitachi Data Systems Craig Harmer, April 2011 SNIA Legal Notice The material contained in this tutorial is copyrighted by the SNIA. Member companies and individuals may use this material in presentations and literature under the following conditions: Any slide or slides used must be reproduced without modification The SNIA must be acknowledged as source of any material used in the body of any document containing material from these presentations. This presentation is a project of the SNIA Education Committee. Neither the Author nor the Presenter is an attorney and nothing in this presentation is intended to be nor should be construed as legal advice or opinion. If you need legal advice or legal opinion please contact an attorney. The information presented herein represents the Author's personal opinion and current understanding of the issues involved. The Author, the Presenter, and the SNIA do not assume any responsibility or liability for damages arising out of any reliance on or use of this information. NO WARRANTIES, EXPRESS OR IMPLIED. USE AT YOUR OWN RISK. The Evolution of File Systems 2 © 2012 Storage Networking Industry Association. All Rights Reserved. 2 Abstract The File Systems Evolution Over time additional file systems appeared focusing on specialized requirements such as: data sharing, remote file access, distributed file access, parallel files access, HPC, archiving, security, etc. Due to the dramatic growth of unstructured data, files as the basic units for data containers are morphing into file objects, providing more semantics and feature- rich capabilities for content processing This presentation will: Categorize and explain the basic principles of currently available file system architectures (e.g.
    [Show full text]
  • Content Addressed, Versioned, P2P File System (DRAFT 3)
    IPFS - Content Addressed, Versioned, P2P File System (DRAFT 3) Juan Benet [email protected] ABSTRACT parties invested in the current model. But from another per- The InterPlanetary File System (IPFS) is a peer-to-peer dis- spective, new protocols have emerged and gained wide use tributed file system that seeks to connect all computing de- since the emergence of HTTP. What is lacking is upgrading vices with the same system of files. In some ways, IPFS design: enhancing the current HTTP web, and introducing is similar to the Web, but IPFS could be seen as a sin- new functionality without degrading user experience. gle BitTorrent swarm, exchanging objects within one Git Industry has gotten away with using HTTP this long be- repository. In other words, IPFS provides a high through- cause moving small files around is relatively cheap, even for put content-addressed block storage model, with content- small organizations with lots of traffic. But we are enter- addressed hyper links. This forms a generalized Merkle ing a new era of data distribution with new challenges: (a) DAG, a data structure upon which one can build versioned hosting and distributing petabyte datasets, (b) computing file systems, blockchains, and even a Permanent Web. IPFS on large data across organizations, (c) high-volume high- combines a distributed hashtable, an incentivized block ex- definition on-demand or real-time media streams, (d) ver- change, and a self-certifying namespace. IPFS has no single sioning and linking of massive datasets, (e) preventing ac- point of failure, and nodes do not need to trust each other.
    [Show full text]
  • The File Systems Evolution
    The File Systems Evolution Christian Bandulet Principal Engineer, Sun Microsystems SNIA Legal Notice The material contained in this tutorial is copyrighted by the SNIA. Member companies and individuals may use this material in presentations and literature under the following conditions: Any slide or slides used must be reproduced without modification The SNIA must be acknowledged as source of any material used in the body of any document containing material from these presentations. This presentation is a project of the SNIA Education Committee. Neither the Author nor the Presenter is an attorney and nothing in this presentation is intended to be nor should be construed as legal advice or opinion. If you need legal advice or legal opinion please contact an attorney. The information presented herein represents the Author's personal opinion and current understanding of the issues involved. The Author, the Presenter, and the SNIA do not assume any responsibility or liability for damages arising out of any reliance on or use of this information. NO WARRANTIES, EXPRESS OR IMPLIED. USE AT YOUR OWN RISK. The File Systems Evolution © 2009 Storage Networking Industry Association. All Rights Reserved. 2 Abstract The File Systems Evolution File Systems impose structure on the address space of one or more physical or virtual devices. Starting with local file systems over time additional file systems appeared focusing on specialized requirements such as data sharing, remote file access, distributed file access, parallel files access, HPC, archiving, security etc.. Due to the dramatic growth of unstructured data files as the basic units for data containers are morphing into file objects providing more semantics and feature-rich capabilities for content processing.
    [Show full text]
  • February 2012 Vol
    Server Message Block in the Age of Microsoft Glasnost CHRISTOPHER R. HERTEL Tasting Client/Network/Server Pie STUART KENDRICK Three Years of Python 3 DAVID BEAZLEY Conference Reports from the 14th International FEBRUARY 2012 VOL. 37, NO. 1 Workshop on High Performance Transaction Systems (HPTS) UPCOMING EVENTS In Cooperation: EuroSys 2012 3rd USENIX Conference on Web Application SPONSORED BY ACM SIGOPS IN COOPERATION WITH USENIX Development (WebApps ’12) April 10–13, 2012, Bern, Switzerland June 13–14, 2012 http://eurosys2012.unibe.ch http://www.usenix.org/webapps12 4th USENIX Workshop on Hot Topics in Cloud 2nd USENIX Workshop on Hot Topics in Man- Computing (HotCloud ’12) agement of Internet, Cloud, and Enterprise June 12–13, 2012 Networks and Services (Hot-ICE ’12) http://www.usenix.org/hotcloud12 CO-LOCATED WITH NSDI ’12 Submissions due: March 8, 2012 April 24, 2012, San Jose, CA, USA 4th USENIX Workshop on Hot Topics in Storage http://www.usenix.org/hotice12 and File Systems (HotStorage ’12) 5th USENIX Workshop on Large-Scale Exploits June 13–14, 2012 and Emergent Threats (LEET ’12) http://www.usenix.org/hotstorage12 Submissions due: March 12, 2012 CO-LOCATED WITH NSDI ’12 April 24, 2012, San Jose, CA, USA 4th USENIX Workshop on the Theory and http://www.usenix.org/leet12 Practice of Provenance (TaPP ’12) June 14–15, 2012 9th USENIX Symposium on Networked Systems http://www.usenix.org/tapp12 Design and Implementation (NSDI ’12) Submissions due: March 31, 2012 SPONSORED BY USENIX IN COOPERATION WITH ACM SIGCOMM AND ACM SIGOPS 6th
    [Show full text]
  • Filesystems” by Vince Freeh (NCSU) Journaling
    ECE590-03 Enterprise Storage Architecture Fall 2016 File Systems Tyler Bletsch Duke University The file system layer User code open, read, write, seek, close, stat, mkdir, rmdir, unlink, ... Kernel VFS layer File system drivers ext4 fat nfs ... Disk driver NIC driver read_block, write_block packets Could be a single drive or a RAID HDD / SSD 2 Disk file systems • All have same goal: • Fulfill file system calls (open, seek, read, write, close, mkdir, etc.) • Store resulting data on a block device • The big (non-academic) file systems • FAT (“File Allocation Table”): Primitive Microsoft filesystem for use on floppy disks and later adapted to hard drives • FAT32 (1996) still in use (default file system for USB sticks, SD cards, etc.) • Bad performance, poor recoverability on crash, but near-universal and easy for simple systems to implement • ext2, ext3, ext4: Popular Linux file system. • Ext2 (1993) has inode-based on-disk layout – much better scalability than FAT • Ext3 (2001) adds journaling – much better recoverability than FAT • Ext4 (2008) adds various smaller benefits • NTFS: Current Microsoft filesystem (1993). • Like ext3, adds journaling to provide better recoverability than FAT • More expressive metadata (e.g. Access Control Lists (ACLs)) • HFS+: Current Mac filesystem (1998). Probably good I guess? • “Next gen” file systems: ZFS (2005), btrfs (2009), WAFL (1998), and others • Block indirection allows snapshots, copy-on-write clones, and deduplication • Often, file system handles redundancy itself – no separate RAID layer 3 FAT
    [Show full text]
  • Fossil an Archival File Server
    Fossil an archival file server Russ Cox [email protected] PDOS Group Meeting January 7, 2003 http://pdos/~rsc/talks History ..................................................................................................................... Cached WORM file server (Quinlan and Thompson): ߝ active file system on magnetic disk acts as worm cache ߝ mark all disk blocks copy-on-write at 5am to take snapshot ߝ slowly dribble snapshot to worm ߝ maintain forward linked list of snapshots ߝ present snapshot tree to users ߝ became integral part of our computing environment % ls -lp /n/dump/*/*/386/bin/8c | uniq --rwxrwxr-x presotto sys 243549 Jan 21 1997 8c ... --rwxrwxr-x presotto sys 298289 Dec 14 18:55 8c % % yesterday -D authsrv.c diff -n /n/dump/2003/0106/sys/src/cmd/auth/authsrv.c authsrv.c /n/dump/2003/0106/sys/src/cmd/auth/authsrv.c:100 c authsrv.c:100 < break; --- > exits(0); % Quinlan, ‘‘A Cached WORM File System’’, SP&E December 1991. http://plan9.bell-labs.com/~seanq/cw.pdf History, ii................................................................................................................ WORM was right choice in 1990 ߝ one jukebox is infinite: capacity grows faster than our storage needs ߝ no head crashes ߝ plausible random access times ߝ magnetic disks too small, tape too slow ߝ bootes (1990): 100MB mem, 1GB disk, 300GB juke box ߝ emelie (1997): 350MB mem, 54GB disk, 1.2TB juke box What about 1999? ߝ disks cheap and big, getting cheaper and bigger ߝ disks cheaper and bigger than optical disk ߝ disks much faster than optical disk ߝ disks have head crashes ߝ build a better base out of magnetic disk? Venti ........................................................................................................................
    [Show full text]
  • Snapshots in a Flash with Iosnap
    Snapshots in a Flash with ioSnap Sriram Subramanian, Andrea C. Arpaci-Dusseau, Swaminathan Sundararaman, Remzi H. Arpaci-Dusseau Nisha Talagala Computer Sciences Department, University of Fusion-IO Inc. Wisconsin-Madison {srsubramanian,swami,ntalagala}@fusionio.com {dusseau,remzi}@cs.wisc.edu Abstract tation of the state of a data volume, used in backups, repli- Snapshots are a common and heavily relied upon feature in cation, and other functions. storage systems. The high performance of flash-based stor- However, the world of storage is in the midst of a sig- age systems brings new, more stringent, requirements for nificant change, due to the advent of flash-based persistent this classic capability. We present ioSnap, a flash optimized memory [2, 5, 9]. Flash provides much higher throughput snapshot system. Through careful design exploiting com- and lower latency to applications, particularly for those with mon snapshot usage patterns and flash oriented optimiza- random I/O needs, and thus is increasingly integral in mod- tions, including leveraging native characteristics of Flash ern storage stacks. Translation Layers, ioSnap delivers low-overhead snapshots As flash-based storage becomes more prevalent by the with minimal disruption to foreground traffic. Through our rapid fall in prices (from hundreds of dollars per GB to evaluation, we show that ioSnap incurs negligible perfor- about $1 per GB in the span of 10 years [8]) and significant mance overhead during normal operation, and that common- increase in capacity (up to several TBs [7]), customers re- case operations such as snapshot creation and deletion in- quire flash devices to deliver the same features as traditional cur little cost.
    [Show full text]
  • Comparison of File Systems
    Comparison of file systems - Wikipedia, the free e... http://en.wikipedia.org/wiki/Comparison_of_file_s... Comparison of file systems From Wikipedia, the free encyclopedia The following tables compare general and technical information for a number of file systems. Contents 1 General information 2 Limits 3 Metadata 4 Features 5 Allocation and layout policies 6 Supporting operating systems 7 See also 8 Notes 9 External links General information Year File system Creator Original operating system introduced DECtape DEC 1964 PDP-6 Monitor Level-D DEC 1968 TOPS-10 George 2 ICT (later ICL) 1968 George 2 V6FS Bell Labs 1972 Version 6 Unix ODS-1 DEC 1972 RSX-11 RT-11 file system DEC 1973 RT-11 DOS (GEC) GEC 1973 Core Operating System CP/M file systemG ary Kildall 1974 CP/M OS4000 GEC 1977 OS4000 FAT12 Microsoft 1977 Microsoft Disk BASIC DOS 3.x Apple Computer 1978 Apple DOS Pascal Apple Computer 1978 Apple Pascal CBM DOS Commodore 1978 Microsoft BASIC (for CBM PET) V7FS Bell Labs 1979 Version 7 Unix ODS-2 DEC 1979 OpenVMS DFS Acorn Computers Ltd 1982 Acorn BBC Micro MOS ADFS Acorn Computers Ltd 1983 Acorn Electron (later Arthur RISC OS) FFS Kirk McKusick 1983 4.2BSD ProDOS Apple Computer 1983 ProDOS 8 MFS Apple Computer 1984 Mac OS Elektronika BK tape NPO "Scientific centre" (now 1985 Vilnius Basic, BK monitor program format Sitronics) HFS Apple Computer 1985 Mac OS Amiga OFS[11] Metacomco for Commodore 1985 Amiga OS High Sierra Ecma International 1985 MS-DOS, Mac OS NWFS Novell 1985 NetWare 286 FAT16 Microsoft 1987 MS-DOS 3.31 Minix V1 FS Andrew S.
    [Show full text]
  • Replication, History, and Grafting in the Ori File System
    Replication, History, and Grafting in the Ori File System Ali Jose´ Mashtizadeh, Andrea Bittau, Yifeng Frank Huang, David Mazieres` Stanford University Abstract backup, versioning, access from any device, and multi- user sharing—over storage capacity. But is the cloud re- Ori is a file system that manages user data in a modern ally the best place to implement data management fea- setting where users have multiple devices and wish to tures, or could the file system itself directly implement access files everywhere, synchronize data, recover from them better? disk failure, access old versions, and share data. The In terms of technological changes, disk space has in- key to satisfying these needs is keeping and replicating creased dramatically and has outgrown the increase in file system history across devices, which is now prac- wide-area bandwidth. In 1990, a typical desktop machine tical as storage space has outpaced both wide-area net- had a 60 MB hard disk, whose entire contents could tran- work (WAN) bandwidth and the size of managed data. sit a 9,600 baud modem in under 14 hours [2]. Today, Replication provides access to files from multiple de- $120 can buy a 3 TB disk, which requires 278 days to vices. History provides synchronization and offline ac- transfer over a 1 Mbps broadband connection! Clearly, cess. Replication and history together subsume backup cloud-based storage solutions have a tough time keep- by providing snapshots and avoiding any single point of ing up with disk capacity. But capacity is also outpac- failure. In fact, Ori is fully peer-to-peer, offering oppor- ing the size of managed data—i.e., the documents on tunistic synchronization between user devices in close which users actually work (as opposed to large media proximity and ensuring that the file system is usable so files or virtual machine images that would not be stored long as a single replica remains.
    [Show full text]