DOCSLIB.ORG

Sistemas De Archivos Distribuido Para Clúster HPC Utilizando Ceph

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Sistemas De Archivos Distribuido Para Clúster HPC Utilizando Ceph Departamento de Telecomunicaciones y Electrónica Título: Sistemas de archivos distribuido para Clúster HPC utilizando Ceph Autor: Daniel Placencia Alvarez Tutor: Ing. Javier Antonio Ruiz Bosch , Junio, 2019 Este documento es Propiedad Patrimonial de la Universidad Central “Marta Abreu” de Las Villas, y se encuentra depositado en los fondos de la Biblioteca Universitaria “Chiqui Gómez Lubian” subordinada a la Dirección de Información Científico Técnica de la mencionada casa de altos estudios. Se autoriza su utilización bajo la licencia siguiente: Atribución- No Comercial- Compartir Igual Para cualquier información contacte con: Dirección de Información Científico Técnica. Universidad Central “Marta Abreu” de Las Villas. Carretera a Camajuaní. Km 5½. Santa Clara. Villa Clara. Cuba. CP. 54 830 Teléfonos.: +53 01 42281503-1419 Hago constar que el presente trabajo de diploma fue realizado en la Universidad Central “Marta Abreu” de Las Villas como parte de la culminación de estudios de la especialidad de Ingeniería en Telecomunicaciones y Electrónica, autorizando a que el mismo sea utilizado por la Institución, para los fines que estime conveniente, tanto de forma parcial como total y que además no podrá ser presentado en eventos, ni publicados sin autorización de la Universidad. Firma del Autor Los abajo firmantes certificamos que el presente trabajo ha sido realizado según acuerdo de la dirección de nuestro centro y el mismo cumple con los requisitos que debe tener un trabajo de esta envergadura referido a la temática señalada. Firma del Tutor Firma del Jefe de Departamento donde se defiende el trabajo Firma del Responsable de Información Científico-Técnica i PENSAMIENTO Muchos de los fracasos en la vida lo experimentan personas que no se dan cuenta de cuan cerca estuvieron del éxito cuando decidieron darse por vencidos. Thomas Edison ii DEDICATORIA A mi familia, especialmente a mis padres y a mi tía Carmen Rosa, por guiarme, apoyarme incondicionalmente y estar presente en cada momento. iii AGRADECIMIENTOS - A mi familia, especialmente a mis padres, mi hermana y mi tía Carmen Rosa, por su cariño, su apoyo incondicional y su dedicación. - A mi tutor Javier Antonio Ruiz Bosch, por su dedicación. - A mis compañeros de aula, que se convirtieron en grandes amigos en los peores momentos. - A todos los profesores que durante estos cinco años han contribuido a mi formación profesional. - A todos aquellos a los que de una forma u otra participaron en la realización de este trabajo. iv TAREA TÉCNICA Para el logro de los objetivos propuestos en el presente trabajo, la investigación sigue una línea de trabajo definida por un grupo de tareas, las cuales son: Revisión bibliográfica referida a los sistemas de almacenamiento de datos para Clúster HPC. Análisis del hardware disponible para la implementación de esta tecnología. Selección de la configuración de hardware y software más apropiada para implementar este sistema en el escenario de desarrollo. Instalación, configuración y despliegue del software propuesto. Evaluación del desempeño del sistema con diferentes herramientas. Comparación del sistema propuesto con los sistemas actualmente implementados. Análisis de los resultados de la implementación y las comparaciones realizadas. Confección del trabajo de diploma. Firma del Autor Firma del Tutor v RESUMEN Los sistemas de archivos distribuidos paralelos se hacen cada vez más populares y usados por las grandes posibilidades que brindan. Ceph se presenta como una plataforma de almacenamiento unificada, definida por software, con excelentes prestaciones para ambientes donde la velocidad es determinante como es el caso de los clústeres HPC. La presente investigación se dedica a la implementación de un sistema de archivos Ceph para el clúster HPC del Centro de Datos de la UCLV. Inicialmente se analizan las principales tecnologías de almacenamiento empleadas en la actualidad. Se explica paso a paso el proceso de instalación de un sistema de archivos Ceph. Se presenta el proceso de administración y gestión de un clúster Ceph, resaltando las principales variables que se monitorean y los fallos más comunes. Se realizan pruebas al clúster Ceph de estabilidad y rendimiento empleando diferentes herramientas. Además, se realizan pruebas de rendimiento al sistema NFS que brinda servicios al HPC, lo que permite realizar importantes comparaciones. Como conclusión se obtiene que el clúster Ceph permanece estable ante fallos de software y hardware que no superen su dominio de fallo y presenta un alto rendimiento en todas las operaciones con archivos, superior al del servidor NFS. vi ÍNDICE PENSAMIENTO ...................................................................................................................... i DEDICATORIA ...................................................................................................................... ii AGRADECIMIENTOS .......................................................................................................... iii TAREA TÉCNICA ................................................................................................................. iv RESUMEN .............................................................................................................................. v INTRODUCCIÓN ................................................................................................................... 1 CAPÍTULO 1. SISTEMAS DE ARCHIVOS ...................................................................... 4 1.1 Sistemas de archivos tradicionales ................................................................... 5 1.1.1 ¿Qué es un sistema de archivos? ........................................................... 5 1.1.2 Sistemas de archivos tradicionales y modernos ..................................... 6 1.2 Soluciones para alto desempeño y escalabilidad ............................................... 8 1.2.1 Sistemas de archivos de red ................................................................... 8 1.2.4 Almacenamiento basado en objetos y basado en bloques ..................... 15 1.3 Arquitecturas modernas para clúster HPC .................................................... 16 1.3.1 GPFS .................................................................................................. 16 1.3.2 HDFS .................................................................................................. 17 1.3.3 BeeGFS ............................................................................................... 18 1.3.4 Lustre ................................................................................................. 19 1.3.5 GlusterFS ............................................................................................ 21 1.3.6 Ceph ................................................................................................... 23 1.4 Selección del sistema de archivos a implementar en el Clúster HPC .............. 26 CAPÍTULO 2. IMPLEMENTACIÓN DEL SISTEMA DE ARCHIVOS CEPH EN EL CLÚSTER HPC .................................................................................................................... 29 2.1 Preparación del hardware y el software necesario ......................................... 30 2.1.1 Arquitectura básica del clúster Ceph .................................................. 30 2.1.2 Recomendaciones del hardware y software ......................................... 33 2.1.3 Preparación del entorno de instalación ............................................... 37 2.2 Procedimiento de instalación de Ceph empleando ceph-deploy ....................... 39 2.3 Administración y supervisión del clúster Ceph .............................................. 48 2.4 Conclusiones del capítulo ............................................................................... 52 vii CAPÍTULO 3. RESULTADOS DE LA IMPLEMENTACIÓN DEL SISTEMA DE ARCHIVOS CEPH EN EL CLÚSTER HPC ......................................................................... 53 3.1 Estabilidad del clúster Ceph ante fallos de software y hardware .................... 54 3.2 Rendimiento del clúster Ceph ........................................................................ 57 3.2.1 RADOS Bench .................................................................................... 58 3.2.2 DD ....................................................................................................... 60 3.2.3 Bonnie++ ............................................................................................. 61 3.3 Comparación con el sistema de archivos NFS ................................................ 64 3.4 Análisis de los resultados obtenidos ................................................................ 66 3.5 Conclusiones del capítulo ............................................................................... 66 CONCLUSIONES Y RECOMENDACIONES ...................................................................... 67 Conclusiones .............................................................................................................. 67 Recomendaciones ....................................................................................................... 68 BIBLIOGRAFÍA ................................................................................................................... 69 ANEXOS ............................................................................................................................... 72 Anexo I: Códigos de chequeo de salud del clúster Ceph más comunes
Load more

Recommended publications
  • LLNL Computation Directorate Annual Report (2014)
    PRODUCTION TEAM LLNL Associate Director for Computation Dona L. Crawford Deputy Associate Directors James Brase, Trish Damkroger, John Grosh, and Michel McCoy Scientific Editors John Westlund and Ming Jiang Art Director Amy Henke Production Editor Deanna Willis Writers Andrea Baron, Rose Hansen, Caryn Meissner, Linda Null, Michelle Rubin, and Deanna Willis Proofreader Rose Hansen Photographer Lee Baker LLNL-TR-668095 3D Designer Prepared by LLNL under Contract DE-AC52-07NA27344. Ryan Chen This document was prepared as an account of work sponsored by an agency of the United States government. Neither the United States government nor Lawrence Livermore National Security, LLC, nor any of their employees makes any warranty, expressed or implied, or assumes any legal liability or responsibility for the accuracy, completeness, or usefulness of any information, apparatus, product, or process disclosed, or represents that its use would not infringe privately owned rights. Reference herein to any specific commercial product, process, or service by trade name, trademark, Print Production manufacturer, or otherwise does not necessarily constitute or imply its endorsement, recommendation, or favoring by the United States government or Lawrence Livermore National Security, LLC. The views and opinions of authors expressed herein do not necessarily state or reflect those of the Charlie Arteago, Jr., and Monarch Print Copy and Design Solutions United States government or Lawrence Livermore National Security, LLC, and shall not be used for advertising or product endorsement purposes. CONTENTS Message from the Associate Director . 2 An Award-Winning Organization . 4 CORAL Contract Awarded and Nonrecurring Engineering Begins . 6 Preparing Codes for a Technology Transition . 8 Flux: A Framework for Resource Management .
    [Show full text]
  • Replication, History, and Grafting in the Ori File System
    Replication, History, and Grafting in the Ori File System Ali Jose´ Mashtizadeh, Andrea Bittau, Yifeng Frank Huang, David Mazieres` Stanford University Abstract backup, versioning, access from any device, and multi- user sharing—over storage capacity. But is the cloud re- Ori is a file system that manages user data in a modern ally the best place to implement data management fea- setting where users have multiple devices and wish to tures, or could the file system itself directly implement access files everywhere, synchronize data, recover from them better? disk failure, access old versions, and share data. The In terms of technological changes, disk space has in- key to satisfying these needs is keeping and replicating creased dramatically and has outgrown the increase in file system history across devices, which is now prac- wide-area bandwidth. In 1990, a typical desktop machine tical as storage space has outpaced both wide-area net- had a 60 MB hard disk, whose entire contents could tran- work (WAN) bandwidth and the size of managed data. sit a 9,600 baud modem in under 14 hours [2]. Today, Replication provides access to files from multiple de- $120 can buy a 3 TB disk, which requires 278 days to vices. History provides synchronization and offline ac- transfer over a 1 Mbps broadband connection! Clearly, cess. Replication and history together subsume backup cloud-based storage solutions have a tough time keep- by providing snapshots and avoiding any single point of ing up with disk capacity. But capacity is also outpac- failure. In fact, Ori is fully peer-to-peer, offering oppor- ing the size of managed data—i.e., the documents on tunistic synchronization between user devices in close which users actually work (as opposed to large media proximity and ensuring that the file system is usable so files or virtual machine images that would not be stored long as a single replica remains.
    [Show full text]
  • The Evolution of File Systems
    The Evolution of File Systems Thomas Rivera, Hitachi Data Systems Craig Harmer, April 2011 SNIA Legal Notice The material contained in this tutorial is copyrighted by the SNIA. Member companies and individuals may use this material in presentations and literature under the following conditions: Any slide or slides used must be reproduced without modification The SNIA must be acknowledged as source of any material used in the body of any document containing material from these presentations. This presentation is a project of the SNIA Education Committee. Neither the Author nor the Presenter is an attorney and nothing in this presentation is intended to be nor should be construed as legal advice or opinion. If you need legal advice or legal opinion please contact an attorney. The information presented herein represents the Author's personal opinion and current understanding of the issues involved. The Author, the Presenter, and the SNIA do not assume any responsibility or liability for damages arising out of any reliance on or use of this information. NO WARRANTIES, EXPRESS OR IMPLIED. USE AT YOUR OWN RISK. The Evolution of File Systems 2 © 2012 Storage Networking Industry Association. All Rights Reserved. 2 Abstract The File Systems Evolution Over time additional file systems appeared focusing on specialized requirements such as: data sharing, remote file access, distributed file access, parallel files access, HPC, archiving, security, etc. Due to the dramatic growth of unstructured data, files as the basic units for data containers are morphing into file objects, providing more semantics and feature- rich capabilities for content processing This presentation will: Categorize and explain the basic principles of currently available file system architectures (e.g.
    [Show full text]
  • Content Addressed, Versioned, P2P File System (DRAFT 3)
    IPFS - Content Addressed, Versioned, P2P File System (DRAFT 3) Juan Benet [email protected] ABSTRACT parties invested in the current model. But from another per- The InterPlanetary File System (IPFS) is a peer-to-peer dis- spective, new protocols have emerged and gained wide use tributed file system that seeks to connect all computing de- since the emergence of HTTP. What is lacking is upgrading vices with the same system of files. In some ways, IPFS design: enhancing the current HTTP web, and introducing is similar to the Web, but IPFS could be seen as a sin- new functionality without degrading user experience. gle BitTorrent swarm, exchanging objects within one Git Industry has gotten away with using HTTP this long be- repository. In other words, IPFS provides a high through- cause moving small files around is relatively cheap, even for put content-addressed block storage model, with content- small organizations with lots of traffic. But we are enter- addressed hyper links. This forms a generalized Merkle ing a new era of data distribution with new challenges: (a) DAG, a data structure upon which one can build versioned hosting and distributing petabyte datasets, (b) computing file systems, blockchains, and even a Permanent Web. IPFS on large data across organizations, (c) high-volume high- combines a distributed hashtable, an incentivized block ex- definition on-demand or real-time media streams, (d) ver- change, and a self-certifying namespace. IPFS has no single sioning and linking of massive datasets, (e) preventing ac- point of failure, and nodes do not need to trust each other.
    [Show full text]
  • The File Systems Evolution
    The File Systems Evolution Christian Bandulet Principal Engineer, Sun Microsystems SNIA Legal Notice The material contained in this tutorial is copyrighted by the SNIA. Member companies and individuals may use this material in presentations and literature under the following conditions: Any slide or slides used must be reproduced without modification The SNIA must be acknowledged as source of any material used in the body of any document containing material from these presentations. This presentation is a project of the SNIA Education Committee. Neither the Author nor the Presenter is an attorney and nothing in this presentation is intended to be nor should be construed as legal advice or opinion. If you need legal advice or legal opinion please contact an attorney. The information presented herein represents the Author's personal opinion and current understanding of the issues involved. The Author, the Presenter, and the SNIA do not assume any responsibility or liability for damages arising out of any reliance on or use of this information. NO WARRANTIES, EXPRESS OR IMPLIED. USE AT YOUR OWN RISK. The File Systems Evolution © 2009 Storage Networking Industry Association. All Rights Reserved. 2 Abstract The File Systems Evolution File Systems impose structure on the address space of one or more physical or virtual devices. Starting with local file systems over time additional file systems appeared focusing on specialized requirements such as data sharing, remote file access, distributed file access, parallel files access, HPC, archiving, security etc.. Due to the dramatic growth of unstructured data files as the basic units for data containers are morphing into file objects providing more semantics and feature-rich capabilities for content processing.
    [Show full text]
  • February 2012 Vol
    Server Message Block in the Age of Microsoft Glasnost CHRISTOPHER R. HERTEL Tasting Client/Network/Server Pie STUART KENDRICK Three Years of Python 3 DAVID BEAZLEY Conference Reports from the 14th International FEBRUARY 2012 VOL. 37, NO. 1 Workshop on High Performance Transaction Systems (HPTS) UPCOMING EVENTS In Cooperation: EuroSys 2012 3rd USENIX Conference on Web Application SPONSORED BY ACM SIGOPS IN COOPERATION WITH USENIX Development (WebApps ’12) April 10–13, 2012, Bern, Switzerland June 13–14, 2012 http://eurosys2012.unibe.ch http://www.usenix.org/webapps12 4th USENIX Workshop on Hot Topics in Cloud 2nd USENIX Workshop on Hot Topics in Man- Computing (HotCloud ’12) agement of Internet, Cloud, and Enterprise June 12–13, 2012 Networks and Services (Hot-ICE ’12) http://www.usenix.org/hotcloud12 CO-LOCATED WITH NSDI ’12 Submissions due: March 8, 2012 April 24, 2012, San Jose, CA, USA 4th USENIX Workshop on Hot Topics in Storage http://www.usenix.org/hotice12 and File Systems (HotStorage ’12) 5th USENIX Workshop on Large-Scale Exploits June 13–14, 2012 and Emergent Threats (LEET ’12) http://www.usenix.org/hotstorage12 Submissions due: March 12, 2012 CO-LOCATED WITH NSDI ’12 April 24, 2012, San Jose, CA, USA 4th USENIX Workshop on the Theory and http://www.usenix.org/leet12 Practice of Provenance (TaPP ’12) June 14–15, 2012 9th USENIX Symposium on Networked Systems http://www.usenix.org/tapp12 Design and Implementation (NSDI ’12) Submissions due: March 31, 2012 SPONSORED BY USENIX IN COOPERATION WITH ACM SIGCOMM AND ACM SIGOPS 6th
    [Show full text]
  • Máster En Bioinformática Y Biología Computacional Trabajo Fin De
    Universidad Autónoma de Madrid Escuela politécnica superior Máster en Bioinformática y Biología Computacional Trabajo Fin de Máster PLASMIDNET. SISTEMA DE INFORMACIÓN DE MÓDULOS FUNCIONALES DE PLÁSMIDOS Autor: Fernando Freire Gómez Tutor: David Abia Holgado Ponente: Gonzalo Martínez Muñoz Febrero 2020 PLASMIDNET. SISTEMA DE INFORMACIÓN DE MÓDULOS FUNCIONALES DE PLÁSMIDOS Autor: Fernando Freire Gómez Tutor: David Abia Holgado Ponente: Gonzalo Martínez Muñoz Escuela Politécnica Superior Universidad Autónoma de Madrid Febrero 2020 i Resumen Resumen La publicación en web de los datos obtenidos en proyectos bioinformáticos o la exposición de sus algoritmos o procesos de forma que puedan ser utilizados con fuentes de datos alternativas suministradas por el usuario, requiere el diseño y construcción de sistemas capaces de lidiar con grandes volúmenes de datos, habillitados para configurar, distribuir, ejecutar y monitorear flujos de procesos distribuidos entre varios nodos, y conseguir presentar la información de forma interactiva, numérica y gráficamente, confiriendo significación a los datos y a las relaciones entre las diferentes entidades de información. El sistema desarrollado afronta estos retos, teniendo en cuenta además las limitaciones de los entornos humanos de investigación, donde los recursos están sobre todo orientados a la investigación en sí misma. Por ello el sistema se ha construido bajo un estricto criterio de au- tomatización máxima, desde el empaquetamiento y distribución de software hasta la generación de la documentación, eludiendo al máximo la instalación de productos software que conllevarían dificultades de administración y actualización, homogeneizando las pautas de diseño y construc- ción en todos los ámbitos de la aplicación (servidor, navegador), construyendo un conjunto de utilidades para facilitar las pruebas y utilizando los patrones de diseño de software mínimos y necesarios para acometer las diferentes tareas.
    [Show full text]
  • Filesystems” by Vince Freeh (NCSU) Journaling
    ECE590-03 Enterprise Storage Architecture Fall 2016 File Systems Tyler Bletsch Duke University The file system layer User code open, read, write, seek, close, stat, mkdir, rmdir, unlink, ... Kernel VFS layer File system drivers ext4 fat nfs ... Disk driver NIC driver read_block, write_block packets Could be a single drive or a RAID HDD / SSD 2 Disk file systems • All have same goal: • Fulfill file system calls (open, seek, read, write, close, mkdir, etc.) • Store resulting data on a block device • The big (non-academic) file systems • FAT (“File Allocation Table”): Primitive Microsoft filesystem for use on floppy disks and later adapted to hard drives • FAT32 (1996) still in use (default file system for USB sticks, SD cards, etc.) • Bad performance, poor recoverability on crash, but near-universal and easy for simple systems to implement • ext2, ext3, ext4: Popular Linux file system. • Ext2 (1993) has inode-based on-disk layout – much better scalability than FAT • Ext3 (2001) adds journaling – much better recoverability than FAT • Ext4 (2008) adds various smaller benefits • NTFS: Current Microsoft filesystem (1993). • Like ext3, adds journaling to provide better recoverability than FAT • More expressive metadata (e.g. Access Control Lists (ACLs)) • HFS+: Current Mac filesystem (1998). Probably good I guess? • “Next gen” file systems: ZFS (2005), btrfs (2009), WAFL (1998), and others • Block indirection allows snapshots, copy-on-write clones, and deduplication • Often, file system handles redundancy itself – no separate RAID layer 3 FAT
    [Show full text]
  • Fossil an Archival File Server
    Fossil an archival file server Russ Cox [email protected] PDOS Group Meeting January 7, 2003 http://pdos/~rsc/talks History ..................................................................................................................... Cached WORM file server (Quinlan and Thompson): ߝ active file system on magnetic disk acts as worm cache ߝ mark all disk blocks copy-on-write at 5am to take snapshot ߝ slowly dribble snapshot to worm ߝ maintain forward linked list of snapshots ߝ present snapshot tree to users ߝ became integral part of our computing environment % ls -lp /n/dump/*/*/386/bin/8c | uniq --rwxrwxr-x presotto sys 243549 Jan 21 1997 8c ... --rwxrwxr-x presotto sys 298289 Dec 14 18:55 8c % % yesterday -D authsrv.c diff -n /n/dump/2003/0106/sys/src/cmd/auth/authsrv.c authsrv.c /n/dump/2003/0106/sys/src/cmd/auth/authsrv.c:100 c authsrv.c:100 < break; --- > exits(0); % Quinlan, ‘‘A Cached WORM File System’’, SP&E December 1991. http://plan9.bell-labs.com/~seanq/cw.pdf History, ii................................................................................................................ WORM was right choice in 1990 ߝ one jukebox is infinite: capacity grows faster than our storage needs ߝ no head crashes ߝ plausible random access times ߝ magnetic disks too small, tape too slow ߝ bootes (1990): 100MB mem, 1GB disk, 300GB juke box ߝ emelie (1997): 350MB mem, 54GB disk, 1.2TB juke box What about 1999? ߝ disks cheap and big, getting cheaper and bigger ߝ disks cheaper and bigger than optical disk ߝ disks much faster than optical disk ߝ disks have head crashes ߝ build a better base out of magnetic disk? Venti ........................................................................................................................
    [Show full text]
  • Snapshots in a Flash with Iosnap
    Snapshots in a Flash with ioSnap Sriram Subramanian, Andrea C. Arpaci-Dusseau, Swaminathan Sundararaman, Remzi H. Arpaci-Dusseau Nisha Talagala Computer Sciences Department, University of Fusion-IO Inc. Wisconsin-Madison {srsubramanian,swami,ntalagala}@fusionio.com {dusseau,remzi}@cs.wisc.edu Abstract tation of the state of a data volume, used in backups, repli- Snapshots are a common and heavily relied upon feature in cation, and other functions. storage systems. The high performance of flash-based stor- However, the world of storage is in the midst of a sig- age systems brings new, more stringent, requirements for nificant change, due to the advent of flash-based persistent this classic capability. We present ioSnap, a flash optimized memory [2, 5, 9]. Flash provides much higher throughput snapshot system. Through careful design exploiting com- and lower latency to applications, particularly for those with mon snapshot usage patterns and flash oriented optimiza- random I/O needs, and thus is increasingly integral in mod- tions, including leveraging native characteristics of Flash ern storage stacks. Translation Layers, ioSnap delivers low-overhead snapshots As flash-based storage becomes more prevalent by the with minimal disruption to foreground traffic. Through our rapid fall in prices (from hundreds of dollars per GB to evaluation, we show that ioSnap incurs negligible perfor- about $1 per GB in the span of 10 years [8]) and significant mance overhead during normal operation, and that common- increase in capacity (up to several TBs [7]), customers re- case operations such as snapshot creation and deletion in- quire flash devices to deliver the same features as traditional cur little cost.
    [Show full text]
  • Comparison of File Systems
    Comparison of file systems - Wikipedia, the free e... http://en.wikipedia.org/wiki/Comparison_of_file_s... Comparison of file systems From Wikipedia, the free encyclopedia The following tables compare general and technical information for a number of file systems. Contents 1 General information 2 Limits 3 Metadata 4 Features 5 Allocation and layout policies 6 Supporting operating systems 7 See also 8 Notes 9 External links General information Year File system Creator Original operating system introduced DECtape DEC 1964 PDP-6 Monitor Level-D DEC 1968 TOPS-10 George 2 ICT (later ICL) 1968 George 2 V6FS Bell Labs 1972 Version 6 Unix ODS-1 DEC 1972 RSX-11 RT-11 file system DEC 1973 RT-11 DOS (GEC) GEC 1973 Core Operating System CP/M file systemG ary Kildall 1974 CP/M OS4000 GEC 1977 OS4000 FAT12 Microsoft 1977 Microsoft Disk BASIC DOS 3.x Apple Computer 1978 Apple DOS Pascal Apple Computer 1978 Apple Pascal CBM DOS Commodore 1978 Microsoft BASIC (for CBM PET) V7FS Bell Labs 1979 Version 7 Unix ODS-2 DEC 1979 OpenVMS DFS Acorn Computers Ltd 1982 Acorn BBC Micro MOS ADFS Acorn Computers Ltd 1983 Acorn Electron (later Arthur RISC OS) FFS Kirk McKusick 1983 4.2BSD ProDOS Apple Computer 1983 ProDOS 8 MFS Apple Computer 1984 Mac OS Elektronika BK tape NPO "Scientific centre" (now 1985 Vilnius Basic, BK monitor program format Sitronics) HFS Apple Computer 1985 Mac OS Amiga OFS[11] Metacomco for Commodore 1985 Amiga OS High Sierra Ecma International 1985 MS-DOS, Mac OS NWFS Novell 1985 NetWare 286 FAT16 Microsoft 1987 MS-DOS 3.31 Minix V1 FS Andrew S.
    [Show full text]
  • Replication, History, and Grafting in the Ori File System
    Replication, History, and Grafting in the Ori File System Ali Jose´ Mashtizadeh, Andrea Bittau, Yifeng Frank Huang, David Mazieres` Stanford University Abstract backup, versioning, access from any device, and multi- user sharing—over storage capacity. But is the cloud re- Ori is a file system that manages user data in a modern ally the best place to implement data management fea- setting where users have multiple devices and wish to tures, or could the file system itself directly implement access files everywhere, synchronize data, recover from them better? disk failure, access old versions, and share data. The In terms of technological changes, disk space has in- key to satisfying these needs is keeping and replicating creased dramatically and has outgrown the increase in file system history across devices, which is now prac- wide-area bandwidth. In 1990, a typical desktop machine tical as storage space has outpaced both wide-area net- had a 60 MB hard disk, whose entire contents could tran- work (WAN) bandwidth and the size of managed data. sit a 9,600 baud modem in under 14 hours [2]. Today, Replication provides access to files from multiple de- $120 can buy a 3 TB disk, which requires 278 days to vices. History provides synchronization and offline ac- transfer over a 1 Mbps broadband connection! Clearly, cess. Replication and history together subsume backup cloud-based storage solutions have a tough time keep- by providing snapshots and avoiding any single point of ing up with disk capacity. But capacity is also outpac- failure. In fact, Ori is fully peer-to-peer, offering oppor- ing the size of managed data—i.e., the documents on tunistic synchronization between user devices in close which users actually work (as opposed to large media proximity and ensuring that the file system is usable so files or virtual machine images that would not be stored long as a single replica remains.
    [Show full text]