DOCSLIB.ORG

Wayback: a User-Level Versioning File System for Linux

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Wayback: a User-Level Versioning File System for Linux Wayback: A User-level Versioning File System For Linux Brian Cornell, Peter Dinda, and Fabián Bustamante Computer Science Department Northwestern University 1 Really excited last night ● Great idea for this presentation ● 3d rotating slides... ● Woke up this morning, Oh no! ● Too excited: no backup copy ● No RCS1 or CVS2 check-in ● Lucky, Wayback was there! ● Reverted back to yesterday afternoon, and here we are, aren't you glad? 1: F. Tichy, 1980. 2: B. Berliner, 2001. 2 Outline ● Overview ● Related work ● How it works – User's perspective ● Behind the scenes – System's perspective ● Design decisions ● Performance ● Future work 3 Wayback: Automatic Versioning ● Automatic: Versions files and directories without user interaction ● Universal: Versions files of all types ● Comprehensive: Never loses data ● Generic: Works over any directory on any base FS ● User-space: Implemented in user-space Free (GPL): http://wayback.sourceforge.net 4 Related Work ● VMS operating system [K McCoy, 1990] – Version on close ● Elephant [D Santry, et al, 1999] – Exploit massive disks ● VersionFS stackable file system [Muniswamy- Reddy, et al, 2004] – Contemporaneous to this project – Implements different features ● Cedar [D. K. Gifford, et al, 1998], 3DFS [D. G. Korn, et al, 1989], CVFS [C. A. Soules, et al, 2003] 5 How does it work? ● Three simple steps ● Remount folder ● Edit and work as normal ● Revert or extract old versions 6 Remount folder ● Run wayback executable to remount – wayback /mnt/data /home/brian/Projects ● Use mount.wayback script to simplify and mount for all users – mount.wayback /mnt/data /home/brian/Projects ● Files still at /mnt/data, but if you edit them at /home/brian/Projects, they are versioned 7 Work normally ● Edit files with any editor ● Modify binary files too ● Move, rename, delete files ● Edit directories as much as you want ● Everything you do is recorded and remembered 8 Revert or extract versions ● List the versions of a file – vstat Wayback.sxi ● Revert a file to an old version – To a time: vrevert -d 15:00:00 Wayback.sxi – To a date: vrevert -d 2004:06:29:15:00:00 Wayback.sxi – To a numbered version as listed by vstat: vrevert -n 5 Wayback.sxi ● Revert a directory to a date/time – vrevert -d 2004:06:29:15:00:00 Documents ● Note: Even reverting can be undone 9 Revert or extract versions ● Extract versions to a different file – Same format as vrevert with a destination – vextract -d 2004:06:29:15:00:00 Wayback.sxi GoodWayback.sxi ● Delete the versioning if you're sure you don't need it anymore – vrm Wayback.sxi – All versioning info for the file is permanently deleted – File is not deleted. To delete file and versioning, first delete file, then versioning. 10 Behind the Scenes ● FUSE1 module sends all FS calls to Wayback ● Everything kept track of transparently ● Hidden log files, one per file or directory ● File operations recorded ● Directory operations recorded 1: M. Szeredi,, Filesystem in USEr space, http://sourceforge.net/projects/avf 11 Log Files ● Every versioned file has a hidden log file – Wayback.sxi. versionfs! version ● Every directory has a hidden catalog – Documents/. versionfs! version ● Logs created as soon as needed ● Logs omitted from directory listing by Wayback ● Logs in binary custom format 12 File Operations ● On every write or Log file truncate, entry Previous appended to log Log Entries ● Data needed to undo ... write or truncate is Timestamp written Write Offset Overwritten Size File Enlarged Flag Overwritten Data 13 Directory Operations ● When files are changed Catalog file in a way that changes Previous their directory entries, Catalog directory catalog is Entries ... updated Timestamp ● Every operation writes Action ID timestamp and action ID ● Action Specific data per Specific operation Data 14 Dir Ops: Create, Mkdir ● Just needs to know name so it can be deleted on revert Length of Name File or Directory Name 15 Dir Ops: Rm ● First, file is truncated to 0 bytes – File version log now has backup ● Record file attributes – Mode, owner, times ● Record filename ● Record destination for symbolic link Attributes Contents: Size of Data Attributes Mode UID GID Filename Access Time Modified Time Create Time [Link dest] 16 Dir Ops: Rmdir, Rename ● Rmdir actually does rename and omits from directory listing – Don't want to lose logs for files in directory ● Record old and new names Length of Names Old and New Names 17 Dir Ops: Chmod, Chown, Utime ● Just need to know the filename and the old attributes Size of Data Attributes Filename 18 Design Decisions ● User vs kernel level – User: Remount any base FS – User: Development easier, more stable – Kernel: Faster ● Undo vs redo log – Undo: Everything always retained as long as current version is not lost – Undo: No overhead of initial file copy – Redo: Files recoverable even if the current version is lost 19 Design Decisions ● FUSE vs other user level modules – It's modern and actively developed – It works with kernels 2.4 and 2.6 – It gives complete but simple access to FS operations – It's written in C (with which we are more familiar) 20 Design Decisions ● Comprehensive versioning vs others – Record every operation that ever happens – Periodic: Choose time-granularity of versioning – Version on close: Assume one logical operation per file open ● Individual log per file vs central log – Individual: No central point of failure – Individual: Simple, logs stay with files – Central: Doesn't consume inodes as fast 21 What's the cost of Wayback? ● 3 performance tests – Bonnie1: Large file reads and writes – Andrew2: Typical use and compilation estimation – RCS: Versioning comparison with RCS ● Tested with ext3, FUSE, and Wayback 1: T. Bray, http://www.textuality.com/bonnie 2: J. Howard, et al, 1988. 22 Test machines ● Tests on 3 machines – Machine A: Normal ● Athlon XP 2400 ● 512 Mb memory ● 2.5 in notebook hard drive – Machine B: Slow disk ● Pentium IV 2.2 Ghz ● 512 Mb memory ● USB 1.1 hard drive – Machine C: Slow processor ● Celeron 500 Mhz ● 128 Mb memory ● 2.5 in notebook hard drive 23 Bonnie: 30-70% max overhead 18 16 14 12 s / B 10 ext3 M FUSE 8 Wayback 6 4 2 0 Write per Write per Rewrite Read per Read per character block character block 24 Modified Andrew Benchmark ● Andrew uses Makefile to run five phases with Sun graphical source code ● Modification uses Perl and Linux wm ● Five phases: – 1: Directory creation (mkdir) – 2: Copying files (cp) – 3: Stating files (ls -l) – 4: Reading files (grep and wc) – 5: Compilation (gcc) 25 Andrew: Compile 15% slower 1100 1000 900 800 ) ms 700 ( Ext3 e 600 FUSE Wayback Tim 500 400 300 200 100 0 mkdir cp ls -l grep, wc gcc / 20 26 RCS Comparison ● 3 traces – Binary write: ● Random small changes with write system call – Binary file: ● Random small changes, but rewrite whole file – Text: ● Random changes to lines in text file ● 11 configurations – Wayback – RCS checkin after every k changes, k=1..10 27 RCS: Wayback Faster 50 45 40 ) 35 (s e 30 Binary write m i Binary file T 25 Text x 10 20 15 10 5 0 Wb 1 2 3 4 5 6 7 8 9 10 28 Future Work ● Log compression – Remove unnecessary granularity ● Garbage collection – Remove old versions ● Limiting – Limit number or size of versions ● Customizable versioning – Filter to only version some files, not others ● Migration of virtual machines – Undo/redo to sync virtual machines 29 Summary ● Don't lose important data! ● RCS, CVS, etc require user action ● Wayback automatically versions – Remount directory with versioning – Edit files as normal – Writes are written as undo information – Easy to revert or extract to any time ● Wayback faster than RCS even excluding user time 30 Get it! It's free! (Gnu General Public License) http://wayback.sourceforge.net 31 RCS Size Comparison (MB) File Type Binary Write Binary File Text Wayback 1157456.4 106347428.0 2182218.0 RCS Period 1 2242325.4 3856521.8 101062.2 RCS Period 2 2237020.7 3779180.4 96134.2 RCS Period 3 2233854.1 3731427.8 94336.4 RCS Period 4 2234384.4 3719578.2 93597.1 RCS Period 5 2233716.4 3700853.4 93095.3 RCS Period 6 2227924.3 3621657.1 92375.5 RCS Period 7 2230300.3 3635107.2 92321.2 RCS Period 8 2227552.2 3590195.5 91960.0 RCS Period 9 2231124.4 3625548.8 92060.8 RCS Period 10 2232218.7 3629717.4 92045.2 32 Slow Disk: Bonnie Performance 1.1 1 0.9 0.8 s 0.7 / B ext3 0.6 M FUSE 0.5 Wayback 0.4 0.3 0.2 0.1 0 Write per Write per Rewrite Read per Read per character block character block 33 Slow Processor: Bonnie Performance 18 16 14 12 s / B 10 ext3 M FUSE 8 Wayback 6 4 2 0 Write per Write per Rewrite Read per Read per character block character block 34 Slow Disk: Andrew Performance 2250 2000 1750 ) s 1500 (m Ext3 e 1250 FUSE m i T Wayback 1000 750 500 250 0 mkdir cp ls -l grep, wc gcc / 20 35 Slow Processor: Andrew Performance 3000 2750 2500 2250 ) 2000 ms ( 1750 Ext3 e FUSE 1500 Tim Wayback 1250 1000 750 500 250 0 mkdir cp ls -l grep, wc gcc / 20 36 Slow Disk: RCS Performance 60 55 50 45 ) (s 40 e Binary write m 35 i Binary file T 30 Text x 10 25 20 15 10 5 0 Wb 1 2 3 4 5 6 7 8 9 10 37 Slow Processor: RCS Performance 110 100 90 80 ) s ( 70 e Binary write 60 Binary file Tim Text x 10 50 40 30 20 10 0 Wb 1 2 3 4 5 6 7 8 9 10 38 .
Load more

Recommended publications
  • Online Layered File System (OLFS): a Layered and Versioned Filesystem and Performance Analysis
    Loyola University Chicago Loyola eCommons Computer Science: Faculty Publications and Other Works Faculty Publications 5-2010 Online Layered File System (OLFS): A Layered and Versioned Filesystem and Performance Analysis Joseph P. Kaylor Konstantin Läufer Loyola University Chicago, [email protected] George K. Thiruvathukal Loyola University Chicago, [email protected] Follow this and additional works at: https://ecommons.luc.edu/cs_facpubs Part of the Computer Sciences Commons Recommended Citation Joe Kaylor, Konstantin Läufer, and George K. Thiruvathukal, Online Layered File System (OLFS): A layered and versioned filesystem and performance analysi, In Proceedings of Electro/Information Technology 2010 (EIT 2010). This Conference Proceeding is brought to you for free and open access by the Faculty Publications at Loyola eCommons. It has been accepted for inclusion in Computer Science: Faculty Publications and Other Works by an authorized administrator of Loyola eCommons. For more information, please contact [email protected]. This work is licensed under a Creative Commons Attribution-Noncommercial-No Derivative Works 3.0 License. Copyright © 2010 Joseph P. Kaylor, Konstantin Läufer, and George K. Thiruvathukal 1 Online Layered File System (OLFS): A Layered and Versioned Filesystem and Performance Analysis Joe Kaylor, Konstantin Läufer, and George K. Thiruvathukal Loyola University Chicago Department of Computer Science Chicago, IL 60640 USA Abstract—We present a novel form of intra-volume directory implement user mode file system frameworks such as FUSE layering with hierarchical, inheritance-like namespace unifica- [16]. tion. While each layer of an OLFS volume constitutes a subvol- Namespace Unification: Unix supports the ability to ume that can be mounted separately in a fan-in configuration, the entire hierarchy is always accessible (online) and fully navigable mount external file systems from external resources or local through any mounted layer.
    [Show full text]
  • Enabling Security in Cloud Storage Slas with Cloudproof
    Enabling Security in Cloud Storage SLAs with CloudProof Raluca Ada Popa Jacob R. Lorch David Molnar Helen J. Wang Li Zhuang MIT Microsoft Research [email protected] {lorch,dmolnar,helenw,lizhuang}@microsoft.com ABSTRACT ers [30]. Even worse, LinkUp (MediaMax), a cloud storage Several cloud storage systems exist today, but none of provider, went out of business after losing 45% of client data them provide security guarantees in their Service Level because of administrator error. Agreements (SLAs). This lack of security support has been Despite the promising experiences of early adopters and a major hurdle for the adoption of cloud services, especially benefits of cloud storage, issues of trust pose a significant for enterprises and cautious consumers. To fix this issue, we barrier to wider adoption. None of today's cloud stor- present CloudProof, a secure storage system specifically de- age services|Amazon's S3, Google's BigTable, HP, Mi- signed for the cloud. In CloudProof, customers can not only crosoft's Azure, Nirvanix CloudNAS, or others|provide se- detect violations of integrity, write-serializability, and fresh- curity guarantees in their Service Level Agreements (SLAs). ness, they can also prove the occurrence of these violations For example, S3's SLA [1] and Azure's SLA [10] only guar- to a third party. This proof-based system is critical to en- antee availability: if availability falls below 99:9%, clients abling security guarantees in SLAs, wherein clients pay for are reimbursed a contractual sum of money. As cloud stor- a desired level of security and are assured they will receive age moves towards a commodity business, security will be a certain compensation in the event of cloud misbehavior.
    [Show full text]
  • VERSIONFS a Versatile and User-Oriented Versioning File System
    VERSIONFS A Versatile and User-Oriented Versioning File System A THESIS PRESENTED BY KIRAN-KUMAR MUNISWAMY-REDDY TO THE GRADUATE SCHOOL IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF MASTER OF SCIENCE IN COMPUTER SCIENCE STONY BROOK UNIVERSITY Technical Report FSL-03-03 December 2003 Abstract of the Thesis Versionfs A Versatile and User-Oriented Versioning File System by Kiran-Kumar Muniswamy-Reddy Master of Science in Computer Science Stony Brook University 2003 File versioning is a useful technique for recording a history of changes. Applications of ver- sioning include backups and disaster recovery, as well as monitoring intruders’ activities. Alas, modern systems do not include an automatic and easy-to-use file versioning system. Existing backup systems are slow and inflexible for users. Even worse, they often lack backups for the most recent day’s activities. Online disk snapshotting systems offer more fine-grained versioning, but still do not record the most recent changes to files. Moreover, existing systems also do not give individual users the flexibility to control versioning policies. We designed a lightweight user-oriented versioning file system called Versionfs. Versionfs works with any file system, whether local or remote, and provides a host of user-configurable policies: versioning by users, groups, processes, or file names and extensions; version retention policies by maximum number of versions kept, age, or total space consumed by versions of a file; version storage policies using full copies, compressed copies, or deltas. Versionfs creates file versions automatically, transparently, and in a file-system portable manner—while maintaining Unix semantics. A set of user-level utilities allow administrators to configure and enforce default policies; users are able to set policies within configured boundaries, as well as view, control, and recover files and their versions.
    [Show full text]
  • Metadata Efficiency in Versioning File Systems
    2nd USENIX Conference on File and Storage Technologies, San Francisco, CA, Mar 31 - Apr 2, 2003. Metadata Efficiency in Versioning File Systems Craig A.N. Soules, Garth R. Goodson, John D. Strunk, Gregory R. Ganger Carnegie Mellon University Abstract of time over which comprehensive versioning is possi- ble. To be effective for intrusion diagnosis and recovery, Versioning file systems retain earlier versions of modi- this duration must be greater than the intrusion detection fied files, allowing recovery from user mistakes or sys- latency (i.e., the time from an intrusion to when it is de- tem corruption. Unfortunately, conventional versioning tected). We refer to the desired duration as the detection systems do not efficiently record large numbers of ver- window. In practice, the duration is limited by the rate sions. In particular, versioned metadata can consume of data change and the space efficiency of the version- as much space as versioned data. This paper examines ing system. The rate of data change is an inherent aspect two space-efficient metadata structures for versioning of a given environment, and an analysis of several real file systems and describes their integration into the Com- environments suggests that detection windows of several prehensive Versioning File System (CVFS), which keeps weeks or more can be achieved with only a 20% cost in all versions of all files. Journal-based metadata encodes storage capacity [41]. each metadata version into a single journal entry; CVFS uses this structure for inodes and indirect blocks, reduc- In a previous paper [41], we described a prototype self- ing the associated space requirements by 80%.
    [Show full text]
  • Replication, History, and Grafting in the Ori File System
    Replication, History, and Grafting in the Ori File System Ali Jose´ Mashtizadeh, Andrea Bittau, Yifeng Frank Huang, David Mazieres` Stanford University Abstract backup, versioning, access from any device, and multi- user sharing—over storage capacity. But is the cloud re- Ori is a file system that manages user data in a modern ally the best place to implement data management fea- setting where users have multiple devices and wish to tures, or could the file system itself directly implement access files everywhere, synchronize data, recover from them better? disk failure, access old versions, and share data. The In terms of technological changes, disk space has in- key to satisfying these needs is keeping and replicating creased dramatically and has outgrown the increase in file system history across devices, which is now prac- wide-area bandwidth. In 1990, a typical desktop machine tical as storage space has outpaced both wide-area net- had a 60 MB hard disk, whose entire contents could tran- work (WAN) bandwidth and the size of managed data. sit a 9,600 baud modem in under 14 hours [2]. Today, Replication provides access to files from multiple de- $120 can buy a 3 TB disk, which requires 278 days to vices. History provides synchronization and offline ac- transfer over a 1 Mbps broadband connection! Clearly, cess. Replication and history together subsume backup cloud-based storage solutions have a tough time keep- by providing snapshots and avoiding any single point of ing up with disk capacity. But capacity is also outpac- failure. In fact, Ori is fully peer-to-peer, offering oppor- ing the size of managed data—i.e., the documents on tunistic synchronization between user devices in close which users actually work (as opposed to large media proximity and ensuring that the file system is usable so files or virtual machine images that would not be stored long as a single replica remains.
    [Show full text]
  • A Distributed File System for Distributed Conferencing System
    A DISTRIBUTED FILE SYSTEM FOR DISTRIBUTED CONFERENCING SYSTEM By PHILIP S. YEAGER A THESIS PRESENTED TO THE GRADUATE SCHOOL OF THE UNIVERSITY OF FLORIDA IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF MASTER OF SCIENCE UNIVERSITY OF FLORIDA 2003 Copyright 2003 by Philip S. Yeager ACKNOWLEDGMENTS I would like to thank Dr. Richard Newman for his help and guidance with this project. I would like to express my gratitude to Dr. Jonathan C.L. Liu and Dr. Beverly Sanders for serving on my committee. I would like to thank Dr. Joseph Wilson for serving as a substitute at my defense. I would also like to thank Vijay Manian and the other DCS group members for their advice and contributions. I thank my parents and friends for their encouragement and support. Finally, I would like to thank Candice Williams for everything she has done. Without these people this work would not have been possible. iii TABLE OF CONTENTS Page ACKNOWLEDGMENTS ................................................................................................. iii LIST OF FIGURES ......................................................................................................... viii ABSTRACT....................................................................................................................... ix CHAPTER 1 INTRODUCTION ........................................................................................................1 1.1 Introduction........................................................................................................1
    [Show full text]
  • How to Copy Files
    How to Copy Files Yang Zhan, The University of North Carolina at Chapel Hill and Huawei; Alexander Conway, Rutgers University; Yizheng Jiao and Nirjhar Mukherjee, The University of North Carolina at Chapel Hill; Ian Groombridge, Pace University; Michael A. Bender, Stony Brook University; Martin Farach-Colton, Rutgers University; William Jannen, Williams College; Rob Johnson, VMWare Research; Donald E. Porter, The University of North Carolina at Chapel Hill; Jun Yuan, Pace University https://www.usenix.org/conference/fast20/presentation/zhan This paper is included in the Proceedings of the 18th USENIX Conference on File and Storage Technologies (FAST ’20) February 25–27, 2020 • Santa Clara, CA, USA 978-1-939133-12-0 Open access to the Proceedings of the 18th USENIX Conference on File and Storage Technologies (FAST ’20) is sponsored by How to Copy Files Yang Zhan Alex Conway Yizheng Jiao Nirjhar Mukherjee UNC Chapel Hill and Huawei Rutgers Univ. UNC Chapel Hill UNC Chapel Hill Ian Groombridge Michael A. Bender Martín Farach-Colton William Jannen Pace Univ. Stony Brook Univ. Rutgers Univ. Williams College Rob Johnson Donald E. Porter Jun Yuan VMware Research UNC Chapel Hill Pace Univ. Abstract Docker make heavy use of logical copies to package and deploy applications [34, 35, 37, 44], and new container cre- Making logical copies, or clones, of files and directories ation typically begins by making a logical copy of a reference is critical to many real-world applications and workflows, directory tree. including backups, virtual machines, and containers. An ideal Duplicating large objects is so prevalent that many file sys- clone implementation meets the following performance goals: tems support logical copies of directory trees without making (1) creating the clone has low latency; (2) reads are fast in full physical copies.
    [Show full text]
  • Abréviations De L'informatique Et De L'électronique/Version Imprimable
    Abréviations de l'informatique et de l'électronique/Version imprimable... https://fr.wikibooks.org/w/index.php?title=Abréviations_de_l'informat... Un livre de Wikilivres. Une version à jour et éditable de ce livre est disponible sur Wikilivres, une bibliothèque de livres pédagogiques, à l'URL : http://fr.wikibooks.org/wiki/Abr%C3%A9viations_de_l%27informatique_et_de_l %27%C3%A9lectronique Vous avez la permission de copier, distribuer et/ou modifier ce document selon les termes de la Licence de documentation libre GNU, version 1.2 ou plus récente publiée par la Free Software Foundation ; sans sections inaltérables, sans texte de première page de couverture et sans Texte de dernière page de couverture. Une copie de cette licence est inclue dans l'annexe nommée « Licence de documentation libre GNU ». A AA : Anti-Aliasing (Anticrènelage) AAA : Authentication Authorization Accounting AAC : Advanced Audio Coding AAD : Analogique-Analogique-Digital ABAP : Allgemeiner Berichtsaufbereitungsprozessor, (en anglais : Advanced Business Application Programming) Langage dans SAP Ack : Acknowledge character (Acquittement) AC97 : Audio Codec '97 (Audio Codec 1997) ACE Access Control Entry, voir Access Control List Adaptive Communication Environment, voir anglais Adaptive Communication Environment ACE ACL : Access Control List 1 sur 36 14/07/2015 00:18 Abréviations de l'informatique et de l'électronique/Version imprimable... https://fr.wikibooks.org/w/index.php?title=Abréviations_de_l'informat... ACPI : advanced configuration and power interface ACRONYM
    [Show full text]
  • A Hybrid Memory Versioning File System
    HMVFS: A Hybrid Memory Versioning File System Shengan Zheng, Linpeng Huang, Hao Liu, Linzhu Wu, Jin Zha Department of Computer Science and Engineering Shanghai Jiao Tong University Email: fvenero1209, lphuang, liuhaosjtu, wulinzhu, [email protected] Abstract—The byte-addressable Non-Volatile Memory (NVM) Versioning, snapshotting and journaling are three well offers fast, fine-grained access to persistent storage, and a known fault tolerance mechanisms used in a large number large volume of recent researches are conducted on devel- of modern file systems. Journaling file systems use journal oping NVM-based in-memory file systems. However, existing to record all the changes and commit these changes after approaches focus on low-overhead access to the memory and rebooting. Versioning file systems allow users to create con- only guarantee the consistency between data and metadata. In sistent on-line backups, roll back corruptions or inadvertent this paper, we address the problem of maintaining consistency changes of files. Some file systems achieve versioning by among continuous snapshots for NVM-based in-memory file keeping a few versions of the changes to single file and systems. We propose a Hybrid Memory Versioning File System directory, others create snapshots to record all the files in (HMVFS) that achieves fault tolerance efficiently and has low the file system at a particular point in time [5]. Snapshotting impact on I/O performance. Our results show that HMVFS file systems provide strong consistency guarantees to users provides better performance on snapshotting compared with for their ability to recover all the data of the file system to a the traditional versioning file systems for many workloads.
    [Show full text]
  • The File System Interface Is an Anachronism
    View metadata, citation and similar papers at core.ac.uk brought to you by CORE provided by Harvard University - DASH The File System Interface is an Anachronism The Harvard community has made this article openly available. Please share how this access benefits you. Your story matters Citation Ellard, Daniel. 2003. The File System Interface is an Anachronism. Harvard Computer Science Group Technical Report TR-15-03. Citable link http://nrs.harvard.edu/urn-3:HUL.InstRepos:25619463 Terms of Use This article was downloaded from Harvard University’s DASH repository, and is made available under the terms and conditions applicable to Other Posted Material, as set forth at http:// nrs.harvard.edu/urn-3:HUL.InstRepos:dash.current.terms-of- use#LAA ✂✁☎✄✝✆✟✞✡✠☛✄✌☞✎✍✑✏✓✒✔✄✖✕ ✗✓✘✙✒✔✄✛✚✢✜✤✣✦✥✧✄★✞☛✏✩✣✪✘ ✫ ✘☎✣✦✥✬✁☎✚✢✭✙✘✮✞☛✏✔✕ ✯✱✰✳✲✦✴✶✵✸✷✑✹✺✷✶✷✻✰✳✼✾✽ ✿✟❀❂❁❄❃✔❅❆❁❈❇❊❉ ❋❂●❊❍✩■✪❏▲❑ ●❊❏❚■ ✵▼✼✟◆❖▼✴✶✵✸✲✦❖▼✵❘◗❙✼ ❑❈❫ ❯❙✰✬✼❲❱✳✰✬✼❳✽❩❨❬✲✦✴❭❱✧✵✸✼✾❪❲✴ ❋ ❍❵❴ ❏ ❑❲❑ ✰ ✼❳✴✶✽❚❛❊✵❝❜✦❞✝✰❝❪❲❪❳✰✳❖❄❡ ❪❢✵ ❪ The File System Interface is an Anachronism Daniel Ellard [email protected] November 18, 2003 Abstract Contemporary file systems implement a set of abstractions and semantics that are suboptimal for many (if not most) purposes. The philosophy of using the simple mechanisms of the file system as the basis for a vast array of higher-level mechanisms leads to inefficient and incorrect implementations. We propose several extensions to the canonical file system model, including explicit support for lock files, indexed files, and resource forks, and the benefit of session semantics for write updates. We also discuss the desirability of application-level file system transactions and file system support for versioning. 1 Introduction The canonical UNIX file system interface is elegant, ubiquitous, and suboptimal. The seductive appeal of such a clean and minimal abstraction is undeniable– and its simplicity and generality has unquestionably contributed to the success of operating systems such as UNIX [14], which in turn made this model pervasive.
    [Show full text]
  • J. Parallel Distrib. Comput. HMVFS: a Versioning File System on DRAM/NVM Hybrid Memory
    J. Parallel Distrib. Comput. 120 (2018) 355–368 Contents lists available at ScienceDirect J. Parallel Distrib. Comput. journal homepage: www.elsevier.com/locate/jpdc HMVFS: A Versioning File System on DRAM/NVM Hybrid Memory Shengan Zheng, Hao Liu, Linpeng Huang *, Yanyan Shen, Yanmin Zhu Shanghai Jiao Tong University, China h i g h l i g h t s • A Hybrid Memory Versioning File System on DRAM/NVM is proposed. • A stratified file system tree maintains the consistency among snapshots. • Snapshots are created automatically, transparently and space-efficiently. • Byte-addressability is utilized to improve the performance of snapshotting. • Snapshot efficiency of HMVFS outperforms existing file systems by 7.9x and 6.6x. article info a b s t r a c t Article history: The byte-addressable Non-Volatile Memory (NVM) offers fast, fine-grained access to persistent storage, Received 9 January 2017 and a large volume of recent researches are conducted on developing NVM-based in-memory file systems. Received in revised form 12 October 2017 However, existing approaches focus on low-overhead access to the memory and only guarantee the Accepted 27 October 2017 consistency between data and metadata. In this paper, we address the problem of maintaining consistency Available online 16 November 2017 among continuous snapshots for NVM-based in-memory file systems. We propose an efficient versioning Keywords: mechanism and implement it in Hybrid Memory Versioning File System (HMVFS), which achieves fault Versioning tolerance efficiently and has low impact on I/O performance. Our results show that HMVFS provides better Checkpoint performance on snapshotting and recovering compared with the traditional versioning file systems for Snapshot many workloads.
    [Show full text]
  • Fist : Alanguagefors Ta Ckablefilesystems
    Proceedings of 2000 USENIX Annual Technical Conference San Diego, California, USA, June 18–23, 2000 F I S T : A L A N G U A G E F O R S TA C K A B L E F I L E S Y S T E M S Erez Zadok and Jason Nieh THE ADVANCED COMPUTING SYSTEMS ASSOCIATION © 2000 by The USENIX Association All Rights Reserved For more information about the USENIX Association: Phone: 1 510 528 8649 FAX: 1 510 548 5738 Email: [email protected] WWW: http://www.usenix.org Rights to individual papers remain with the author or the author's employer. Permission is granted for noncommercial reproduction of the work for educational or research purposes. This copyright notice must be included in the reproduced paper. USENIX acknowledges all trademarks herein. FiST: A Language for Stackable File Systems Erez Zadok and Jason Nieh Computer Science Department, Columbia University ezk,nieh ¡ @cs.columbia.edu Abstract Media Common Avg. Code Size Type File System (C lines) Traditional file system development is difficult. Stack- Hard Disks UFS, FFS, EXT2FS 5,000–20,000 able file systems promise to ease the development of file Network NFS 6,000–30,000 systems by offering a mechanism for incremental develop- CD-ROM HSFS, ISO-9660 3,000–6,000 ment. Unfortunately, existing methods often require writ- Floppy PCFS, MS-DOS 5,000–6,000 ing complex low-level kernel code that is specific to a sin- gle operating system platform and also difficult to port. Table 1: Common Native Unix File Systems and Code Sizes for Each Medium We propose a new language, FiST, to describe stackable file systems.
    [Show full text]