DOCSLIB.ORG

The Effects of Age on File System Performance

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text

Load more

Masaryk University Faculty of Informatics The Effects of Age on File System Performance Bachelor’s Thesis Samuel Petrovič Brno, Spring 2017 Declaration Hereby I declare that this paper is my original authorial work, which I have worked out on my own. All sources, references, and literature used or excerpted during elaboration of this work are properly cited and listed in complete reference to the due source. Samuel Petrovič Advisor: Adam Rambousek i Acknowledgement I would like to thank my consultant Matúš Kocka for tremendous help and guidance during writing of this thesis. I would also like to thank Red Hat for collaboration and provision of necessary testing equipment. iii Abstract The purpose of this thesis is to research effects of age on file system performance. For the testing purpose, two automated tests were im- plemented, which use open source benchmarks fs-drift and FIO to simulate aging and to measure performance. Research was aimed at effects of aging on popular Linux file systems, XFS and ext4. Further research examined effects of underlying storage technology on perfor- mance of aged file systems. Implemented file system aging test was used to expand testing matrix of Red Hat Kernel Performance team. iv Keywords file system, performance, aging, benchmarking, fragmentation, stor- age, trim, fs-drift, XFS, ext4 v Contents 1 Introduction 1 2 Related work 5 2.1 Aging file system using real-life data .............5 2.2 Synthetic aging simulation ..................6 3 File system and storage devices 7 3.1 File systems ..........................7 3.1.1 XFS . .8 3.1.2 Ext4 . .8 3.2 Storage ............................9 3.2.1 Hard disk drive . .9 3.2.2 Solid state drive . .9 4 Environment setup and benchmark tools 11 4.1 Beaker ............................. 11 4.2 Storage generator ....................... 11 4.3 FIO .............................. 12 4.4 Fs-drift ............................ 12 4.4.1 Changes in original code . 13 5 Aging the file system 17 5.1 Fs-drift configuration ..................... 17 5.2 File system images ....................... 18 5.3 Implementation details of file system aging test ....... 19 6 Performance testing of aged file system 23 6.1 FIO configuration ....................... 23 6.2 Implementation details of aged file system performance test . 24 7 Results 27 7.1 Testing environment ..................... 27 7.2 Data processing ........................ 29 7.2.1 Fragmentation . 29 7.2.2 Evaluation of data generated by fs-drift . 30 7.3 Testing on images of aged file system ............. 30 vii 7.4 Performance of aged file system ................ 30 7.5 Differences between XFS and ext4 .............. 33 7.6 Performance accross different storage devices ......... 34 7.7 Testing with fstrim ..................... 37 8 Conclusion 41 Bibliography 43 A Reports 47 A.1 Test of medium utilisation using XFS on HDD ....... 47 A.2 Test of high utilisation using XFS on HDD ......... 51 A.3 Test of high utilisation using ext4 on HDD ......... 55 A.4 Test of XFS on SSD with regular trimming ......... 59 A.5 Test of ext4 on SSD with regular trimming ......... 63 A.6 Test of XFS on SSD without regular trimming ....... 67 B Examples 71 C Source code 75 viii List of Tables 7.1 Comparison of latencies of medium and high utilises file system. 33 7.2 Comparison of latencies gathered during testing with XFS and ext4 on SSD and HDD. 35 7.3 Comparison of latencies of gathered during aging test on HDD and SSD. 36 7.4 Differences in latencies of trimmed and not trimmed file system (XFS). 38 ix List of Figures 4.1 Normal distribution of file access 14 4.2 Moving random distribution 15 4.3 File size distribution as measured by Agrawal. [20] 16 4.4 Log-normal distribution of file size. 16 5.1 Activity diagram of file system aging test 21 6.1 Activity diagram of testing of aged file system. 25 7.1 Evolution of sequential read of XFS during testing of high and medium utilisation of HDD 32 7.2 Evolution of latency of create operation, in untrimmed file system (XFS) 39 A.1 Free space fragmentation of XFS during testing of medium utilisation of HDD 47 A.2 Size distribution of file extents of XFS during testing of medium utilisation of HDD 48 A.3 Usage of available space of XFS during testing of medium utilisation of HDD 49 A.4 Evolution of latencies of XFS during testing of medium utilisation of HDD 50 A.5 Evolution of free space fragmentation of XFS during testing of high utilisation of HDD 51 A.6 Size distribution of file extents of XFS during testing high utilisation of HDD 52 A.7 Usage of available space of XFS during testing of high utilisation of HDD 53 A.8 Evolution of latencies of XFS during testing of high utilisation of HDD 54 A.9 Evolution of free space fragmentation of ext4 during testing of high utilisation of HDD 55 A.10 Size distribution of file extents of ext4 during testing of high utilisation of HDD 56 A.11 Usage of available space of ext4 during testing of high utilisation of HDD 57 A.12 Evolution of latencies of ext4 during testing of high utilisation of HDD 58 xi A.13 Evolution of free space fragmentation of XFS during testing on SSD with regular trimming 59 A.14 Size distribution of file extents of XFS during testing on SSD with regular trimming 60 A.15 Usage of available space of XFS during testing on SSD with regular trimming 61 A.16 Evolution of latencies of XFS during testing on SSD with regular trimming 62 A.17 Evolution of free space fragmentation of ext4 during testing on SSD with regular trimming 63 A.18 Size distribution of file extents of ext4 during testing on SSD with regular trimming 64 A.19 Usage of available space of ext4 during testing on SSD with regular trimming 65 A.20 Evolution of latencies of ext4 during testing on SSD with regular trimming 66 A.21 Evolution of free space fragmentation of XFS during testing on SSD without regular trimming 67 A.22 Size distribution of file extents of XFS during testing on SSD without regular trimming 68 A.23 Usage of available space of XFS during testing on SSD without regular trimming 69 A.24 Evolution of latencies of XFS during testing on SSD without regular trimming 70 xii 1 Introduction File systems remain an important part of modern storage solutions. Large, growing databases, multimedia and other storage based appli- cations need to be supported by high-performing infrastructure layer of storing and retrieving information. Such infrastructure have to be provided by the operating systems (OS) in a form of file system. Originally, file system was a simple tool developed to handle com- munication between OS and physical device, but today, it is a very complex piece of software with a large set of tools and features to go with. Performance testing is an integral part of development cycle of most of produced software. Because of growing complexity of file systems, performance testing took of as an important part of file system evaluation. The standard workflow of performance testing is called out-of- box testing. Its principle is to run benchmark (i.e. testing tool) on a clean instance of OS and on a clean instance of tested file system [1]. Generally, this workflow presents stable and meaningful results, yetit only gives overall idea of file system behavior in early stage of its life cycle. File systems, as well as other complex software, are subjected to progressive degradation, referred to as software aging [2]. Causes of file system aging are many, but mostly fragmentation of free and used space and unclustered data [3]. This degradation cause prob- lems in performance and functionality over time. Understanding of performance changes of aged file system can help developers toim- plement various preventions of aging related problems. Furthermore, aging testing can help developers with implementation of long-term performance affecting features. Researching of file system aging can by done in two stages, aging the file system and testing of the aged instance. In the first stage, observation about evolution of fragmentation and performance can be made. The second stage brings insight into state of performance of aged file system. In the first part, this thesis describes implementation of two flexible tests, which correspond with aforementioned stages. 1 1. Introduction In the first-stage testing, file system is aged using open-source benchmark fs-drift. While aging the file system, fragmentation of free space and latency of IO operations is periodically recorded. After the aging process, additional information about file systems block layout is recorded as well. Once this information is gathered, image of aged file system instance is created. To save space, only metadata of created file system is used, since content of created files israndom and therefore irrelevant. Created images are then used in second-stage testing. By using created images, higher stability and consistency of results is achieved. By reloading the file system image on the device, it is possible to bring the file system to the original, aged, state therefore for every performance test the same file system layout is available. In the second-stage testing, file system is brought to the aged state by reloading image corresponding with desired testing configuration. Before every performance test, some volume can be released from file system, so the performance test has space to test on. After environment initialization (sync, fstrim), the performance test can begin. Testing of fresh instance of file system is done as well to conclude ifany performance change occur. In the latter parts of this thesis, usage of developed tests on different configurations of file systems and storage is demonstrated.
Recommended publications
  • CS 5600 Computer Systems

    CS 5600 Computer Systems

    CS 5600 Computer Systems Lecture 10: File Systems What are We Doing Today? • Last week we talked extensively about hard drives and SSDs – How they work – Performance characterisEcs • This week is all about managing storage – Disks/SSDs offer a blank slate of empty blocks – How do we store files on these devices, and keep track of them? – How do we maintain high performance? – How do we maintain consistency in the face of random crashes? 2 • ParEEons and MounEng • Basics (FAT) • inodes and Blocks (ext) • Block Groups (ext2) • Journaling (ext3) • Extents and B-Trees (ext4) • Log-based File Systems 3 Building the Root File System • One of the first tasks of an OS during bootup is to build the root file system 1. Locate all bootable media – Internal and external hard disks – SSDs – Floppy disks, CDs, DVDs, USB scks 2. Locate all the parEEons on each media – Read MBR(s), extended parEEon tables, etc. 3. Mount one or more parEEons – Makes the file system(s) available for access 4 The Master Boot Record Address Size Descripon Hex Dec. (Bytes) Includes the starEng 0x000 0 Bootstrap code area 446 LBA and length of 0x1BE 446 ParEEon Entry #1 16 the parEEon 0x1CE 462 ParEEon Entry #2 16 0x1DE 478 ParEEon Entry #3 16 0x1EE 494 ParEEon Entry #4 16 0x1FE 510 Magic Number 2 Total: 512 ParEEon 1 ParEEon 2 ParEEon 3 ParEEon 4 MBR (ext3) (swap) (NTFS) (FAT32) Disk 1 ParEEon 1 MBR (NTFS) 5 Disk 2 Extended ParEEons • In some cases, you may want >4 parEEons • Modern OSes support extended parEEons Logical Logical ParEEon 1 ParEEon 2 Ext.
  • W4118: Linux File Systems

    W4118: Linux File Systems

    W4118: Linux file systems Instructor: Junfeng Yang References: Modern Operating Systems (3rd edition), Operating Systems Concepts (8th edition), previous W4118, and OS at MIT, Stanford, and UWisc File systems in Linux Linux Second Extended File System (Ext2) . What is the EXT2 on-disk layout? . What is the EXT2 directory structure? Linux Third Extended File System (Ext3) . What is the file system consistency problem? . How to solve the consistency problem using journaling? Virtual File System (VFS) . What is VFS? . What are the key data structures of Linux VFS? 1 Ext2 “Standard” Linux File System . Was the most commonly used before ext3 came out Uses FFS like layout . Each FS is composed of identical block groups . Allocation is designed to improve locality inodes contain pointers (32 bits) to blocks . Direct, Indirect, Double Indirect, Triple Indirect . Maximum file size: 4.1TB (4K Blocks) . Maximum file system size: 16TB (4K Blocks) On-disk structures defined in include/linux/ext2_fs.h 2 Ext2 Disk Layout Files in the same directory are stored in the same block group Files in different directories are spread among the block groups Picture from Tanenbaum, Modern Operating Systems 3 e, (c) 2008 Prentice-Hall, Inc. All rights reserved. 0-13-6006639 3 Block Addressing in Ext2 Twelve “direct” blocks Data Data BlockData Inode Block Block BLKSIZE/4 Indirect Data Data Blocks BlockData Block Data (BLKSIZE/4)2 Indirect Block Data BlockData Blocks Block Double Block Indirect Indirect Blocks Data Data Data (BLKSIZE/4)3 BlockData Data Indirect Block BlockData Block Block Triple Double Blocks Block Indirect Indirect Data Indirect Data BlockData Blocks Block Block Picture from Tanenbaum, Modern Operating Systems 3 e, (c) 2008 Prentice-Hall, Inc.
  • Ext3 = Ext2 + Journaling

    Ext3 = Ext2 + Journaling

    FS Sistem datoteka-skup metoda i struktura podataka koje operativni sistem koristi za čuvanje podataka Struktura sistema datoteka: - 1. zaglavlje→neophodni podaci za funkcionisanje sistema datoteka - 2. strukture za organizaciju podataka na medijumu→meta podaci - 3. podaci→datoteke i direktorijumi Strukture podataka neophodne za realizaciju sistema datoteka: - PCB(Partition Control Block) - BCB(Boot control Block) - Kontrolne strukture za alokaciju datoteka(i-node tabela kod Linux-a) - Direktorijumske strukture koje sadrže kontrolne blokove datoteka - FCB(File Control Block) ext3 Slide 1 of 51 VIRTUELNI SISTEM DATOTEKA(VFS) Linux podržava rad sa velikim brojem sistema datoteka(ext2,ext3, XFS,FAT, NTFS...) VFS-objektno orjentisani način realizacije sistema datoteka koji omogućava korisniku da na isti način pristupa svim sistemima datoteka Način obraćanja korisnika sistemu datoteka - korisnik->API - VFS->sistem datoteka ext3 Slide 2 of 51 Linux FS Linux posmatra svaki sistem datoteka kao nezavisnu hijerarhijsku strukturu objekata(datoteka i direktorijuma) na čijem se vrhu nalazi root(/) direktorijum Objekti Linux sistema datoteka: Super block - zaglavlje(superblock) - i-node tabela I-Node Table - blokovi sa podacima - direktorijumski blokovi - blokovi indirektnih pokazivača Data Area i-node-opisuje objekte, oko 128B na disku Kompromis između veličine i-node tabele i brzine rada sistema datoteka - prvih 10-12 pokazivača na blokove sa podacima - za alokaciju većih datoteka koristi se single indirection block - za još veće datoteke
  • Silicon Graphics, Inc. Scalable Filesystems XFS & CXFS

    Silicon Graphics, Inc. Scalable Filesystems XFS & CXFS

    Silicon Graphics, Inc. Scalable Filesystems XFS & CXFS Presented by: Yingping Lu January 31, 2007 Outline • XFS Overview •XFS Architecture • XFS Fundamental Data Structure – Extent list –B+Tree – Inode • XFS Filesystem On-Disk Layout • XFS Directory Structure • CXFS: shared file system ||January 31, 2007 Page 2 XFS: A World-Class File System –Scalable • Full 64 bit support • Dynamic allocation of metadata space • Scalable structures and algorithms –Fast • Fast metadata speeds • High bandwidths • High transaction rates –Reliable • Field proven • Log/Journal ||January 31, 2007 Page 3 Scalable –Full 64 bit support • Large Filesystem – 18,446,744,073,709,551,615 = 264-1 = 18 million TB (exabytes) • Large Files – 9,223,372,036,854,775,807 = 263-1 = 9 million TB (exabytes) – Dynamic allocation of metadata space • Inode size configurable, inode space allocated dynamically • Unlimited number of files (constrained by storage space) – Scalable structures and algorithms (B-Trees) • Performance is not an issue with large numbers of files and directories ||January 31, 2007 Page 4 Fast –Fast metadata speeds • B-Trees everywhere (Nearly all lists of metadata information) – Directory contents – Metadata free lists – Extent lists within file – High bandwidths (Storage: RM6700) • 7.32 GB/s on one filesystem (32p Origin2000, 897 FC disks) • >4 GB/s in one file (same Origin, 704 FC disks) • Large extents (4 KB to 4 GB) • Request parallelism (multiple AGs) • Delayed allocation, Read ahead/Write behind – High transaction rates: 92,423 IOPS (Storage: TP9700)
  • Measuring Parameters of the Ext4 File System

    Measuring Parameters of the Ext4 File System

    File System Forensics : Measuring Parameters of the ext4 File System Madhu Ramanathan Venkatesh Karthik Srinivasan Department of Computer Sciences, UW Madison Department of Computer Sciences, UW Madison [email protected] [email protected] Abstract An extent is a group of physically contiguous blocks. Allocating Operating systems are rather complex software systems. The File extents instead of indirect blocks reduces the size of the block map, System component of Operating Systems is defined by a set of pa- thus, aiding the quick retrieval of logical disk block numbers and rameters that impact both the correct functioning as well as the per- also minimizes external fragmentation. An extent is represented in formance of the File System. In order to completely understand and an inode by 96 bits with 48 bits to represent the physical block modify the behavior of the File System, correct measurement of number and 15 bits to represent length. This allows one extent to have a length of 215 blocks. An inode can have at most 4 extents. those parameters and a thorough analysis of the results is manda- 15 tory. In this project, we measure the various key parameters and If the file is fragmented, every extent typically has less than 2 a few interesting properties of the Fourth Extended File System blocks. If the file needs more than four extents, either due to frag- (ext4). The ext4 has become the de facto File System of Linux ker- mentation or due to growth, an extent HTree rooted at the inode is nels 2.6.28 and above and has become the default file system of created.
  • Oracle® Linux 7 Managing File Systems

    Oracle® Linux 7 Managing File Systems

    Oracle® Linux 7 Managing File Systems F32760-07 August 2021 Oracle Legal Notices Copyright © 2020, 2021, Oracle and/or its affiliates. This software and related documentation are provided under a license agreement containing restrictions on use and disclosure and are protected by intellectual property laws. Except as expressly permitted in your license agreement or allowed by law, you may not use, copy, reproduce, translate, broadcast, modify, license, transmit, distribute, exhibit, perform, publish, or display any part, in any form, or by any means. Reverse engineering, disassembly, or decompilation of this software, unless required by law for interoperability, is prohibited. The information contained herein is subject to change without notice and is not warranted to be error-free. If you find any errors, please report them to us in writing. If this is software or related documentation that is delivered to the U.S. Government or anyone licensing it on behalf of the U.S. Government, then the following notice is applicable: U.S. GOVERNMENT END USERS: Oracle programs (including any operating system, integrated software, any programs embedded, installed or activated on delivered hardware, and modifications of such programs) and Oracle computer documentation or other Oracle data delivered to or accessed by U.S. Government end users are "commercial computer software" or "commercial computer software documentation" pursuant to the applicable Federal Acquisition Regulation and agency-specific supplemental regulations. As such, the use, reproduction, duplication, release, display, disclosure, modification, preparation of derivative works, and/or adaptation of i) Oracle programs (including any operating system, integrated software, any programs embedded, installed or activated on delivered hardware, and modifications of such programs), ii) Oracle computer documentation and/or iii) other Oracle data, is subject to the rights and limitations specified in the license contained in the applicable contract.
  • Migrating from Netware to OES 2 Linux

    Migrating from Netware to OES 2 Linux

    Best Practice Guide www.novell.com Migrating from NetWare to OES 2 prepared for Novell OES 2 User Community Published: November, 2007 Disclaimer Novell, Inc. makes no representations or warranties with respect to the contents or use of this document, and specifically disclaims any express or implied warranties of merchantability or fitness for any particular purpose. Trademarks Novell is a registered trademark of Novell, Inc. in the United States and other countries. * All third-party trademarks are property of their respective owner. Copyright 2007 Novell, Inc. All rights reserved. No part of this publication may be reproduced, photocopied, stored on a retrieval system, or transmitted without the express written consent of Novell, Inc. Novell, Inc. 404 Wyman Suite 500 Waltham Massachusetts 02451 USA Prepared By Novell Services and User Community Migrating from NetWare to OES 2—Best Practice Guide November, 2007 Novell OES 2 User Community The latest version of this document, along with other OES 2 Linux Best Practice Guides, can be found with the NetWare to Linux Migration Resources at: http://www.novell.com/products/openenterpriseserver/netwaretolinux/view/all/-9/tle/all Contents Acknowledgments.................................................................................. iv Getting Started...................................................................................... 1 Why OES 2?..............................................................................................1 Which Services Are Right for OES 2? ................................................................4
  • The Third Extended File System with Copy-On-Write

    The Third Extended File System with Copy-On-Write

    Limiting Liability in a Federally Compliant File System Zachary N. J. Peterson The Johns Hopkins University Hopkins Storage Systems Lab, Department of Computer Science Regulatory Requirements z Data Maintenance Acts & Regulations – HIPAA, GISRA, SOX, GLB – 4,000+ State and Federal Laws and Regulations with regards to storage z Audit Trail – creating a “chain of trust” – Files are versioned over time – Authenticated block sharing (copy-on-write) between versions. z Disk Encryption – Privacy and Confidentiality – Non-repudiation Hopkins Storage Systems Lab, Department of Computer Science Secure Deletion in a Regulatory Environment z Desire to limit liability when audited – Records that go out of audit scope do so forever – When a disk is subpoenaed old or irrelevant data are inaccessible z Existing Techniques – Secure overwrite [Gutmann] – File key disposal in disk encrypted systems [Boneh & Lipton] z Existing solutions don’t work well in block- versioning file systems Hopkins Storage Systems Lab, Department of Computer Science Technical Problems z Secure overwriting of noncontiguous data blocks is slow and inefficient – When versions share blocks, data to be overwritten may be noncontiguous z Cannot dispose file keys in a versioning file system – Blocks encrypted with a particular key need to be available in future versions z User space tools are inadequate – Can’t delete metadata – Can’t be interposed between file operations – Truncate may leak data – Difficult to be synchronous Hopkins Storage Systems Lab, Department of Computer Science
  • A Novel Term Weighing Scheme Towards Efficient Crawl Of

    A Novel Term Weighing Scheme Towards Efficient Crawl Of

    International Journal of Computer Engineering and Applications, Volume X, Special Issue, ICRTCST -2016 www.ijcea.com, ISSN 2321-3469 FEATURES AND DIFFERENCES OF LINUX EXT, EXT2, EXT3 AND EXT4 FILE SYSTEMS Sanku Sinha 1, Sadique Nayeem 2 1, 2 Department of Computer Science and Engineering, R.V.S. College of Engineering and Technology, Jamshedpur ABSTRACT: File System is a way used by operating systems to store and organize files so that the files can be accessed efficiently. File system is a component of operating system and it interacts with the abstraction of the underlying hardware known as Block Layer. An open source environment like Linux has over 50 different file systems since its existence. Though, in recent years, the speed of CPUs have been increased and the Hard drive technology have tremendous growth, the file systems have also been evolved so that operating system can provide better performance to the user. To measure the performance of the file systems there are different benchmarks used by different developers. In this work, a study of Linux file system evolution has been carried out. As there are several file systems for different Linux Distributions and different Linux kernel versions, some popular versions of extended file systems like ext, ext2, ext3 and ext4 has been considered. The features and differences on the inode structures of these file systems have been discussed. Keywords: Ext, ext2, ext3, ext4, inode, Linux [1] INTRODUCTION Operating System of a particular system is chosen by a user on the basis of applications to be used or better system performance. For better system performance, the performance of the file system is a key part.
  • Design and Implementation of the Spad Filesystem

    Design and Implementation of the Spad Filesystem

    Charles University in Prague Faculty of Mathematics and Physics DOCTORAL THESIS Mikul´aˇsPatoˇcka Design and Implementation of the Spad Filesystem Department of Software Engineering Advisor: RNDr. Filip Zavoral, Ph.D. Abstract Title: Design and Implementation of the Spad Filesystem Author: Mgr. Mikul´aˇsPatoˇcka email: [email protected]ff.cuni.cz Department: Department of Software Engineering Faculty of Mathematics and Physics Charles University in Prague, Czech Republic Advisor: RNDr. Filip Zavoral, Ph.D. email: Filip.Zavoral@mff.cuni.cz Mailing address (advisor): Dept. of Software Engineering Charles University in Prague Malostransk´en´am. 25 118 00 Prague, Czech Republic WWW: http://artax.karlin.mff.cuni.cz/~mikulas/spadfs/ Abstract: This thesis describes design and implementation of the Spad filesystem. I present my novel method for maintaining filesystem consistency — crash counts. I describe architecture of other filesystems and present my own de- sign decisions in directory management, file allocation information, free space management, block allocation strategy and filesystem checking algorithm. I experimentally evaluate performance of the filesystem. I evaluate performance of the same filesystem on two different operating systems, enabling the reader to make a conclusion on how much the performance of various tasks is affected by operating system and how much by physical layout of data on disk. Keywords: filesystem, operating system, crash counts, extendible hashing, SpadFS Acknowledgments I would like to thank my advisor Filip Zavoral for supporting my work and for reading and making comments on this thesis. I would also like to thank to colleague Leo Galamboˇsfor testing my filesystem on his search engine with 1TB RAID array, which led to fixing some bugs and improving performance.
  • Ntfs, Fat, Fat32, Ext2, Ext3, Ext4)

    Ntfs, Fat, Fat32, Ext2, Ext3, Ext4)

    6 VIII August 2018 International Journal for Research in Applied Science & Engineering Technology (IJRASET) ISSN: 2321-9653; IC Value: 45.98; SJ Impact Factor: 6.887 Volume 6 Issue VIII, August 2018- Available at www.ijraset.com Comparative study of File systems (NTFS, FAT, FAT32, EXT2, EXT3, EXT4) Akash Bundele1, Prof. Dr. S. E. Yedey2 PG Department of Computer Science & Technology, Hanuman Vyayam Prasarak Mandal, Amravati, Maharashta Abstract: Over the years, hard drives and the systems used to store data on them have constantly evolved. There are Windows file systems and Linux file system. And have several advantages and disadvantages. File systems have traditionally been a major area of research and development. This is evident from the existence of over 50 file systems of varying popularity in the current version of the Linux kernel. Windows 2000 supports several file systems, the most important of which are FAT-16, FAT-32, and NTFS (NT File System). This paper looks at various file systems (FAT NTFS EXT2 EXT3 EXT4) and performing comparative study. I. INTRODUCTION In today’s world everything revolves around data. Data is critical for day-to-day operation of any system. Data management is taken care of by file systems which reliably store data on disks. Users typically have varied requirements ranging from scalability, availability, fault-tolerance, performance guarantees in business environment to small memory footprints, security, and reliability in desktop environments. This has driven the file system community to develop a variety of systems that cater to different user requirements. Since they were developed over twenty years ago, the role of personal computers in our lives has drastically increased.
  • Analyzing Metadata Performance in Distributed File Systems

    Analyzing Metadata Performance in Distributed File Systems

    Inaugural-Dissertation zur Erlangung der Doktorwurde¨ der Naturwissenschaftlich-Mathematischen Gesamtfakultat¨ der Ruprecht-Karls-Universitat¨ Heidelberg vorgelegt von Diplom-Informatiker Christoph Biardzki aus Thorn Tag der mundlichen¨ Prufung:¨ 19.1.2009 Analyzing Metadata Performance in Distributed File Systems Gutachter: Prof. Dr. Thomas Ludwig Abstract Distributed file systems are important building blocks in modern computing environments. The challenge of increasing I/O bandwidth to files has been largely resolved by the use of parallel file systems and sufficient hardware. However, determining the best means by which to manage large amounts of metadata, which contains information about files and directories stored in a distributed file system, has proved a more difficult challenge. The objective of this thesis is to analyze the role of metadata and present past and current implementations and access semantics. Understanding the development of the current file system interfaces and functionality is a key to understanding their performance limitations. Based on this analysis, a distributed metadata benchmark termed DMetabench is presented. DMetabench significantly improves on existing benchmarks and allows stress on meta- data operations in a distributed file system in a parallelized manner. Both intra-node and inter-node parallelity, current trends in computer architecture, can be explicitly tested with DMetabench. This is due to the fact that a distributed file system can have different seman- tics inside a client node rather than semantics between multiple nodes. As measurements in larger distributed environments may exhibit performance artifacts difficult to explain by reference to average numbers, DMetabench uses a time-logging tech- nique to record time-related changes in the performance of metadata operations and also protocols additional details of the runtime environment for post-benchmark analysis.