DOCSLIB.ORG

Operating Systems, Assignment 4 File System

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Operating Systems, Assignment 4 File System Ben-Gurion University of the Negev, Operating Systems 2012 OPERATING SYSTEMS, ASSIGNMENT 4 FILE SYSTEM SUBMISSION DATE: 26/06/2012 22:00 In this assignment you are to extend the file system of xv6. xv6 implements a Unix-like file system, and when running on top of QEMU, stores its data on a virtual IDE disk (fs.img) for persistence. To get familiar with the xv6’s file system design and capabilities, it is recommended to read Chapter 5 of the xv6 book. Assignment overview The assignment consists of the following parts: 1. “Hacking” the xv6’s file system. This part includes expanding the maximal file size supported by the file system, and adding support for symbolic links. 2. File tagging (bonus). In this part you are to add support for adding key-value pairs to files. 3. Find application. In this part you are to implement a find application, to search the file system for files matching specified criteria. Task 0: Running xv6 Begin by downloading our revision of xv6 from the os122 svn repository: Open a shell and traverse to the desired working directory. Check-out the project files using svn by calling: svn checkout http://bgu-os-122-xv6-rev6-1.googlecode.com/svn/ass4 Build xv6 by calling: make Run xv6 on top of QEMU by calling: make qemu When working through a remote connection to the department’s computers use: screen make qemu-nox 1 Ben-Gurion University of the Negev, Operating Systems 2012 PART 1: "HACKING" THE XV6’S FILE SYSTEM Xv6, as well as other modern operating systems, supports a fixed maximum file size. Xv6's implementation was inspired by the UNIX operating system. Both operating systems use the i-nodes architecture for their file systems. In xv6, the i-node structure contains pointers to 12 direct data blocks, each of size 512 bytes (totaling to 6KB), as well as a single indirect pointer; the latter points to a block which maps additional 128 blocks. This one level of indirection gives us access to 64KB of additional data. This allows files of size of up to 70KB. Expanding the maximal file size Extend the i-node structure to support files of size up to 8MB, by adding a double indirection layer to the i-nodes. This will require changes to the i-node structure, as well as to the disk representation of i-nodes (dinode). Hint: the file mkfs.c is written using standard C libraries and is built and executed by the makefile outside of xv6 to create the virtual drive, fs.img. One of its tasks is to write the superblock, which contains metadata characterizing the file system. You will have to modify the contents of the superblock to be consistent with the changes you make. The virtual disk, fs.img, should contain at least 215 blocks (totaling to 16MB). Hint: the size of the dinode structure should be a divisor of the block size, i.e. an integer number of dinodes should fit in a block. You may want to add padding for this purpose. Sanity test Write a simple user application which creates a text file of size 1MB and writes a notification message to the screen after writing the first 12 direct blocks, after writing the single indirect blocks, and after writing the double indirect blocks. The purpose of this application is to test your implementation, and in case it contains bugs, help you identify where the bugs are. You are free to change the output or add additional printings as you see fit. The output should look something like: Finished writing 6KB (direct) Finished writing 70KB (single indirect) Finished writing 1MB Adding support for symbolic links Xv6 supports hard links via the user space program ln. Hard links allow different file names to reference the same actual file by using the same i-node number. For example, when a hard link named "b.txt" is created for a file named "a.txt", both "a.txt" and "b.txt" refer to the same file (data) on disk. Changes made to “a.txt” are reflected in “b.txt” and vice versa. Deleting one will not affect the other (will only decrease the link count). In this task you will expand the program ln to support symbolic links, also referred to as soft links. When a new symbolic link is created, it doesn't share the same i-node number as the pointed file (target). Instead, a new file is created and a new i-node number is assigned to it. The contents of the new symbolic link file will contain the path to the target. 2 Ben-Gurion University of the Negev, Operating Systems 2012 Symbolic links are a special type of files, and should be assigned a unique enumeration value in the file type enum. A symbolic link can point to any type of file: regular file, directory and even another symbolic link. A symbolic link can be either absolute or relative. Naturally, moving a relative link to a different location will result in a broken link. The following syntax should be used to create a symbolic link: ln –s old_path new_path Example: ln –s /home/os/a.txt /home/algo/b.txt Will create the new symbolic link /home/algo/b.txt and it will point to the existing file /home/os/a.txt The following system calls should be implemented to support symbolic links: int symlink(cont char *oldpath, const char *newpath); symlink() creates a symbolic link, whose name is specified by the parameter newpath, and which points to the file whose name is specified by the parameter oldpath. The latter file can be of any type, and may not even exist. symlink() returns 0 upon success, and a negative integer upon failure. int readlink (const char *pathname, char *buf, size_t bufsiz); readlink() reads the name of the file to which the symbolic link points. The name of the link is specified by the parameter pathname. The target path is stored in the buffer buf, whose size is specified by bufsiz. readlink() returns the number of bytes which have been placed in the buffer buf, or the value -1 upon failure. readlink() should dereference any symbolic links encountered in the path. Protection from loops Symbolic links to directories may cause infinite loops. We shall tackle this issue in the same manner as it is done in UNIX, by limiting the degree of a chain of links to 16. I.e. when retrieving the target of a link, which points to a link, which points to a link, and so on… The max number of jumps we will allow is 16. Longer chains will be considered as loops. Extending user applications Extend the user applications cat, grep, sh and wc to handle symbolic links, as if it were the target file. I.e. cat some_link should display the contents of the target file, to which the symbolic link some_link points. The deletion of a symbolic link shouldn’t affect the target file, i.e. the user application rm, when applied to a symbolic link, only deletes the link. The shell application, sh, should handle traversal to directories pointed to by symbolic links (cd some_link), as well as execution of programs pointed to by symbolic links. Don’t forget to handle broken links, by displaying an appropriate error message. 3 Ben-Gurion University of the Negev, Operating Systems 2012 PART 2: FILE TAGGING This part is a bonus which weights 15 points. In this part of the assignment, you will add a new feature to the xv6 file system in order to allow users to tag files with arbitrary key-value pairs they define. This tagging feature can be very useful, and exists in many real world applications already. For example, you can add tags to PDF, JPG, MP3 and many other media files. However, these application- specific implementations are ad hoc and must be re-implemented for new applications. You'll provide a general implementation in the xv6 file system so that it applies to all file types. Hint: You can extend the i-node structure to include a pointer to a block which will contain the key-value pairs. You'll need to implement the following three system calls: int ftag(int fd, char *key, char *val); System call ftag() tags a file identified by file descriptor fd with a new key-value pair, with the key being key and value val. Argument key is a null-terminated string with a maximum length of 10 and minimum length of 2, counting the ending null byte. Argument val specifies the value in the key-value pair, and is a null-terminated string as well, with a maximum length of 30 and minimum length of 2, counting the ending null byte. If key already exists in file fd's tags, this system call overwrites the old tag with the new one. int funtag(int fd, char *key); System call funtag() removes a tag with key key from the file fd. If there is no key-value pair matching key, funtag() should return an error code. int gettag(int fd, char *key, char *buf); System call gettag() retrieves the value mapped to key from the file fd. The retrieved value is stored in buf. gettag() should return the length of the value if it exists and -1 otherwise. Implementation notes: To implement these system calls, you'll need to change the on-disk representation of an i-node and add code to fs.c. Depending on your design, you may need to change mkfs.c so that the layout of the new file system created by mkfs is consistent with the file system layout expected by fs.c.
Load more

Recommended publications
  • Configuring UNIX-Specific Settings: Creating Symbolic Links : Snap
    Configuring UNIX-specific settings: Creating symbolic links Snap Creator Framework NetApp September 23, 2021 This PDF was generated from https://docs.netapp.com/us-en/snap-creator- framework/installation/task_creating_symbolic_links_for_domino_plug_in_on_linux_and_solaris_hosts.ht ml on September 23, 2021. Always check docs.netapp.com for the latest. Table of Contents Configuring UNIX-specific settings: Creating symbolic links . 1 Creating symbolic links for the Domino plug-in on Linux and Solaris hosts. 1 Creating symbolic links for the Domino plug-in on AIX hosts. 2 Configuring UNIX-specific settings: Creating symbolic links If you are going to install the Snap Creator Agent on a UNIX operating system (AIX, Linux, and Solaris), for the IBM Domino plug-in to work properly, three symbolic links (symlinks) must be created to link to Domino’s shared object files. Installation procedures vary slightly depending on the operating system. Refer to the appropriate procedure for your operating system. Domino does not support the HP-UX operating system. Creating symbolic links for the Domino plug-in on Linux and Solaris hosts You need to perform this procedure if you want to create symbolic links for the Domino plug-in on Linux and Solaris hosts. You should not copy and paste commands directly from this document; errors (such as incorrectly transferred characters caused by line breaks and hard returns) might result. Copy and paste the commands into a text editor, verify the commands, and then enter them in the CLI console. The paths provided in the following steps refer to the 32-bit systems; 64-bit systems must create simlinks to /usr/lib64 instead of /usr/lib.
    [Show full text]
  • Where Do You Want to Go Today? Escalating
    Where Do You Want to Go Today? ∗ Escalating Privileges by Pathname Manipulation Suresh Chari Shai Halevi Wietse Venema IBM T.J. Watson Research Center, Hawthorne, New York, USA Abstract 1. Introduction We analyze filename-based privilege escalation attacks, In this work we take another look at the problem of where an attacker creates filesystem links, thereby “trick- privilege escalation via manipulation of filesystem names. ing” a victim program into opening unintended files. Historically, attention has focused on attacks against priv- We develop primitives for a POSIX environment, provid- ileged processes that open files in directories that are ing assurance that files in “safe directories” (such as writable by an attacker. One classical example is email /etc/passwd) cannot be opened by looking up a file by delivery in the UNIX environment (e.g., [9]). Here, an “unsafe pathname” (such as a pathname that resolves the mail-delivery directory (e.g., /var/mail) is often through a symbolic link in a world-writable directory). In group or world writable. An adversarial user may use today's UNIX systems, solutions to this problem are typ- its write permission to create a hard link or symlink at ically built into (some) applications and use application- /var/mail/root that resolves to /etc/passwd. A specific knowledge about (un)safety of certain directories. simple-minded mail-delivery program that appends mail to In contrast, we seek solutions that can be implemented in the file /var/mail/root can have disastrous implica- the filesystem itself (or a library on top of it), thus providing tions for system security.
    [Show full text]
  • File Permissions Do Not Restrict Root
    Filesystem Security 1 General Principles • Files and folders are managed • A file handle provides an by the operating system opaque identifier for a • Applications, including shells, file/folder access files through an API • File operations • Access control entry (ACE) – Open file: returns file handle – Allow/deny a certain type of – Read/write/execute file access to a file/folder by – Close file: invalidates file user/group handle • Access control list (ACL) • Hierarchical file organization – Collection of ACEs for a – Tree (Windows) file/folder – DAG (Linux) 2 Discretionary Access Control (DAC) • Users can protect what they own – The owner may grant access to others – The owner may define the type of access (read/write/execute) given to others • DAC is the standard model used in operating systems • Mandatory Access Control (MAC) – Alternative model not covered in this lecture – Multiple levels of security for users and documents – Read down and write up principles 3 Closed vs. Open Policy Closed policy Open Policy – Also called “default secure” • Deny Tom read access to “foo” • Give Tom read access to “foo” • Deny Bob r/w access to “bar” • Give Bob r/w access to “bar • Tom: I would like to read “foo” • Tom: I would like to read “foo” – Access denied – Access allowed • Tom: I would like to read “bar” • Tom: I would like to read “bar” – Access allowed – Access denied 4 Closed Policy with Negative Authorizations and Deny Priority • Give Tom r/w access to “bar” • Deny Tom write access to “bar” • Tom: I would like to read “bar” – Access
    [Show full text]
  • Answers to Even-Numbered Exercises
    4 Answers to Even-numbered Exercises 1. 2. List the commands you can use to perform these operations: a. Make your home directory the working directory b. Identify the working directory a. cd; b. pwd 3. 4. The df utility displays all mounted filesystems along with information about each. Use the df utility with the –h (human-readable) option to answer the following questions. $ df -h Filesystem Size Used Avail Capacity Mounted on /dev/disk2s10 20G 2.6G 17G 13% / devfs 114K 114K 0B 100% /dev fdesc 1.0K 1.0K 0B 100% /dev <volfs> 512K 512K 0B 100% /.vol /dev/disk0s9 77G 37G 39G 49% /Volumes/Scratch /dev/disk1s9 25G 16G 9.5G 63% /Volumes/Sys /dev/disk2s12 94G 43M 94G 0% /Volumes/New /dev/disk1s10 86G 71G 15G 83% /Volumes/Home automount -nsl [223] 0B 0B 0B 100% /Network automount -fstab [232] 0B 0B 0B 100% /automount/Servers automount -static [232] 0B 0B 0B 100% /automount/static a. How many filesystems are mounted on your Mac OS X system? b. Which filesystem stores your home directory? c. Assuming that your answer to exercise 4a is two or more, attempt to create a hard link to a file on another filesystem. What error message do you get? What happens when you attempt to create a symbolic link to the file instead? 1 2 Answers to Even-numbered Exercises Following are sample answers to these questions. Your answers will be different because your filesystem is different. a. five; b. /dev/disk2s10; c. ln: xxx: Cross-device link. No problem creating a cross-device symbolic link.
    [Show full text]
  • File System (Interface)
    File System (Interface) Dave Eckhardt [email protected] 1 Synchronization Today Chapter 11, File system interface Not: remote/distributed (11.5.2!!) Don't forget about Chapter 13 Reviewing might help demystify readline() some “Fourth Wave” of readings posted to web site 2 Synchronization Two interesting papers about disks http://www.seagate.com/content/docs/pdf/whitepaper/ D2c_More_than_Interface_ATA_vs_SCSI_042003.p df Google for “200 ways to revive a hard drive” 3 What's a file? Abstraction of persistent storage Hide details of storage devices sector addressing: CHS vs. LBA SCSI vs. IDE Logical grouping of data May be physically scattered Programs, data Some internal structure 4 Typical file attributes Name – 14? 8.3? 255? Unicode? ASCII? 6-bit? RADIX-50? Identifier - “file number” Type (or not) Location – device, location Size – real or otherwise Protection – Who can do what? Time, date, last modifier – monitoring, curiousity 5 “Extended” file attributes BSD Unix archived nodump append-only (user/system) immutable (user/system) MacOS icon color 6 Operations on Files Create – locate space, enter into directory Write, Read – according to position pointer/cursor Seek – adjust position pointer Delete – remove from directory, release space Truncate Trim data from end Often all of it Append, Rename 7 Open-file State Expensive to specify name for each read()/write() String-based operation Directory look-up “Open-file” structure stores File-system / partition File-system-relative file number Read vs. write Cursor position 8 Open files (Unix Model) “In-core” / “Open file” file state Mirror of on-disk structure File number, size, permissions, modification time, ... Housekeeping info Back pointer to containing file system #readers, #writers Most-recently-read block How to access file (vector of methods) Pointer to file's type-specific data Shared when file is opened multiple times 9 Open files (Unix Model) “File-open” state (result of one open() call) Access mode (read vs.
    [Show full text]
  • Mac OS X Server File Services Administration for Version 10.4 Or Later
    Mac OS X Server File Services Administration For Version 10.4 or Later K Apple Computer, Inc. © 2005 Apple Computer, Inc. All rights reserved. The owner or authorized user of a valid copy of Mac OS X Server software may reproduce this publication for the purpose of learning to use such software. No part of this publication may be reproduced or transmitted for commercial purposes, such as selling copies of this publication or for providing paid-for support services. Every effort has been made to ensure that the information in this manual is accurate. Apple Computer, Inc., is not responsible for printing or clerical errors. Apple 1 Infinite Loop Cupertino CA 95014-2084 www.apple.com The Apple logo is a trademark of Apple Computer, Inc., registered in the U.S. and other countries. Use of the “keyboard” Apple logo (Option-Shift-K) for commercial purposes without the prior written consent of Apple may constitute trademark infringement and unfair competition in violation of federal and state laws. Apple, the Apple logo, AppleShare, AppleTalk, Mac, Macintosh, QuickTime, Xgrid, and Xserve are trademarks of Apple Computer, Inc., registered in the U.S. and other countries. Finder is a trademark of Apple Computer, Inc. Adobe and PostScript are trademarks of Adobe Systems Incorporated. UNIX is a registered trademark in the United States and other countries, licensed exclusively through X/Open Company, Ltd. Other company and product names mentioned herein are trademarks of their respective companies. Mention of third-party products is for informational purposes only and constitutes neither an endorsement nor a recommendation.
    [Show full text]
  • Process Need Outline File Systems
    1/26/2016 Motivation – Process Need • Processes store, retrieve information • When process terminates, memory lost Distributed Computing Systems • How to make it persist? • What if multiple processes want to share? File Systems • Requirements: – large Solution? Files – persistent are large, – concurrent access persistent! Motivation – Disk Functionality (1 of 2) Motivation – Disk Functionality (2 of 2) • Questions that quickly arise – How do you find information? – How to map blocks to files? bs – boot sector sb – super block – How do you keep one user from reading another’s data? – How do you know which blocks are free? Solution? File Systems • Sequence of fixed-size blocks • Support reading and writing of blocks Outline File Systems • Abstraction to disk (convenience) • Files (next) – “The only thing friendly about a disk is that it has • Directories persistent storage.” – Devices may be different: tape, USB, SSD, IDE/SCSI, • Disk space management NFS • Misc • Users • Example file systems – don’t care about implementation details – care about interface • OS – cares about implementation (efficiency and robustness) 1 1/26/2016 File System Concepts Files: The User’s Point of View • Files - store the data • Naming: how does user refer to it? • Directories - organize files • Does case matter? Example: blah , BLAH , Blah – Users often don’t distinguish, and in much of Internet no • Partitions - separate collections of directories (also difference (e.g., domain name), but sometimes (e.g., URL called “volumes”) path) – all directory information
    [Show full text]
  • Today: Distributed File Systems File System Basics
    Today: Distributed File Systems • Overview of stand-alone (UNIX) file systems • Issues in distributed file systems • Next two classes: case studies of distributed file systems • NFS • Coda • xFS • Log-structured file systems (time permitting) • HDFS; object storage systems Computer Science CS677: Distributed OS Lecture 19, page !1 File System Basics • File: named collection of logically related data – Unix file: an uninterpreted sequence of bytes • File system: – Provides a logical view of data and storage functions – User-friendly interface – Provides facility to create, modify, organize, and delete files – Provides sharing among users in a controlled manner – Provides protection Computer Science CS677: Distributed OS Lecture 19, page !2 Unix File System Review • User file: linear array of bytes. No records, no file types • Directory: special file not directly writable by user • File structure: directed acyclic graph [directories may not be shared, files may be shared (why?) ] • Directory entry for each file – File name – inode number – Major device number – Minor device number • All inodes are stored at a special location on disk [super block] – Inodes store file attributes and a multi-level index that has a list of disk block locations for the file Computer Science CS677: Distributed OS Lecture 19, page !3 Inode Structure • Fields – Mode – Owner_ID, group_id – Dir_file – Protection bits – Last access time, last write time, last inode time – Size, no of blocks – Ref_cnt – Address[0], … address[14] • Multi-level index: 12 direct blocks,
    [Show full text]
  • Z/OS UNIX Shared File System Environment and How It Works
    z/OS Basics: z/OS UNIX Shared File System environment and how it works Jim Showalter IBM March 1, 2011 Session 9024 1 Trademarks The following are trademarks of the International Business Machines Corporation in the United States and/or other countries. • DFS • RACF • DFSMS • RMF • DFSMSdss • S/390 • IBM • z/OS • MVS • zSeries * Registered trademarks of IBM Corporation The following are trademarks or registered trademarks of other companies. Java and all Java-related trademarks and logos are trademarks of Sun Microsystems, Inc., in the United States and other countries. Linux is a registered trademark of Linus Torvalds in the United States, other countries, or both. Microsoft, Windows and Windows NT are registered trademarks of Microsoft Corporation. UNIX is a registered trademark of The Open Group in the United States and other countries. SET and Secure Electronic Transaction are trademarks owned by SET Secure Electronic Transaction LLC. * All other products may be trademarks or registered trademarks of their respective companies. Notes : Performance is in Internal Throughput Rate (ITR) ratio based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput that any user will experience will vary depending upon considerations such as the amount of multiprogramming in the user's job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve throughput improvements equivalent to the performance ratios stated here. IBM hardware products are manufactured from new parts, or new and serviceable used parts. Regardless, our warranty terms apply. All customer examples cited or described in this presentation are presented as illustrations of the manner in which some customers have used IBM products and the results they may have achieved.
    [Show full text]
  • A Formally Proved, Complete Algorithm for Path Resolution with Symbolic Links
    A Formally Proved, Complete Algorithm for Path Resolution with Symbolic Links Ran Chen Institute of Software, Chinese Academy of Science, Beijing, China Martin Clochard LRI (CNRS & Univ. Paris-Sud), Universit´eParis-Saclay, F-91405 Orsay and Claude March´e Inria, Universit´eParis-Saclay, F-91120 Palaiseau In the context of file systems like those of Unix, path resolution is the operation that given a character string denoting an access path, determines the target object (a file, a directory, etc.) designated by this path. This operation is not trivial because of the presence of symbolic links. Indeed, the presence of such links may induce infinite loops in the resolution process. We consider a path resolution algorithm that always terminates, detecting if it enters an infinite loop and reports a resolution failure in such a case. We propose a formal specification of path resolution and we formally prove that our algorithm terminates on any input, and is correct and complete with respect to our formal specification. 1. INTRODUCTION The problem of path resolution takes place in the context of the file system compo- nent of operating systems. It is the operation that, given a pathname, determines the target object (typically a file or a directory) it denotes in the current file system, if any. In particular for the operating systems of the Unix family, target objects can also be symbolic links: objects that themselves denote a pathname. When meet- ing a symbolic link, path resolution must proceed with resolution of the pathname denoted by that link. The presence of symbolic links gives to the path resolution process a recursive nature, that may lead to non-termination if caution is not taken.
    [Show full text]
  • The Second Extended File System Internal Layout
    The Second Extended File System Internal Layout Dave Poirier <[email protected]> The Second Extended File System: Internal Layout by Dave Poirier Copyright © 2001-2019 Dave Poirier Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.1 or any later version published by the Free Software Foundation; with no Invariant Sections, with no Front-Cover Texts, and with no Back-Cover Texts. A copy of the license can be acquired electronically from http://www.fsf.org/licenses/fdl.html or by writing to 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA Table of Contents About this book ............................................................................................................... viii 1. Historical Background ...................................................................................................... 1 2. Definitions ..................................................................................................................... 2 Blocks ....................................................................................................................... 2 Block Groups ............................................................................................................. 3 Directories ................................................................................................................. 3 Inodes ....................................................................................................................... 3
    [Show full text]
  • Operating Systems
    Operating Systems Project #4: Writing Files + Improved Shell Project #4: Writing Files + Improved Shell Objective Background Getting Started Writing a Disk Sector Deleting a File Writing a File Improvements Shell Command: delete <file> Shell Command: copy <src> <dest> Shell Command: dir Bonus Features Submission Drew University Washington and Lee University Grading Acknowledgement Objective In this project you will implement functions for deleting and writing files, and add several new commands to your shell. At the end of the project, you will have a fully functional single-process operating system about as powerful as CP/M (an early PC operating system. See: http://en.wikipedia.org/wiki/CP/M). ​ Background Like reading files, writing files requires that you understand the file system and how it keeps track of the names of the files on the disk and their locations. As such, there is no new background for this project. However, you may want to review the material on the Disk Directory and the Disk Map given in the Background section of Project 3. Getting Started There are no new files to be downloaded for this project. Make a copy of your project3 ​ directory as project4. ​ ​ Writing a Disk Sector The functionality for reading files was based on being able to read a sector. Similarly, the functionality for writing files will be based on being able to write a sector. Create a new function named writeSector in your kernel with the prototype: ​ ​ int writeSector(char *buffer, int sector); Writing sectors can be accomplished using the same BIOS call as reading sectors. The only difference is that AH should equal 3 instead of 2.
    [Show full text]