CS 162 Project 3: File Systems
Total Page:16
File Type:pdf, Size:1020Kb
CS 162 Project 3: File Systems Design Document Due: Wednesday, April 17, 2019 Code Due: Tuesday, May 7, 2019 Final Report Due: Thursday, May 9, 2019 Contents 1 Your OS 2 2 Your task 2 2.1 Task 1: Buffer cache . .3 2.2 Task 2: Extensible files . .3 2.3 Task 3: Subdirectories . .3 2.4 Synchronization requirement . .3 3 Deliverables 5 3.1 Design Document (Due Wednesday, April 17, 2019) and Design Review . .5 3.1.1 Design Document Guidelines . .5 3.1.2 Topics for your design document . .6 3.1.3 Design Document Additional Questions . .6 3.1.4 Design Review . .7 3.1.5 Grading . .7 3.2 Code (Due Tuesday, May 7, 2019) . .7 3.2.1 Student Testing Code (Due Tuesday, May 7, 2019) . .7 3.3 Final Report (Due Thursday, May 9, 2019) and Code Quality . .8 3.3.1 Testing Report Due Thursday, May 9, 2019 . .8 4 Reference 9 4.1 Getting Started . .9 4.2 Source Files . .9 4.3 Testing File System Persistence . .9 4.4 Requirements . 10 4.4.1 Buffer Cache . 10 4.4.2 Indexed and Extensible Files . 10 4.4.3 Subdirectories . 11 4.4.4 System Calls . 12 4.5 Pintos user program tests . 13 4.6 How to add tests to Pintos . 13 4.7 Suggested Order of Implementation . 14 4.8 FAQ................................................ 14 1 CS 162 Spring 2019 Project 3: File Systems 1 Your OS At this stage, your operating system is pretty powerful | it can run any user process implemented within Pintos' narrow syscall interface. This means you can run a shell, compiler, Minesweeper, or any other user program your heart desires! However, one big caveat is that we have not implemented a syscall to allocate more memory for either our heap or stack. If you wish to go the extra mile, you can implement this upon completion of project 3. Anyway, luckily for us, we don't even have to write these user programs because they are bundled for us in the src/examples directory. You'll see that we get the following programs: • shell • insult (look at the source • rm • cat code if you wish to see what • mkdir • cmp it does) • and more... • cp • echo • hex-dump • ls Before we dive into project 3 you'll see how we can run these programs on your operating system. We encourage you to take advantage of these in order to do manual testing of your filesystem. To begin, please compile all of the examples by running make in src/examples and in src/userprog. Then run this script in src/userprog/build (the script can be downloaded here). PROGRAMS="bubsort n cat n cmp n cp n echo n hex−dump n i n s u l t n l s n mkdir n pwd n rm n s h e l l" EXAMPLES="../../ examples" CMDLINE="pintos −−f i l e s y s −s i z e=100" CMDLINE END="−f −q run'shell'" f o r PROGRAM in $PROGRAMS; do CMDLINE+=" −p $EXAMPLES/$PROGRAM −a $PROGRAM" done CMDLINE+=" −− $CMDLINE END" echo $CMDLINE e v a l $CMDLINE run.sh The script will install the example binaries in the root directory and start Pintos with the provided shell. Most of these will not work correctly until you finish this project. Some, like insult or hex-dump, should work if proj2 was implemented correctly. Try them out and see what does and doesn't work. Try to understand why. 2 Your task In this project, you will add 3 new features to the Pintos file system. A brief summary of the tasks is provided here. A more detailed explanation can be found in section 4.4. 2 CS 162 Spring 2019 Project 3: File Systems Important: This project requires a working proj2. If you aren't passing all (or nearly all) of the test in userprog, you should fix those first. If they can't be easily fixed, talk to your TA about getting a reference solution. 2.1 Task 1: Buffer cache The functions inode_read_at() and inode_write_at() currently access the filesystem’s underlying block device directly, each time you call them. Your task is to add a buffer cache for the filesystem, to improve the performance of reads and writes. Your buffer cache will cache individual disk blocks, so that (1) you can respond to reads with cached data and (2) you can coalesce multiple writes into a single disk operation. The buffer cache should have a maximum capacity of 64 disk blocks. You may choose a block replacement policy, but it must be at least as good as the clock algorithm. The buffer cache must be a write-back cache, not a write-through cache. You must make sure that ALL disk operations use your buffer cache, not just the two inode functions mentioned earlier. 2.2 Task 2: Extensible files Pintos currently cannot extend the size of files, because the Pintos filesystem allocates each file as a single contiguous set of blocks. Your task is to modify the Pintos filesystem to support extending files. You may want to use an indexed inode structure with direct, indirect, and doubly-indirect pointers, like the Unix file system does. The maximum file size you need to support is 8MiB (223 bytes). You must also add support for a new syscall, \inumber(int fd)", which returns the unique inode number of file associated with a particular file descriptor. 2.3 Task 3: Subdirectories The current Pintos filesystem supports directories, but user programs have no way of using them (files can only be placed in the root directory right now). You must add the following syscalls to allow user programs to manipulate directories: chdir, mkdir, readdir, and isdir. You must also update the following syscalls so that they work with directories: open, close, exec, remove, and inumber. You must also add support for relative paths for any syscall with a file path argument. For example, if a process calls chdir("my_files/") and then open("notes.txt"), you should search for notes.txt relative to the current directory and open the file my_files/notes.txt. You also need to support absolute paths like open("/my_files/notes.txt"). You need to support the special \." and \.." names, when they appear in file path arguments, such as open("../logs/foo.txt"). Child processes should inherit the parent's current working directory. The first user process should have the root directory as its current working directory. 2.4 Synchronization requirement Your project code should always be thread-safe, but for Project 3, we don't want you to just use a single global lock around the entire filesystem. As you plan your design for this project, you must ensure that 2 operations acting on different disk sectors can run simultaneously. If you are writing to the same sector or extending a file, you should serialize those operations to maintain data consistency. Concurrent reads are not required. Some examples: 1. read(open("/my_files/notes.txt")) and write(open("/my_tests/test.c")) should be al- lowed to run concurrently, since they operate on two different files that are stored on different sectors. 3 CS 162 Spring 2019 Project 3: File Systems 2. read(open("/my_files/notes.txt")) and write(open("/my_files/notes.txt")) should not be allowed to run concurrently, since they operate on the same sector (note that open starts at the beginning of the file, so they operate on sector 0). 3. read(open("/my_files/notes.txt")) and read(open("/my_files/notes.txt")) may or may not be allowed to run concurrently, since they read from the same sector. This requirement applies to all 3 tasks: the buffer cache, extensible files, and subdirectories. Note: If you added a global file system lock in Project 2, remember to remove it! 4 CS 162 Spring 2019 Project 3: File Systems 3 Deliverables Your project grade will be made up of 4 components: • 20% Design Document and Design Review • 55% Code • 15% Student Testing • 10% Final Report and Code Quality 3.1 Design Document (Due Wednesday, April 17, 2019) and Design Review Before you start writing any code for your project, you should create an implementation plan for each feature and convince yourself that your design is correct. For this project, you must submit a design document and attend a design review with your project TA. 3.1.1 Design Document Guidelines Write your design document inside the doc/project3.md file, which has already been placed in your group's GitHub repository. You must use GitHub Flavored Markdown1 to format your design document. You can preview your design document on GitHub's web interface by going to the following address: (replace group0 with your group number) https://github.com/Berkeley-CS162/group0/blob/master/doc/project3.md For each of the 3 tasks of this project, you must explain the following 4 aspects of your proposed design. We suggest you create a section for each of the 3 project parts. Then, create subsections for each of these 4 aspects. 1. Data structures and functions { Write down any struct definitions, global (or static) variables, typedefs, or enumerations that you will be adding or modifying (if it already exists). These definitions should be written with the C programming language, not with pseudocode. Include a brief explanation the purpose of each modification. Your explanations should be as concise as possible. Leave the full explanation to the following sections. 2. Algorithms { This is where you tell us how your code will work.